WP-Web- and http-performance sucks - despite APCu, ZendO,...

[Martin_Sauer]

Situation:
- 10 WP sites and a well-sized server (still running at 1-10% CPU and 25% memory on peak times; normally at 15% memory and 1-5% CPU load)
- using Plesk 12.0.18 with nginx/Apache sandwich
- installed, configured and activated most current ZendOptimizer (set php 5.6 as default-version)
- installed, configured and activated most current APCu
- WP-sites running (with loads of testing) on well-configured W3 Total Cache
- the initial html-file is highly dynamic

Observation:
1. Initial html-file (first of 60+ files for initial page) has an unreal wait-time of 1.8 seconds. Upon reload within a few minutes (flushed browser cache aka full reload of the site), the wait time drops to 0.2 seconds.
2. all files after the initial html file are normally fast (send, wait, load,...), no matter the size.

Questions:
A. What can/should I do to improve performance? (yes, am aware a general question). Basically much faster load time on reload with full content hints at either php or mysql. How can I pinpoint the issue and or are there general recommendations?
B. Maybe just using varnish or squid as front end helps? Since the html file is highly dynamic generated on the fly (responsive akl adapting to each device type AND browser) I am unsure if varnish/squid would be of any use?
C. any additional advice?

TIA

 

[Martin_Sauer]

Advice would be kinda cool

 

[Jose_Antonio_Morales]

Maybe too much optimizations?
Zend Optimizer + Apc + W3 Total cache can be maybe too much optimizations... Sometimes this mixes generates conflicts...
2 secs are not a WP load that sucks... if you are loading a page with many pics for the first time on a browser.
It also depends on how much KB ( or MB maybe ) is loading per page.
If W3 total cache generates static files... nginx should deliver it alone... not a single call to Apache or php...
You must review your configuration on the panel... and be sure that static files are delivered by nginx, this is a huge change on server and memory load, as nginx liberates calling apache for static files...

 

[Martin_Sauer]

Maybe too much optimizations?
Zend Optimizer + Apc + W3 Total cache can be maybe too much optimizations... Sometimes this mixes generates conflicts...
2 secs are not a WP load that sucks... if you are loading a page with many pics for the first time on a browser.
It also depends on how much KB ( or MB maybe ) is loading per page.
If W3 total cache generates static files... nginx should deliver it alone... not a single call to Apache or php...
You must review your configuration on the panel... and be sure that static files are delivered by nginx, this is a huge change on server and memory load, as nginx liberates calling apache for static files...

Yep. Thank you for helping me work out the underlying issue. In detail:

1. nginx delivers static files - fast - verified.
2. Optimization works together (its APCu, not APC due to ZendOpCache included in php from 5.5 on) - verified and tested.
3. What remains are highly dynamic files, which are different for almost each browser and client as well. Those are two files: index.php and an ajax file. All other files are served (lightning) fast. For those index.php and main.css we got quite some php-code and mysql queries running. Since its fast on reload, apparently ZendOpCache and APCu together got it cached. Yet those two caches are not made for long time storage, more in the timeframe of 1 min - 1 hour. So I either have to improve on mysql and php speed or cache the result for longer.

According to 3. any advice would be cool on:

• varnish - wouldnt varnish or squid solve all issues easily? Yet varnish or squid cant handle cookies, meaning Id have to delete most cookies. Any advice/idea/experience?
• mysql - any advice on wordpress-specific mysql optimization? I am a newb to mysql.
• php - same as mysql - any advice on optimization? I am a newb to php speed optimization.
• Any advice on how to further pinpoint the issue?

TIA
Martin

 

[Jose_Antonio_Morales]

Ajax runs on the client, not on the server... so you need to focus on the php part.

About tuning Mysql I have used in the past http://mysqltuner.com/ and eventually helped me to understand and resolve some issues at caching queries

Its an script in perl, you need to let the server run for at least 48 hours before running this in order to get some real picture of the queries.
send me a message to my inbox with the url and I will check it out

 

[Martin_Sauer]

Ajax runs on the client, not on the server... so you need to focus on the php part.

About tuning Mysql I have used in the past http://mysqltuner.com/ and eventually helped me to understand and resolve some issues at caching queries

Its an script in perl, you need to let the server run for at least 48 hours before running this in order to get some real picture of the queries.
send me a message to my inbox with the url and I will check it out

Its more difficult

1. the Ajax file is generated on the server side and uses up ressources (to be generated) - I can verify that by eg. pingdom, showing me the wait time.
   
2. got mysqltuner - tbh... its a good start but apparently far from perfect

Right now I try to solve it this way:

• running my WP-sites under nginx with php-fpm on php5.6
• diving into mariadb config: (i) caching/optimization, (ii) indexes for WP - unfortunately there is not much info on the web about it
• tuning APCu and ZendOpCache

Right now looking for a stresstest script to further calculate my tests, so I can compare results on each step.

 

[Upgrade]

_Hey news for optimization?

 

[Martin_Sauer]

_Hey news for optimization?

Yupp, I did find my almost perfect optimization

Sorry, if I dont share exact values - here a start:
- install google pagespeed and recompile nginx
- configure google pagespeed (hint: set disk-cache to a tmpfs)
- let nginx not only serve static files, but also php via php-fpm
- update your OS php (your default php - thats important) to 5.6.1. Why 5.6.1? Some aspects of memory consumption and handling improved a lot
- install and configure Zend OpCache and APCu (not APC, but APCu)
- if using Wordpress, configure W3TC (sorry you gotta test a lot to find a sweet spot for your system and theme)

Sites with low response of 4+ seconds dropped to 700ms and less.

So yes - am content so far

Next step: if I got free time, I will recompile Apache and nginx with icc, maybe also the mariadb and the kernel. Here I do assume another speed increase of the server itself by app. 50%. Consireding network latency, in above 700ms example I do believe to achieve some solid 500ms response with icc. Apart from that: pagespeed compiled with icc offers a better and more reliable memory management, so it also saves system ressources.

Some additional current stats:
- I configured huge cache sizes (ZendOpcache for atm 12+ WP-sites = 1GB)
- based on tests with "ab" my system can handle app. 14 million http-connections a day (with app 150KB each index file)
- greatest impact on my sites for now are the time to receive and DNS/connection issues. Wait times on the server are minimal
- Other huge websites of customers went from 4-7ms down to 1.33ms
- overall the responsiveness of websites, esspecially of complex sites, drastically increased
- for the first time I am confident to offer WP even for high traffic sites myself

 

[Martin_Sauer]

PS: and before I forget - Apache as mod_php (only real drawback atm in my config)

 

[JueJueBean]

I just sent you a private message about this Martin, can you answer? Thanks

 

[waseem]

@Martin_Sauer do you offer server customization??

 

[cookkoo]

Hello, I've already install memcached , Zend Opcache but i can not choose it on W3 Total Cache. Please help. Thank you