[Cialug] Last night's excitement when Reddit melted our server

[Nicolai]

On Fri, Apr 19, 2013 at 02:14:38PM -0500, Matthew Nuzum wrote:

> http://www.bearfruit.org/2013/04/19/reddit-is-melting-our-server-heres-what-we-did-nginx-apache-django-and-mysql/
> 
> The takeaway is that, for best results, more static is more good. :-)

Totally... can't be said enough!

> Hindsight is alway 20/20, but if you have any insight you'd like to
> share, I'd love to hear it. Feel free to either reply here or on the
> post if you think all of the web would like to read it.

This is a huge subject but I'll mention two things.

First, Google has an excellent public resource for web developers that
they currently call PageSpeed Insights.

https://developers.google.com/speed/pagespeed/insights

It's really helpful, and lists several improvements that could be made
to the server in question.  Try a bunch of sites for comparison!

Second, Google (again!  thanks google) recently released zopfli:

 "Zopfli Compression Algorithm is a new zlib (gzip, deflate)
  compatible compressor. This compressor takes more time
  (~100x slower), but compresses around 5% better than zlib
  and better than any other zlib-compatible compressor we have
  found."

https://code.google.com/p/zopfli/

I use it for webpages and find about a 4% improvement over "gzip -9".
Just a comparison of /etc/services copied to ~ for quick typing:

$ ls -l services* | awk '{print $5, $9}'|sort -rn 
10423 services
4161 services.gz
4090 services.bz2
4070 services.7z
4017 services.zopfli.gz

Here I used -9 or -mx=9 depending on the tool for maximum compression.

Nicolai