December 11, 2009 § Leave a comment
As of late November 2009 – December 2009, It appears that Reddit is using Tornado web server. Interesting.
You can get the same type of information by using curl (curl -I http://www.reddit.com)
September 20, 2009 § Leave a comment
First of all, there are several wiki posts at pylonshq.com already, below are some of them:
I believe there’s not a single best solution in choosing any of these strategies. This post is intended as sharing my experience in some of these. I’m definitely not set on particular one and will change my mind once I gained better understanding.
My testing methodology
I’m currently testing multiple deployment configurations on the same app, thus giving me the opportunity to blog about it. I use Apache Benchmark (on client side), top and ps afx on server side. My machine is 1 Linode 1440 instance.
My AB setup are:
ab -n 500 -c 50 -k http://rootapp.com/ ab -n 1200 -c 50 -k http://rootapp.com/ ab -n 1500 -c 50 -k http://rootapp.com/ ab -n 800 -c 800 -k http://rootapp.com/
The results varied insignificantly with some subtle interesting differences. I’ll explain those below.
Note: I am testing these configuration on dynamic AJAX-y web application, thus reporting hard numbers is not very useful.
CherryPy vs Paste HTTP Server behind NGINX
One thing that I noticed immediately is that CherryPy has better performance than Paste’s HTTP server. On both, having multiple processes does not help much on overall performance, but significantly reduces number of failed requests. When run under multiple processes, CherryPy consistently have the least number of failed requests.
For my setup having 7-8 processes is the sweet spot. When I have more than that, top is telling me that the latter processes are under utilized.
Setting up CherryPy on your production.ini is painless:
use = egg:PasteScript#cherrypy numthreads = 20 request_queue_size = 512 host = %(http_host)s port = %(http_port)s
By just comparing the two, CherryPy is easily the winner.
Lighttpd and SCGI
This gist is basic configuration to get SCGI up and running on lighttpd while the following is setup for your production.ini:
use = egg:Flup#scgi_thread host = %(http_host)s port = %(http_port)s
given the same AB configuration as CherryPy and Paste counterpart, lighttpd and SCGI consistently capable of handling 30 requests/seconds. About 8-10 requests/seconds more than CherryPy. Even though this setup is better, I noticed that memory consumption continues to go up after 2 weeks. I haven’t spend much time in investigating why. The reason I didn’t choose this path is more because I simply like NGINX better.
If only SCGI module on NGINX isn’t so experimental.
NGINX and FastCGI
This gist is basic configuration to get FastCGI running on NGINX while the following is setup for your production.ini:
use = egg:Flup#fcgi_thread host = %(http_host)s port = %(http_port)s
With this configuration, I consistently get about 25 requests/seconds. It’s a bit behind lighttpd and SCGI configuration. Interestingly, when run under ab -n 1500 -c 50 -k, this configuration hangs NGINX requiring it to be restarted. It only happen once though.
Again, when load balanced properly (depending on your app), any one of these configurations would work well. Hopefully this post can help others to get up to speed in Pylons deployment.
September 6, 2009 § Leave a comment
Someone dear to me put together this not too long ago:
pink, lightpink, palevioletred, hotpink, deeppink, red, tomato, crimson, firebrick, indianred
darkred, maroon, brown, sienna, saddlebrown, rosybrown, tan, darkkhaki, BurlyWood, chocolate, peru, darkgoldenrod
lightcoral, coral, lightsalmon, salmon, darksalmon,
orangered, darkorange, orange, sandybrown, goldenrod, khaki, gold, yellow
greenyellow, lightgreen, lawngreen, chartreuse, lime, springgreen, mediumspringgreen,
limegreen, green, forestgreen, darkgreen,
seagreen, mediumseagreen, darkseagreen,
olive, olivedrab, darkolivegreen
azure, aliceblue, lightcyan, paleturquoise, lightblue, lightsteelblue, powderblue, cyan, aqua, aquamarine, turquoise, lightskyblue, skyblue, mediumaquamarine,
mediumturquoise, darkturquoise, deepskyblue, cadetblue, cornflowerblue, steelblue, slateblue, mediumslateblue,royalblue, dodgerblue, lightseagreen, teal,darkcyan, blue, mediumblue,darkslateblue, navy, darkblue, midnightblue
indigo, blueviolet, mediumpurple, mediumorchid, purple, darkmagenta, darkviolet, darkorchid, lavender, thistle, plum, violet, orchid, magenta, fuchsia, mediumvioletred
darkslategray, dimgray, gray, slategray, darkgray, silver
August 25, 2009 § Leave a comment
First of all, I love Tokyo and already use Tokyo as secondary database for Pylons development and so far it has been a great success.
Since my box does not have a lot of memory, and a lot more disk space, it make sense to use Tokyo as caching solution instead of memcache.
Quick googling revealed that Jack Hsu has already implemented beaker’s Tokyo extension. His snippet works out of the box.
For my use case, I change the serializing strategy to using pickle instead of json. My reasoning is that pickle allows serializing complex object and I don’t have requirement for portability on cache data.
You can find the Tokyo extension here. I added Redis extension as well since it is very similar to Tokyo.
Edit (08/26/2009): Added extension for Dynomite
Edit (08/26/2009): Added extension for Ringo
[fixing bad layout]