December 11, 2009 § Leave a comment
As of late November 2009 – December 2009, It appears that Reddit is using Tornado web server. Interesting.
You can get the same type of information by using curl (curl -I http://www.reddit.com)
September 20, 2009 § Leave a comment
First of all, there are several wiki posts at pylonshq.com already, below are some of them:
I believe there’s not a single best solution in choosing any of these strategies. This post is intended as sharing my experience in some of these. I’m definitely not set on particular one and will change my mind once I gained better understanding.
My testing methodology
I’m currently testing multiple deployment configurations on the same app, thus giving me the opportunity to blog about it. I use Apache Benchmark (on client side), top and ps afx on server side. My machine is 1 Linode 1440 instance.
My AB setup are:
ab -n 500 -c 50 -k http://rootapp.com/ ab -n 1200 -c 50 -k http://rootapp.com/ ab -n 1500 -c 50 -k http://rootapp.com/ ab -n 800 -c 800 -k http://rootapp.com/
The results varied insignificantly with some subtle interesting differences. I’ll explain those below.
Note: I am testing these configuration on dynamic AJAX-y web application, thus reporting hard numbers is not very useful.
CherryPy vs Paste HTTP Server behind NGINX
One thing that I noticed immediately is that CherryPy has better performance than Paste’s HTTP server. On both, having multiple processes does not help much on overall performance, but significantly reduces number of failed requests. When run under multiple processes, CherryPy consistently have the least number of failed requests.
For my setup having 7-8 processes is the sweet spot. When I have more than that, top is telling me that the latter processes are under utilized.
Setting up CherryPy on your production.ini is painless:
use = egg:PasteScript#cherrypy numthreads = 20 request_queue_size = 512 host = %(http_host)s port = %(http_port)s
By just comparing the two, CherryPy is easily the winner.
Lighttpd and SCGI
This gist is basic configuration to get SCGI up and running on lighttpd while the following is setup for your production.ini:
use = egg:Flup#scgi_thread host = %(http_host)s port = %(http_port)s
given the same AB configuration as CherryPy and Paste counterpart, lighttpd and SCGI consistently capable of handling 30 requests/seconds. About 8-10 requests/seconds more than CherryPy. Even though this setup is better, I noticed that memory consumption continues to go up after 2 weeks. I haven’t spend much time in investigating why. The reason I didn’t choose this path is more because I simply like NGINX better.
If only SCGI module on NGINX isn’t so experimental.
NGINX and FastCGI
This gist is basic configuration to get FastCGI running on NGINX while the following is setup for your production.ini:
use = egg:Flup#fcgi_thread host = %(http_host)s port = %(http_port)s
With this configuration, I consistently get about 25 requests/seconds. It’s a bit behind lighttpd and SCGI configuration. Interestingly, when run under ab -n 1500 -c 50 -k, this configuration hangs NGINX requiring it to be restarted. It only happen once though.
Again, when load balanced properly (depending on your app), any one of these configurations would work well. Hopefully this post can help others to get up to speed in Pylons deployment.
August 25, 2009 § Leave a comment
First of all, I love Tokyo and already use Tokyo as secondary database for Pylons development and so far it has been a great success.
Since my box does not have a lot of memory, and a lot more disk space, it make sense to use Tokyo as caching solution instead of memcache.
Quick googling revealed that Jack Hsu has already implemented beaker’s Tokyo extension. His snippet works out of the box.
For my use case, I change the serializing strategy to using pickle instead of json. My reasoning is that pickle allows serializing complex object and I don’t have requirement for portability on cache data.
You can find the Tokyo extension here. I added Redis extension as well since it is very similar to Tokyo.
Edit (08/26/2009): Added extension for Dynomite
Edit (08/26/2009): Added extension for Ringo
[fixing bad layout]
July 14, 2009 § Leave a comment
In order to have Beaker’s session applied to all subdomains, set cookie_domain to: .yourdomain.com (notice the dot in front).
In pylons, the cookie domain is available as: beaker.session.cookie_domain.
Also, in Google Group, Cezary mentioned to set also set sub_domain in routing.py
- This howto won’t work in Firefox. As far as I know, only Safari allows me to do this.
June 7, 2009 § Leave a comment
It’s nice that Pylons include nosetest and TestUnit as part of its testing framework.
Unfortunately, using those for unit testing inside Pylons is not trivial, not obvious, and inconvenient.
- I can only run nosetests on top-level directory because config is expecting ‘test.ini’ file. That’s inconvenient.
- It is not obvious how build sqlalchemy’s engine inside tests/__init__.py. Ideally, I can build the engine based on test.ini configurations.
The __init__ file solves various problems:
- I can run nosetests anywhere inside project’s directory tree.
- With simple convention, __init__ will look for test.ini file on top-level directory.
- SqlAlchemy engine is built using configuration set in test.ini
I hope this snippet can help readers in getting up to speed in Pylons even quicker.
May 31, 2009 § 1 Comment
For the longest time I kept staring at ‘plain text mode’ while editing Mako templates. No Mas!
TextMate bundle for Mako template has existed.
cd ~/Library/Application\ Support/TextMate/Bundles/
svn co http://svn.makotemplates.org/contrib/textmate/Mako.tmbundle
After Reloading your bundle, syntax highlighting is available under HTML (Mako).