Scaling Django
• Mark Eschbach
The bread and butter server application I support is a Django application. We’re in a growth spurt right now. This is great news for everyone. Definitely bittersweet for those of us with the wrenches in the trenches.
As with any machine we’re seeing pipes burst. Triaging issues as they arise. Being proven wrong. This is not only a great oppertunity for the organziation to grow but also the team and our expertise of the platform.
For a while now I’ve been trying to crack the nut of getting Django to scale in a reasonable way. I thought there would be great gains to be had if I could get us out of using the Gunicorn’s sync worker setup to using _greenlet_s. And there was great throughput to be had. However whenver I deploy these to our pre-production environments the connection times go from the standard 10-20ms up to 40-100ms.
I’ve crawled through the psycopg2 code. I’ve even crawled through the psycogreen code. But to no avail. I’m wondering if the libraries are missing a critical signal somewhere in the stack the socket connected. Eventually I’ll get back to this problem. And hopefully shortly.