New release - a first crack at scaling issues


Hi all,

As many of you know, we’ve been having some scaling issues in the last week or so. This new release is our first crack at addressing them.

Here’s what we’ve managed:

  • running scheduled tasks on a different server type from consoles
  • implement more async / batch job processing of, eg, quota updates using beanstalk
  • optimise our console launchers and file browser views to remove blocking calls to cgroups binaries (these seemed to be real processor hogs)
  • more load balancing in uwsgi, including max limits on memory usage (1/2 a gig per worker) and request time (10 minutes)

It’s been a fairly hectic couple of weeks! We’re very conscious of unacceptable performance, since we use PythonAnywhere every day - in fact, there’s a bit of a Catch-22, since PythonAnywhere being slow slows down our development of fixes to that slowness… Anyways, we’ve got there in the end, and we look forward to seeing how those improvements pan out. We’re keeping a close eye on things for the next few days!

Lots of stuff still to do - we’re half-way through some more major speedups to the file browser, and we’ve got lots more planned. And, of course, we’re looking forward to getting back to work on the multiple-web-apps / Pro plans, which are tantalisingly 90% finished – so we’ve only got the remaining 90% to do ;-)

comments powered by Disqus