Bringing TCP sockets to Heroku    Posted:

I've been working on a new add-on, called Ruppell's Sockets, that will bring TCP sockets to Heroku and other PaaS providers.

Why TCP sockets? Heroku is a great tool. It makes deploying and running a web app so simple. I've run test apps on it, I ran the Surgeon Simulator ARG phone component on it. If your app wants HTTP (and recently, websockets), Heroku's got your back.

But what about TCP? What if your next big idea needs an IRC or Jabber server? What if you'd like to role your own TCP app? Heroku doesn't have your back.... yet.

Ruppell's Sockets routes a dedicated frontend TCP URI through a tunnel and onto your Heroku dyno, where your app can be listening for connections on a local TCP socket. The frontend TCP URI allocated to your app and the backend tunnel connections are distributed across a cluster of servers to keep everything scalable and reliable, just like your app on Heroku.

Ruppell's Sockets are currently in alpha on Heroku. If you're using Heroku and reading this, and you've got an idea that could do with some TCP sockets, send a mail to and I'll add you to the alpha group. Once you're in the alpha the Heroku Ruppell's Sockets documentation can get you started. Feel free to fork, update it with your experiences (so long as they're relevant for a wide audience) and send me a pull request.

Once I have feedback from Heroku and it's users I'll port the addon to other popular PaaS providers and offer a standalone version that can be used with any PaaS, so if you're interested drop me a mail or keep an eye on my blog.

ARM could be the future    Posted:

Around May last year I bought a Pandaboard to test out an idea I had. What if I could run a cloud platform on ARM severs?

It would be awesome, I could put loads of ARM boards in a rack mount and have an insane core density along with relatively low power and cooling requirements. I could use Arduinos to monitor temperatures and humidities and do useful stuff with that information. Because the price per unit would be so low, hardware failures would cost a lot less. I would either use LXC to slice it up or maybe simply hand out an entire ARM server per instance.

There's questions to be answered tho. Kernel SamePage Merging can, in some cases, make VMs more efficient than bare metal. Could I be smart about the underlying file-systems for the containers and get a similar memory benefit on LXC? If I give each cloud instance an entire board would the loss of shared memory in some cases cost me a lot more than I expect?

I got as far as getting OpenStack running, once I'd fixed a eventlet issue, and it was glorious, but 1GB of RAM just isn't enough to load things up to the point where I could answer my questions, and anyway building my dream in all it's glory would cost millions of £s.

Not long after all this the great guys at TryStack built my dream using some new and amazing Calxeda hardware. It's exciting times for ARM hardware and projects like HP Moonshot mean we'll be seeing a lot more of it in data centers.

I may not have my own data center to play with just yet but having OpenStack running on my Pandaboard last year has me very excited at the possibilities. Especially considering the board has no moving parts and the power source for my tests was USB. Amazing.

Load Testing and PyPy Smoking the Competition    Posted:

Before I put any system I expect to run under load into production I want to know how much load it can handle. Without this knowledge how will I plan for scaling, how will I know where the weak points are, what kind of load will it put on my datastore? There's a lot more to consider, like disk access, CPU usage patterns, network IO, but lets keep it simple for now.

Once I'm happy with the results of my load test I can use the base line load figure for monitoring performance of, scaling and developing that system. If someone changes something and that baseline changes more than I expect I know to take a closer look with them.

I recently load tested an API I'd written using Cyclone on Twisted. It was simple, less than 5 endpoints. Programmatically it was a basic bridge between clients and Redis with minimal authorization and data manipulation. I was very disappointed with the results, managing only about 600 r/s on an m1.small on EC2. Because it was so simple I decided to spend an hour re-implementing it in Node.js, and for completeness in Go and Scala on the Play framework. Node, Go and Play performed very similarly, achieving around 1.2k r/s, almost double what Twisted was doing.

I was sad. I love Twisted. I used to just like it but since I discovered inline callbacks I've loved it. For a while I wanted to marry it and have children with it. Inline callbacks elegantly avoid the callback hell asynchronous frameworks can often land you in and make your code look more sequential. But I couldn't justify running a high traffic API at almost half the speed of Node just for pretty code.

I posted my benchmark to the Twisted mailing list on a Sunday and almost immediately got replies with questions about my system. All answers alluded to the fact that if I was using CPython instead of PyPy I was probably doing it wrong. They were so right.

I was hoping PyPy would allow Twisted to match the other frameworks but for my specific use case it smoked them. As it turns out it's very fast. The same load test that yields 1.2k r/s on Node.js, Go and Play got me only a few hundred shy of 2k r/s with Twisted on PyPy. I'm no mathemetician but that's a 60% performance gain, which happens to translate to a 60% cost saving.

Further to this my load test highlighted an easy optimisation for my app. I cached some objects in a distributed Redis cache we've started to use internally (more on that in another post), and we were using pickle to serialise and de-serialize some objects. I discovered that on PyPy Cyclone's json serializer was as quick as pickle. Of course that's not an optimisation because it's the same speed, but I realised something obvious. Where possible I could store the cached data in JSON which I could send to the client without de-serializing and re-serializing into JSON, making the difference of a few hundred requests a second. I know it sounds obvious but sometimes you need to see what a difference a small change can make to truly appreciate it. Good times.

This is of course not to say Twisted on PyPy is a universal solution that will always result in the fastest apps. It's not and it won't. This is why it's important to understand use cases for various frameworks, do benchmarking and use the most appropriate tool for the job. But in this case PyPy kicked ass.

Devoxx 2013    Posted:

I'm giving a Cassandra for beginners talk at Devoxx London this year with Kim, a good friend if mine. It's my first time giving a talk like this so fingers crossed it's useful for the peeps who'll be listening to us.

As promised in the presentation I'm throwing up some most useful Cassandra related links for your viewing pleasure.

Some shall pass    Posted:

Update for 13 May 2013
Bossa Studios is using Some Shall Pass for an internal project and so allocated me dev time to work on it.
It's a lot more feature complete now. Aside from some work on the admin interface and better error checking (which is comming soon) it's properly usable.

A while ago I watched a great Facebook tech talk. The thing that stuck with me was their feature service they called Gatekeeper, described far better than I could in this TechCrunch article.

Gatekeeper allows Facebook to very easily AB test features, but impressively to change who's in A and who's in B very quickly. They can test a feature on 1000 users for an hour and then drop it back down to only their team members.

Some shall pass is my attempt to write a similar service. It's missing a lot of essential secret sauce and a well defined API, both of which I'll try to add over the coming weeks, but in the mean time feel free to check out some shall pass on Bitbucket.

In it's un-scalable state it'll handle about 3k HTTP requests/sec, one of the many reasons I'm loving Twisted. I suspect the line receiver protocol will do a significant amount more than that. Once I write in the Redis scaling the sky should be the limit.

Turn Python packages into debs for package management goodness    Posted:

I like keeping a tidy house, except in my actual house. I also like Cyclone but there's no package in Ubuntu for it.

When there's a Python package available but no deb, install python-stdeb download the package, extract it and use stdeb to convert it.

python --command-packages=stdeb.command bdist_deb

Hello World    Posted:

There are two reasons I decided to start this blog.

  1. Over the past year a good friend of mine and I have helped to take Monstermind, a Bafta award winning Facebook game, from an outage a week to close to no downtime, at the same time adding 2 more games to the family. I'd like to share what I've learned throughout this process, along with a few other random nuggets of tech I've enjoyed playing with.
  2. I got wicked awesome business cards printed with this web address on them :)

I'm using Pelican (except not any more) to manage this blog because it's written in Python and because it's the first one I found that was under active development.

I'll try make a point of picking a monthly topic to output here, but no promises.