Please take a look at those slides. They’ll give you the gist of Mongrel2’s design and I’ll elaborate on them below. Notice that the slides start by speaking about Mongrel, which is Mongrel2’s predecessor. It is fundamentally different from Mongrel2, but it does share some things, like the HTTP parser, in common.
I am somewhat enamored by Mongrel2 lately. It’s design is simple and focused, yet the server does everything you need.
Mongrel2 is the web server. So your browser, a phone or any app might go to yoursite.com and communicate with a Mongrel2 instance. Mongrel2, however, might communicate with something else to fulfill the request. By nature of using a message passing system, it becomes asynchronous and decoupled from how the work is actually being done. Something big sites all scale into eventually anyway.
By using the ZeroMQ framework for it’s message passing it also gains language independence. I could have a Python process sharing load with a Java process alongside some Lua. Any language that supports ZeroMQ can work. Mongrel2 just does web serving and leaves the rest up to you, via configuration.
It gets better too. ZeroMQ is an extensive framework all in itself. You can open a connection to a computer that doesn’t exist and start sending messages. It will queue them up and wait for when that system comes online. You could have it simply error if you think that’s appropriate.
One of my favorite things to do is turn on Mongrel2, request a URL from my browser, then turn on one of my Python request handlers. After it’s initialized, it reads the message and responds. Mongrel2 sends the response to my browser. ZeroMQ absorbed the down handler until it was ready.
It gets a little better, still. So now I’ll open a second request handler, to pretend I’m scaling up to handle load. Mongrel2, with no config change or restart, immediately starts sending messages round-robin to each handler instance!
Mongrel2 is configured by mapping three types of request methods: Dir, Proxy, Handler. You configure URL routes to each of these, depending on your needs. The configuration looks like below.
# here's a sample directory
media_directory = Dir(base='media/',
# a sample proxy route
tornado_proxy = Proxy(addr='127.0.0.1', port=8000)
# a sample zeromq handler
chat_demo = Handler(send_spec='ipc://127.0.0.1:9999',
Three types of request
If it’s a static file request M2 uses Dir handling. It reads the path and expects to find a static file there or follows some common web handling conventions, such as loading index.html. Useful for hosting media.
Proxy handling is similar to the upstream concept from Nginx. You can simply pass the request and it’s details up to something else, like Tornado or Django, for handling the request.
The Handler type is where Mongrel2 really shows it’s power. This type sends the request down a socket for handling and waits to receive responses on another socket. Communication can happen in both directions using different semantics. The handler can exist basically anywhere. It might consume a local socket, like you see configured above with the ipc socket, or it might use a tcp socket to connect to a remote system. It’s configurable and flexible.
There have been a few interesting ideas for how to handle Mongrel2 sockets. You can build a handler using whatever language inspires you.
MongrEvent is a proof-of-concept for writing a Mongrel2 message handler in Python and splitting the processing into a pipeline of coroutines.
Mongrel2 is a web server by Zed Shaw. Mongrel2 handles everything having to do with HTTP and has facilities for passing request handling to external services using ZeroMQ. MongrEvent is an example of how that service could be written to use Eventlet for coroutines. We also get non-blocking I/O very cheaply.
The Mongrel2 handling code is based on Zed’s python in mongrel2’s source.
Mongrel2 is configured to only answer on /echo/, or http://localhost:6767/echo/. Mongrel2 receives the reqeuest and forwards it down tcp://127.0.0.1:9999, where handling.py is listening. This socket is a ZeroMQ PUSH socket.
Read about the different socket types ZeroMQ offers in the manual.
When handling.py is initialized, it sets up an input and output zmq socket for mongrel2. It’s my personal preference to remember input and output instead of PUSH and PUB (publish), the actual zeromq socket types used.
A coroutine is started to handle each request. The assumption is that a routing system could live in this coroutine. Once a route is determined, a subsequent coroutine is created to handle the request. A third coroutine is then created to handle the result of that request, like a postprocessing system.
The third coroutine responds to Mongrel2 by sending it’s response on handling.py's PUB socket. Mongrel2 picks up the message and then responds, completing the request.
If you’re new to Mongrel2, you should read Zed’s documentation on mongrel2.org. The rest of this doc assumes you’ve installed it successfully.
The gist of it is that we have one host that answers to /echo/. Requests to /echo/ are configured to go to our handler by way of tcp://127.0.0.1:9999. It is also listening on tcp://127.0.0.1:9998 to receive answers for our handler.
You should see mongrel2 register a PUSH socket and a SUB socket as it’s last lines of output.
I have stored the essential parts of my Python environment in a requirements file. I was not able to install pyzmq with pip and have it honor zeromq being installed in /opt/local so I install that separately.
I am starting my morning with a documentary about Fermi Lab called Atom Smashers, by PBS.
While describing the purpose of a particle accelerator, the narrator said something that stuck out. He said they smash two quarks, each made up of 6 main components. These quarks smash into each other and break apart into roughly 50-100 different pieces.
Now, for what stuck out: the scientists are using particles so small that the explosion’s debris is actually manageable to track. 50-100 pieces doesn’t seem impossible, assuming the equipment to observe subatomic particles is also not impossible.