J2 Labs

Instantiating AWS Micro Instances with Erlang

After writing a lot of Python code recently, I’ve been wanting to work in another language for a few days. It’s a bit like a mental vacation, if you pick the right language to play with.

I chose the black sand beaches of Erlang.

About The Code

The idea is to simply connect to Amazon’s API and request some number of AWS micro instances be instantiated. Most of Amazon’s API is implemented for EC2, S3, MTurk, SQS and SimpleDB in the Erlcloud module.

Erlcloud

Installing Erlang modules is easy if you use agner. Agner is a package management tool, like easy_install, pip, gem, cpan, hackage, etc.

$ sudo agner install erlcloud
Password:
remote: Counting objects: 289, done.
...
Compiled src/erlcloud_ec2.erl
[Installing...]
Installed to:
/usr/local/agner/packages/erlcloud-@master
$

Done.

The Code

The first function, init_ec2conn, starts the required Erlang processes for erlcloud to work. It then connects to ec2 and returns the config information about the connection.

The second function, create_instances/2, simply calls erlcloud_ec2:run_instances/2 to create the instances. The first parameter is the request config, like how many instances we want, which ami id, etc. The second is the connection config. We just pass that config along.

-module(amazon).

-include_lib("erlcloud/include/erlcloud.hrl").
-include_lib("erlcloud/include/erlcloud_ec2.hrl").

-export([init_ec2conn/0,
         create_instances/2]).


init_ec2conn() ->
    %% These two are required first
    ssl:start(),
    erlcloud:start(),
    erlcloud_ec2:new("API_KEY",
                     "SECRET_KEY").

create_instances(NumInstances, Config) ->
    erlcloud_ec2:run_instances(#ec2_instance_spec{
                                  image_id="ami-ccf405a5",
                                  instance_type="t1.micro",
                                  max_count=NumInstances,
                                  min_count=NumInstances,
                                  availability_zone="us-east-1a",
                                  key_name="KeyName",
                                  group_set=["SecurityGroup"]
                                 },
                               Config).

Using It

Using it interactively is actually fairly verbose. erlcloud starts up about 14 processes. I have simplified the process output to focus on how to use the code.

$ erl
1> c("amazon").
{ok,amazon}

2> EC2 = amazon:init_ec2conn().
...
=PROGRESS REPORT==== 16-Apr-2011::23:35:37 ===
         application: erlcloud
          started_at: nonode@nohost
{aws_config,"ec2.amazonaws.com","s3.amazonaws.com",
            "sdb.amazonaws.com","elasticloadbalancing.amazonaws.com",
            "queue.amazonaws.com","mechanicalturk.amazonaws.com",
            "YOUR_API_KEY",
            "YOUR_SECRET_KEY"}

3> amazon:create_instances(1, EC2).
[{reservation_id,"r-aaaa1111"},
 {owner_id,"888855552222"},
 {group_set,["WebServer"]},
 {instances_set,[[{instance_id,"i-44442b2b"},
                  {image_id,"ami-cc44a5a5"},
                  {instance_state,[{code,0},{name,"pending"}]},
                  {private_dns_name,[]},
                  {dns_name,[]},
                  {reason,none},
                  {key_name,"testing"},
                  {ami_launch_index,0},
                  {product_codes,[]},
                  {instance_type,"t1.micro"},
                  {launch_time,{{2011,4,17},{3,35,45}}},
                  {placement,[{availability_zone,"us-east-1a"}]},
                  {kernel_id,"aki-44449999"},
                  {ramdisk_id,[]},
                  {monitoring,[{enabled,false},{state,"disabled"}]},
                  {subnet_id,[]},
                  {vpc_id,[]},
                  {private_ip_address,[]},
                  {ip_address,[]},
                  {state_reason,[...]},
                  {architecture,...},
                  {...}|...]]}]

4>

The instance is up.

Erlang calculator process.

I wrote a demonstration of Erlang’s process handling by having a calculator process that simply loops across messages and responds with a calculation result. Or it shuts down.

I find this code interesting because it shows how processes can be generated easily and use the concept of a loop to stay running indefinitely.

First, the complete code

-module(calculator_loop).
-export([start/0, compute/3, calculator_loop/0]).

start() ->
    register(calculator, spawn_link(?MODULE, calculator_loop, [])).

compute(Function, IntA, IntB) ->
    calculator ! {Function, self(), IntA, IntB},
    receive
        {result, Result} -> Result
    after 1000 -> timeout
    end.

calculator_loop() ->
    receive
        {Fun, Pid, IntA, IntB} ->
            case(Fun) of
                add ->
                    Pid ! {result, IntA + IntB};
                divide ->
                    Pid ! {result, IntA / IntB};
                mult ->
                    Pid ! {result, IntA * IntB};
                sub ->
                    Pid ! {result, IntA - IntB}
            end,
            calculator_loop();
        stop ->
            io:format("shutting down calculator~n"),
            ok
    end.

How it works

To start the calculator_loop, you call start/0. This simply spawns a process which starts with a call to calculator_loop/0. Running it looks like this.

2> calculator_loop:start().
true

Because start/0 called register/2 we can now reference our calculator_loop by a name, calculator. Notice that the call to spawn/3 was passed as the second argument to register/2.

start() ->
    register(calculator, spawn_link(?MODULE, calculator_loop, [])).

Now that the calculator loop is running, we need to communicate with it. We do this by sending it a message. The compute/3 function is there to help us with that.

The first line looks like this

calculator ! {Function, self(), IntA, IntB},

This takes the all three arguments from compute/3 and passes it, along with the Pid of the current Erlang process to calculator. The first argument is a variable called Function. The second is a call to self(). The third and fourth are for numbers. So, Function should be the function we call on the two numbers, IntA and IntB.

Function itself can be any atom the calculator loop recognizes: add, divide, mult or sub. We’ll cover that more soon.

Passing the Pid of the current process is important so the calculator process knows who to respond to. Erlang makes it easy to get the current Pid by calling self().

7> self().

Calling receive immediately after passing a message puts the current process into a waiting state to wait for a response message from the calculator process. That code looks like this.

receive
    {result, Result} -> Result
after 1000 -> timeout
end.

In addition to waiting for an answer we have a timeout for 1000 milliseconds in case the process is taking an unusually long time.

calculator_loop

The calculator_loop/0 function is running endlessly in it’s own process, but it is waiting for a message before moving forward. Once it receives a message, it attempts to handle it and immediately calls itself again. This gives Erlang a recursive construct for having functions handle messages whenever they’re available.

This function expects to receive a message matching the pattern {Fun, Pid, IntA, IntB}. If that matches, then the function inspects Fun to see if it matches any of the atoms we recognize: add, divide, mult or sub. That looks like this in code

receive
    {Fun, Pid, IntA, IntB} ->
        case(Fun) of
            add ->
                ...
            divide ->
                ...
            mult ->
                ...
            sub ->
                ...
        end,
        calculator_loop();
    ...
end,

If Fun matches one of the atoms we recognize, a computation is done and a message is sent back to the calling Pid with the result.

It’s also possible the process receives a stop message. If so, the loop finishes when calculator_loop() is not called and the function quits. In code, this looks like this.

calculator_loop() ->
    receive
        {Fun, Pid, IntA, IntB} ->
            ...
            calculator_loop();
        stop ->
            io:format("shutting down calculator~n"),
            ok
    end.

compute

The compute function exists just to take an atom and two numbers and then send a message to the calculate process with that atom and the two numbers. Remember this?

compute(Function, IntA, IntB) ->
    calculator ! {Function, self(), IntA, IntB},
    receive
        {result, Result} -> Result
    after 1000 -> timeout
    end.

We send along the atom and two numbers and return the result if we get one. That’s it.

Example

2> calculator_loop:start().
true
3> calculator_loop:compute(add, 2, 3).
5
4> calculator_loop:compute(mult, 2, 3).
6

It is, however, possible to tell the calculator_loop to stop and the process will finish. Notice that we can no longer call compute/3 after we send the stop message to the calculator process.

5> calculator ! stop.
shutting down calculator
stop
6> calculator_loop:compute(mult, 2, 3).
** exception error: bad argument
     in function  calculator_loop:compute/3