A practical introduction to MochiWeb

Published on 11 July 2011, updated on 11 July 2011, Comments

Introduction

Bob Ippolito, creator or MochiWeb, describes it as “an Erlang library for building lightweight HTTP servers”. It’s not a framework: it doesn’t come with URL dispatch, templating or data persistence. Despite not having an official website or narrative documentation, MochiWeb is a popular choice to build web services in Erlang. The purpose of this article is to help you to get started by gradually building a microframework featuring templates and URL dispatch. Persistence will not be covered.

I assume you’re familiar with basic sequential Erlang. If it’s not the case, I’d suggest you study the first few chapters of this guide. No knowledge of concurrent and distributed Erlang is needed to follow this tutorial.

If you get stuck, you can get the code corresponding to this tutorial. Each commit corresponds to a section of the tutorial so you can easily see the code for a specific step by checking out the corresponding commit.

Getting started

Start by getting a copy of MochiWeb using Git:

$ git clone git://github.com/mochi/mochiweb.git

MochiWeb source tree contains a README file that tells you how to create a new project. Let’s create a project called greeting:

$ cd mochiweb
$ make app PROJECT=greeting

We can now compile the code for our new app and start it:

$ cd ../greeting/
$ make
$ ./start-dev.sh

You should see a bunch of PROGRESS REPORT messages. Among those message you should see something like {port,8080} which tells you that your app is running on port 8080. You can now point your browser at http://localhost:8080 and you should see a message telling you that it’s successfully running.

If you go back to the terminal and press return you’ll get an Erlang shell that you can use to interact directly with your app. It will come very handy for debugging or experimenting with libraries, similarly to the ./script/console of Ruby on Rails or the manage.py shell of Django.

Documentation

When I was first looking for MochiWeb documentation, most of the search results I got were messages from people looking for documentation. This is one of the reasons why I decided to write this tutorial.

Acutually, MochiWeb contains some API reference documentation. You can generate it with:

$ cd ../mochiweb
$ make edoc

You should now find doc/index.html in your mochiweb directory. Open it with your browser and you’ll be able to navigate the API documentation generated from source code. It’s useful to get an overview of modules and functions available and to find details about specific functions.

There’s also a good video tutorial by BeeBole which shows an interesting approach to building a fully AJAX app with MochiWeb.

In this tutorial I present a more traditional approach allowing you to map regular browser requests to Erlang functions.

Basic request handling

When we made our first request to our new app, the message we got came from a static index.html file located in greeting/priv/www/. This is where you can put static content such as CSS, images, etc. but for now it’s probably more interesting to start creating a request handler taking some user input.

Our starting point to plug our request handling code is going to be the file src/greeting_web.erl, which contains a function loop/2:

loop(Req, DocRoot) ->
    "/" ++ Path = Req:get(path),
    try
        case Req:get(method) of
            Method when Method =:= 'GET'; Method =:= 'HEAD' ->
                case Path of
                    _ ->
                        Req:serve_file(Path, DocRoot)
                end;
            'POST' ->
                case Path of
                    _ ->
                        Req:not_found()
                end;
            _ ->
                Req:respond({501, [], []})
        end
    catch
        %% ... exception handling code ...
    end.

If you can read Erlang, it shouldn’t be too hard to understand what this does. It extracts the path of the request, serves a corresponding static file if it’s a GET or HEAD request (implicitly mapping / to /index.html), returns a 404 if it’s a POST and an error if it’s another HTTP verb. Any exception is caught and displayed on the terminal. I don’t show exception handling code here for brevety, but it does not mean you should get rid of it!

What we’re going to do now is add some code to handle a request to /hello, get the user name from the query string and display a greeting to the user. In the conditional branch that handles GET requests, we’re going to add this clause:

"hello" ->
  QueryStringData = Req:parse_qs(),
  Username = proplists:get_value("username", QueryStringData, "Anonymous"),
  Req:respond({200, [{"Content-Type", "text/plain"}],
               "Hello " ++ Username ++ "!\n"});

First we use mochiweb_request:parse_qs/0 to get a proplist of query string parameters. We then use proplist:get_value/3 to get the username parameter, providing a default value if the parameter is missing. Finally we call mochiweb_request:respond/1, passing it a tuple containing the HTTP code, a proplist of headers and the body content. Here is what our new loop/2 function looks like:

loop(Req, DocRoot) ->
    "/" ++ Path = Req:get(path),
    try
        case Req:get(method) of
            Method when Method =:= 'GET'; Method =:= 'HEAD' ->
                case Path of
                    "hello" ->
                        QueryStringData = Req:parse_qs(),
                        Username = proplists:get_value("username", QueryStringData, "Anonymous"),
                        Req:respond({200, [{"Content-Type", "text/plain"}],
                                     "Hello " ++ Username ++ "!\n"});
                    _ ->
                        Req:serve_file(Path, DocRoot)
                end;
            'POST' ->
                case Path of
                    _ ->
                        Req:not_found()
                end;
            _ ->
                Req:respond({501, [], []})
        end
    catch
        %% ... exception handling code ...
    end.

</code></pre>

Compile your project with make and you should be able to go to http://localhost:8080/hello?username=Mike and see the famous quote from Erlang the movie: Hello Mike!

Rendering templates

Your new exciting greeting feature works, but its presentation might be a bit too simple. Let’s see if we can enhance the user experience using cutting edge HTML technology. We’re going to use ErlyDTL, an Erlang implementation of Django Template Language written by Evan Miller. If you’re not familiar with Django templates, you might want to take a look at its documentation but basically I can tell you variable substitution looks like {{my_variable}} and control is achieved using tags with a syntax such as {% tagname param %}here comes some content{% endtagname %}.

Installing ErlyDTL

First let’s add ErlyDTL to our application. Mochiweb uses Rebar, a build and packaging tool for Erlang applications, which we can use to add a dependency to our project. Open the configuration file rebar.config. You’ll see an entry pointing to MochiWeb git repository. Let’s add another entry for ErlyDTL so that our Rebar config file now looks like:

%% -*- erlang -*-
{erl_opts, [debug_info]}.
{deps, [
  {erlydtl, ".*",
   {git, "git://github.com/evanmiller/erlydtl.git", "master"}},
  {mochiweb, ".*",
   {git, "git://github.com/mochi/mochiweb.git", "master"}}]}.
{cover_enabled, true}.
{eunit_opts, [verbose, {report,{eunit_surefire,[{dir,"."}]}}]}.

You should now be able to fetch and compile ErlyDTL by just typing make:

$ make
==> mochiweb (get-deps)
==> greeting (get-deps)
Pulling erlydtl from {git,"git://github.com/evanmiller/erlydtl.git","master"}
Initialized empty Git repository in /home/al/dev/projects/greeting/deps/erlydtl/.git/
==> erlydtl (get-deps)
==> erlydtl (compile)
Compiled src/erlydtl_parser.yrl
[...]

In order for the new library to be taken into account, you will need to restart the application by typing q(). in the Erlang shell and execute ./start-dev.sh again.

Of course, this method is not limited to ErlyDTL. You can add other dependencies the same way using Rebar.

Template compilation

ErlyDTL compiles Django Template source code into Erlang bytecode. While it’s perfectly possible to manage ErlyDTL templates compilation in our code, Rebar happens to provide support for it, so let’s use it.

We’ll create a templates directory, where Rebar looks for templates by default:

$ mkdir templates

Now create a file templates/greeting.dtl with a content similar to:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <title>MochiWeb Tutorial</title>
    <link rel="stylesheet" type="text/css" href="/style.css" media="screen">
  </head>
  <body>
    <p>
        Hello {{ username }}!
    </p>
  </body>
</html>

If you make again you’ll see that Rebar (which is invoked by make) has created an Erlang module ebin/greeting_dtl.beam. Note that Rebar provides options to customize location and file names of template source files and compiled template modules.

You can now use your new template in the request handler using this code:

QueryStringData = Req:parse_qs(),
Username = proplists:get_value("username", QueryStringData, "Anonymous"),
{ok, HTMLOutput} = greeting_dtl:render([{username, Username}]),
Req:respond({200, [{"Content-Type", "text/html"}],
        HTMLOutput});

Run make, refresh your browser and be in awe of the fine piece of web design you’ve created.

As you might have noticed by now, you need to run make when you change your code. You might want to configure a shortcut in your text editor to start make easily. From now on, I won’t tell you when you need to run make.

POST requests

Your app is already getting popular but some users are complaining that they can’t remember the query string syntax and would like to be able to enter their name using a form. You edit your template and add an HTML form with a text field for the user name:

<form method="POST">
    <p>Username: <input type="text" name="username"></p>
    <input type="submit">
</form>

You also change your request handler to support POST requests, so that it now looks like:

loop(Req, DocRoot) ->
    "/" ++ Path = Req:get(path),
    try
        case Req:get(method) of
            Method when Method =:= 'GET'; Method =:= 'HEAD' ->
                case Path of
                    "hello" ->
                        QueryStringData = Req:parse_qs(),
                        Username = proplists:get_value("username", QueryStringData, "Anonymous"),
                        {ok, HTMLOutput} = greeting_dtl:render([{username, Username}]),
                        Req:respond({200, [{"Content-Type", "text/html"}],
                                HTMLOutput});
                    _ ->
                        Req:serve_file(Path, DocRoot)
                end;
            'POST' ->
                case Path of
                    "hello" ->
                        PostData = Req:parse_post(),
                        Username = proplists:get_value("username", PostData, "Anonymous"),
                        {ok, HTMLOutput} = greeting_dtl:render([{username, Username}]),
                        Req:respond({200, [{"Content-Type", "text/html"}],
                                HTMLOutput});
                    _ ->
                        Req:not_found()
                end;
            _ ->
                Req:respond({501, [], []})
        end
    catch
        % ... exception handling code ...
    end.

It seems to work very well but something bothers you. Seeing "hello" in two places in not a good sign. Repetition of code that renders the template and returns a response isn’t great either. We also notice that if we visit /hello/ with a final slash, we get a page not found error. It’s time for some refactoring.

A simple URL dispatcher

We’re going to create a minimalistic URL dispatcher that will allow you to map regular expressions to Erlang functions. Here is what URL configuration will look like:

[
  {"^hello/?$", hello}
]

This says that any request to /hello or /hello/ should be routed to a function named hello.

Here is the code for the dispatcher:

% Iterate recursively on our list of {Regexp, Function} tuples
dispatch(_, []) -> none;
dispatch(Req, [{Regexp, Function}|T]) -> 
    "/" ++ Path = Req:get(path),
    Method = Req:get(method),
    Match = re:run(Path, Regexp, [global, {capture, all_but_first, list}]),
    case Match of
        {match,[MatchList]} -> 
            % We found a regexp that matches the current URL path
            case length(MatchList) of
                0 -> 
                    % We didn't capture any URL parameters
                    greeting_views:Function(Method, Req);
                Length when Length > 0 -> 
                    % We pass URL parameters we captured to the function
                    Args = lists:append([[Method, Req], MatchList]),
                    apply(greeting_views, Function, Args)
            end;
        _ -> 
            dispatch(Req, T)
    end.

Add the dispatcher code somewhere in greeting_web.erl and modify loop/2 to make use of it:

loop(Req, DocRoot) ->
    "/" ++ Path = Req:get(path),
    try
        case dispatch(Req, greeting_views:urls()) of
            none -> 
                % No request handler found
                case filelib:is_file(filename:join([DocRoot, Path])) of
                    true -> 
                        % If there's a static file, serve it
                        Req:serve_file(Path, DocRoot);
                    false ->
                        % Otherwise the page is not found
                        Req:not_found()
                end;
            Response -> 
                Response
        end
    catch
        % ... exception handling code ...
    end.

Now we’re going to create a module that contains URL configuration and request handlers. Create a file src/greeting_views.erl that contains this code:

-module(greeting_views).
-compile(export_all).
-import(greeting_shortcuts, [render_ok/3]).

urls() -> [
      {"^hello/?$", hello}
    ].

hello('GET', Req) ->
    QueryStringData = Req:parse_qs(),
    Username = proplists:get_value("username", QueryStringData, "Anonymous"),
    render_ok(Req, greeting_dtl, [{username, Username}]);
hello('POST', Req) ->
    PostData = Req:parse_post(),
    Username = proplists:get_value("username", PostData, "Anonymous"),
    render_ok(Req, greeting_dtl, [{username, Username}]).

We make use of a function render_ok/3 to avoid a bit of code duplication when returning responses. Let’s define this function in src/greeting_shortcuts.erl:

-module(greeting_shortcuts).
-compile(export_all).

render_ok(Req, TemplateModule, Params) ->
    {ok, Output} = TemplateModule:render(Params),
    % Here we use mochiweb_request:ok/1 to render a reponse
    Req:ok({"text/html", Output}).

Now you’ve got a more generic way to handle requests. We removed some duplication and our code is also more organized because we’ve now defined a place where to put our request handlers and a place where to put utility functions.

This is all good but one of your friend tells you that she likes being able to get a greeting with a GET request, but she finds using the query string rather ugly. Instead she says it would be great if she could get a greeting just by visiting /hello/Alice or /hello/Alice/. Luckily our URL dispatcher makes it easy to add this innovative functionality.

Add a second entry to your URL configuration so now it looks like:

urls() -> [
      {"^hello/?$", hello},
      {"^hello/(.+?)/?$", hello}
    ].

And create the request handler (in greeting_views.erl) that will accept a URL parameter:

hello('GET', Req, Username) ->
    render_ok(Req, greeting_dtl, [{username, Username}]);
hello('POST', Req, _) ->
  % Ignore URL parameter if it's a POST
  hello('POST', Req).

Voila, /hello/Alice or /hello/Alice/ should now work too.

Handling cookies

You’re getting more and more feedback about your application and some users tell you it would be great if next time they visit /hello/ it could remember their name. Let’s use a cookie for that. Edit greeting_shortcuts.erl and add a function that returns a cookie value or a default value if the cookie is not present. Also create a new function render_ok/4, similar to our existing render_ok/3 except that it takes an extra Headers parameter that we’ll use to send the cookie header. Modify render_ok/3 so that it now just calls render_ok/4 with an empty list of headers.

-module(greeting_shortcuts).
-compile(export_all).

render_ok(Req, TemplateModule, Params) ->
    render_ok(Req, [], TemplateModule, Params).

render_ok(Req, Headers, TemplateModule, Params) ->
    {ok, Output} = TemplateModule:render(Params),
    Req:ok({"text/html", Headers, Output}).

get_cookie_value(Req, Key, Default) ->
    case Req:get_cookie_value(Key) of
        undefined -> Default;
        Value -> Value
    end.

Now edit your view module to make use of these new functions. We’ll also remove some duplication while we’re at it.

-module(greeting_views).
-compile(export_all).
-import(greeting_shortcuts, [render_ok/3, render_ok/4, get_cookie_value/3]).

urls() -> [
      {"^hello/?$", hello},
      {"^hello/(.+?)/?$", hello}
    ].

% Return username input if present, otherwise return username cookie if
% present, otherwise return "Anonymous"
get_username(Req, InputData) ->
    proplists:get_value("username", InputData,
        get_cookie_value(Req, "username", "Anonymous")).

make_cookie(Username) ->
    mochiweb_cookies:cookie("username", Username, [{path, "/"}]).

handle_hello(Req, InputData) ->
    Username = get_username(Req, InputData),
    Cookie = make_cookie(Username),
    render_ok(Req, [Cookie], greeting_dtl, [{username, Username}]).

hello('GET', Req) ->
    handle_hello(Req, Req:parse_qs());
hello('POST', Req) ->
    handle_hello(Req, Req:parse_post()).

hello('GET', Req, Username) ->
    Cookie = make_cookie(Username),
    render_ok(Req, [Cookie], greeting_dtl, [{username, Username}]);
hello('POST', Req, _) ->
  hello('POST', Req).

When users set their username, it should now be stored in a cookie and displayed when they visit /hello/.

Conclusion

That’s it for this tutorial. Now that you know how to add a library to your project, get user input, render templates and set cookies, you should have the building blocks to add further functionality such as authentication/authorization, global template context, persistence, etc. You could also modify the URL dispatcher to allow specifying a module as well as a function, or take a different approach, maybe based on conventions rather than configuration.

I hope you’re now slightly more familiar with MochiWeb so that you can come up with the way that best suits your needs. Browse the API documentation to find out more about what MochiWeb has to offer and by all means don’t hesitate to look at the source code. This is something I found to be generally true when working with Erlang libraries: the source code often speaks more that the documentation. Fortunately, if you’re like me, you might find that Erlang code is often easier to understand than other languages, probably because of its functional nature and its simplicity, so use the source Luke!