Realtime timeline with hapi.js, nes and RethinkDB

A question on Stackoverflow inspired me to write this tutorial. I could not find any article showing how to utilize the nes plugin to build a realtime project using websockets with hapi. So I decided to write one - I hope you enjoy it...

Our Application

I thought a little bit to find an appropriate sample application which makes sense. Then I came up with the idea of building a very simple realtime timeline just to show the concepts. Finally I wanted to integrate RethinkDB for persistence and especially to try out their changefeed feature.

This is the result:

It shows the last 5 entries stored in the database. To keep things simple and simulate the creation of new entries the button Generate new timeline entry is used.

Getting started

Make sure you have the latest version of Node installed on your machine as we use some of the new ES6 features.

You should also have a RethinkDB server running on your localhost. Setup instructions can be found here.

We use browserify for bundling the JavaScript for the client. So you need to install it in order to run the application:

npm install -g browserify  

Everything setup? Let's continue...


You want to learn more?

I just created a new online video course on Udemy and offer a 50% discount to loyal readers of my blog...

All you have to do is to click on the course image above. The discount code is included in the link!

I'm happy to see you in the course...


Step 1: Clone the git repository

First clone the git repository and install the dependencies. After that we look at the code.

git clone https://github.com/p-meier/hapi-realtime-timeline.git  
cd hapi-realtime-timeline  
npm install  

Step 2: Looking at the code in detail

The code itself is heavily commented though you might get a good understanding by just reading it. To complement this we look at the code here in a more general sense and explain important concepts.

File structure

This is the file structure of our application.

- node_modules/     // created by 'npm install'
- plugins/
----- db.js         // manage db connection and queries
- public/           // frontend
----- index.html
----- main.js
----- styles.css
- package.json      // dependencies and project info
- server.js         // entry point of app, define server

Node Packages

{
  "name": "hapi-realtime-timeline",
  "version": "1.0.0",
  "description": "A realtime timeline built with hapijs and nes.",
  "main": "server.js",
  "private": true,
  "scripts": {
    "build-client": "browserify public/main.js >  public/bundle.js",
    "start": "npm run build-client; node server"
  },
  "dependencies": {
    "faker": "^3.1.0",
    "handlebars": "^4.0.5",
    "hapi": "^13.4.1",
    "inert": "^4.0.0",
    "jquery": "^3.0.0",
    "nes": "^6.2.1",
    "rethinkdb": "^2.3.2"
  }
}

So let's look at the packages used:

  • faker generate fake data (names, avatar images, text)
  • handlebars templating engine used in frontend
  • hapi should be clear
  • inert static file handling for hapi
  • jquery frontend dependency
  • nes websocket adapter for hapi routes
  • rethinkdb driver for the database

Frontend

The frontend is made up of a little Bootstrap and jQuery as well as the client portion of the nes plugin. I omit the css portion here for brevity.

File main.js:
var Nes = require('nes/client');  
var $ = require('jquery');  
var Handlebars = require('handlebars');

//Compile the template
var source = $("#entry-template").html();  
var template = Handlebars.compile(source);

//Load initial entries
$.getJSON('/timeline', function (data) {

    data.forEach(function (item) {

        var html = template(item);
        $("#timeline").append(html);
    });
});

//Create a new entry
$('#add-entry').click(function () {
    $.get('/timeline/createEntry');
});

//Setup the websocket connection and react to updates
var client = new Nes.Client('ws://localhost:3000');  
client.connect(function (err) {

    var handler = function (item) {

        var html = template(item);
        $("#timeline").prepend(html);
    };

    client.subscribe('/timeline/updates', handler, function (err) {});
});

Nothing too complicated here. The most interesting portion is the last block where the websocket connection is established and we subscribe to /timeline/updates. Handlebars is used as templating engine for rendering new entries.

File index.html:
<!DOCTYPE html>  
<html lang="en">

<head>  
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, minimum-scale=1, user-scalable=no" />

  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css">
  <link rel="stylesheet" href="styles.css">

  <title>hapi realtime timeline with RethinkDB</title>
</head>

<body>

  <div class="container">

    <div class="jumbotron text-center">
      <h1>Realtime timeline</h1>
      <p class="lead">Shows an example of a realtime timeline built with <a href="http://hapijs.com/" target="_blank">hapi</a> and the
        <a href="https://github.com/hapijs/nes" target="_blank">nes</a> plugin. The database used is <a href="https://www.rethinkdb.com/" target="_blank">RethinkDB</a>.</p>
      <p><a id="add-entry" class="btn btn-lg btn-success" href="#" role="button">Generate new timeline entry</a></p>
    </div>

    <div class="row">
      <div id="timeline" class="col-md-12"></div>
    </div>

  </div>
  <!-- /container -->

  <script id="entry-template" type="text/x-handlebars-template">
    <div class="entry">
      <div class="bs-callout bs-callout-primary">
        <h4><img class="avatar" src="{{avatar}}"/> {{user}}</h4>
        <p>{{message}}</p>
      </div>
    </div>
  </script>

  <script src="bundle.js"></script>
</body>

</html>

Server

Time to look at the server side code.

File server.js:
'use strict';

const Hapi = require('hapi');  
const Nes = require('nes');  
const Inert = require('inert');  
const Faker = require('faker');  
const Db = require('./plugins/db');

const server = new Hapi.Server();  
server.connection({  
    port: 3000
});

server.register([Nes, Inert, Db], (err) => {

    if (err) {
        throw err;
    }

    //Serve static files in 'public' directory
    server.route({
        method: 'GET',
        path: '/{param*}',
        handler: {
            directory: {
                path: 'public'
            }
        }
    });

    //Return the last 5 entries stored in db
    server.route({
        method: 'GET',
        path: '/timeline',
        handler: function (request, reply) {

            server.methods.db.findEntries(5, (err, result) => {

                if (err) {
                    return reply().code(500);
                }

                return reply(result);
            });
        }
    });

    //Create a new entry
    server.route({
        method: 'GET',
        path: '/timeline/createEntry',
        handler: function (request, reply) {

            const entry = {
                createdAt: new Date(),
                user: Faker.name.findName(),
                message: Faker.lorem.paragraph(),
                avatar: Faker.image.avatar()
            };

            server.methods.db.saveEntry(entry, (err) => {

                if (err) {
                    return reply().code(500);
                }

                return reply().code(204);
            });
        }
    });

    //Declare the subscription to timeline updates the client can subscribe to
    server.subscription('/timeline/updates');

    // Start the server
    server.start((err) => {

        if (err) {
            throw err;
        }

        //Setup the RethinkDB change-feed and push it to the websocket connection.
        server.methods.db.setupChangefeedPush();

        console.log('Server running at:', server.info.uri);
    });
});

After registering the plugins with the server some routes are setup. The code is very well commented what the different routes are for and it should be pretty self-explanatory.

By invoking /timeline/createEntry with a GET request the following things happen:

  1. A new timeline entry is created using the faker plugin
  2. The newly created entry is stored in RethinkDB using the db.saveEntry server method
  3. The changefeed listener in db.js is invoked
  4. The stored entry is pushed to /timeline/updates and is processed by the frontend
File db.js:
'use strict';

const r = require('rethinkdb');

exports.register = function (server, options, next) {

    const db = 'hapi_timeline';
    const entriesTable = 'entries';
    let conn;

    //Connect and initialize
    r.connect((err, connection) => {

        if (err) {
            return next(err);
        }

        conn = connection;

        //Create db
        r.dbCreate(db).run(connection, (err, result) => {

            //Create entries table
            r.db(db).tableCreate(entriesTable).run(connection, (err, result) => {

                return next();
            });

        });
    });

    server.method('db.saveEntry', (entry, callback) => {

        r.db(db).table(entriesTable).insert(entry).run(conn, callback);
    });

    server.method('db.findEntries', (limit, callback) => {

        r.db(db).table(entriesTable).orderBy(r.desc('createdAt')).limit(limit).run(conn, callback);
    });

    server.method('db.setupChangefeedPush', () => {

        r.db(db).table(entriesTable).changes().run(conn, (err, cursor) => {

            cursor.each((err, item) => {

                server.publish('/timeline/updates', item.new_val);
            });
        });
    }, {
        callback: false
    });
};

exports.register.attributes = {  
    name: 'db'
};

In the first part of the plugin we make sure the database (hapi_timeline) and table (entries) exist.

After that we setup some server methods. The most interesting one is the last which sets up the RethinkDB changefeed. Whenever something in the entries table changes this listener is called and publishes the new entry to /timeline/updates where it is handled by the frontend.

Of course in this example it would not be necessary to utilize the changefeed feature - we could just publish the new entry in the db.saveEntry server method. But I wanted to show the usage of this feature here.

Step 3: Testing in the Browser

Now start the application - this triggers the build of the client side javascript bundle and then fires up the server:

npm start  

Fire up your Browser and navigate to http://localhost:3000. Now do the same with a second browser or window and align them side by side. If you hit the button Generate new timeline entry a new timeline entry should appear in both windows at the same time.

Patrick Meier

I am an entrepreneur and software developer, building scalable, distributed web systems with Java, NodeJs and AngularJs.

Weiden, Germany