Enhancing Shared Mobility Apps with RabbitMQ

By Don Omondi, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

Queues, everybody hates being at the wrong end of them, but nobody doubts the level of organization and efficiency that they bring in service delivery. Civilizations have been using queues for centuries, but it wasn't until the intervention of a Danish mathematician Agner Krarup Erlang, that the concept of queues evolved from just a social culture to a science. Erlang developed the Queueing theory while creating models to describe the Copenhagen telephone exchange. Erlang, the highly concurrent, functional programming language, is named in his honor.

Since its transition to a science, queues have been heavily used when developing applications in almost every sector. They are predominantly found in telecoms, banking, e-commerce, computer telephony and instant messaging. WhatsApp, written in Erlang, is one of the best examples of such implementation.

You don't have to write your application in Erlang to use queues though. As a matter of fact, many people designate the queuing functionality to specialized software. RabbitMQ is at the pinnacle of such software. Also written in Erlang, RabbitMQ claims to be the most widely deployed open source message broker.

The brief history and introductions aside, let's see how we can integrate RabbitMQ in shared mobility apps.

The Rise and Rise of Shared Mobility

The sharing economy, also known as collaborative consumption, has seen a sharp rise in the past years mostly driven by emerging sectors such as social lending, peer-to-peer accommodation and peer-to-peer travel experiences. Shared mobility in particular, has become a lucrative and competitive field more so in the areas of car-sharing, bicycle sharing and even scooter sharing. I'd go on to predict baby stroller sharing as the next big frontier.

The high demand for these services has increased the pressure for the applications behind them to be reliable, fast and efficient. A common way to achieve this is by decoupling different services into separate components that can each be executed or fail, without impacting the others – the so called microservices trend. In the case of failure, automated retry mechanisms should be put in place.

Let's look at a typical example by taking 5 mins to create our own ride sharing app-or at least the core components of it. When a rider within our ride sharing app moves around, we would like to perform certain actions based on that event. The actions could be to:-

1) Log that event in our geolocation capable database such as PostgreSQL or Elasticsearch for future analytics.
2) Notify and update all opened client applications within the rider's location of his new position.
3) Save that new location in a fast in-memory Key-Value store like Redis so that newly opened client applications can see nearby riders for a given time-to-live (ttl) of say 5 minutes.

Other actions could also be done depending on certain conditions and features, like sending the rider a voice text if he is driving too fast, or update current ride price on clients' phones for real-time billing.

The keen eye would notice that all the actions mentioned above can and ideally should be executed independent of the other. With non-concurrent languages like PHP, it's a near impossibility, whereas with highly concurrent ones like Go you'd still have to handle failure and retries manually and you'd lose the data in event of server crash or restart. This is the kind of situation where RabbitMQ shines.

The diagram above depicts at a very high level, what RabbitMQ helps you achieve. Basically, one event comes in along with some data and attributes and RabbitMQ routes (forwards) it to the appropriate channels where consumers await to process that data. Until those consumers acknowledge the message, typically after successful processing is done, RabbitMQ will keep that message in its queue and keep retrying to send it back for processing and acknowledgement.

At a lower level, RabbitMQ achieves this utilizing its concepts of exchanges, bindings and queues. Of course there are tons of other interesting features, but for our simple ride sharing app these are what we need to understand and use.

In a nutshell, our RabbitMQ broker receives the message from our mobility app, formally termed as the producer. Within the broker, the message arrives at the exchange, which is now responsible for routing of the message to queues bound to it. Thus bindings have to be created from the exchange to queues before producing messages. The message routing depends on different factors such as the exchange type. Our simple mobility app uses the fanout exchange type where the exchange routes messages to all of the queues that are bound to it.

If all this is beginning to sound complex, trust me it's not, and the following practical example will prove it.

A Simple Frontend

First, we'll build out a very basic frontend which will fetch our current location and show it on a map. When we move, it will also update that location and its respective map position. Lastly, for every location update, we will send the coordinates and rider_id to our backend which in turn will send it to RabbitMQ for fanout routing and processing.

We'll need a few tools to begin with, make sure nodejs is installed. I also happen to prefer yarn to npm, whose advanced caching comes in handy in intermittent connectivity situations .

yarn add express socket.io leaflet amqplib alertify.js

The 5 packages are all we need for our simple mobility app. The necessary frontend HTML itself is ridiculously simple.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>Mobility App</title>
    <link rel="stylesheet" href="leaflet.css" />
    <link rel="stylesheet" href="css/alertify.css" />
    <style>
    html, body, #map {
      height:100%;
      width: 100%;
      z-index: 0;
    }
    body{
      margin: 0;
      padding: 0;
      font-size: 14px;
    }
    </style>
  </head>
  <body>
    <div id="map"></div>
    <script src="leaflet.js"></script>
    <script src="socket.io.js"></script>
    <script src="js/alertify.js"></script>
    <script src="app.js"></script>
  </body>
</html>

30 lines and that's it! We only need to render a leaflet powered map and have alertify.js popup some simple alerts and notifications on certain events.

Now for the business logic in app.js. Leaflet has some very nice built-in helpers that can automatically ask for permission and, if granted, detect your location, position a blue marker on the map's center and track your movement. It does all the heavy lifting for our simple mobility app. All we have to do, is hook into leaflet's location success event and emit a socket.io event. We'll also add a custom yellow marker to show other riders on the map.

 // Start with the Map Themes
function getMapTheme(theme) {
  let mapTheme;
  // Default Theme
  mapTheme = 'https://cartodb-basemaps-{s}.global.ssl.fastly.net/light_all/{z}/{x}/{y}.png';
  if ('light_all' === theme) {
    mapTheme = 'https://cartodb-basemaps-{s}.global.ssl.fastly.net/light_all/{z}/{x}/{y}.png';
  }
  if ('dark_all' === theme) {
    mapTheme = 'https://cartodb-basemaps-{s}.global.ssl.fastly.net/dark_all/{z}/{x}/{y}.png';
  }
  return mapTheme;
}

// Add Map Attribution
let mapAttribution = '© <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors, © <a href="http://cartodb.com/attributions">CartoDB</a>';
let lighttheme = L.tileLayer(getMapTheme('light_all'), { attribution: mapAttribution });
let darktheme  = L.tileLayer(getMapTheme('dark_all'), { attribution: mapAttribution });

// Add Themes to selectable Map Layers
let baseLayers = {
  "Light Theme": lighttheme,
  "Dark Theme": darktheme
};

// Setup a Riders' Marker Group
let ridersMarkers = L.layerGroup();
let overlayMaps = {
  "Riders": ridersMarkers
};

// Initialize and show the map
let map = L.map('map', {
  attributionControl: true,
  zoom: 16,
  layers: [lighttheme]
}).fitWorld();

// Add selectable controls to it
L.control.layers(baseLayers, overlayMaps).addTo(map);

// Create custom marker icons for riders other than myself
let customIcon = L.Icon.extend({
  options: {
    shadowUrl: "/img/marker-shadow.png",
    iconSize: [25, 39],
    iconAnchor:   [12, 36],
    shadowSize: [41, 41],
    shadowAnchor: [12, 38],
    popupAnchor: [0, -30]
  }
});
let yellowIcon = new customIcon({ iconUrl: "/img/marker-yellow.png" });

// Function to add these custom markers
function setMarker(data) {
  for (i = 0; i < data.coords.length; i++) {
//    let marker = L.marker([data.coords[i].lat, data.coords[i].lng], { icon: yellowIcon }).addTo(map);
    let marker = L.marker([data.coords[i].lat, data.coords[i].lng], { icon: yellowIcon });
    marker.bindPopup("A ride is here!");
    // add marker
    ridersMarkers.addLayer(marker);
    alertify.success("A nearby rider ID " + data.id + " has just connected!");
  }
}

// Socket IO Client Initialization
let connects = {};
let socket = io('http://127.0.0.1:80');
socket.on('receive', function(data) {
  alertify.log("New Socket Event Received");
  if (!(data.id in connects)) {
    setMarker(data);
  }
  connects[data.id] = data;
});

// placeholders for the L.marker and L.circle representing user's current position and accuracy    
let current_position, current_accuracy;

function onLocationFound(e) {
  // if position defined, then remove the existing position marker and accuracy circle from the map
  if (current_position) {
    map.removeLayer(current_position);
    map.removeLayer(current_accuracy);
  }

  let radius = e.accuracy / 2;

  current_position = L.marker(e.latlng)
    .addTo(map)
    .bindPopup('Your current position and a ' + radius + ' meters radius').openPopup();

  current_accuracy = L.circle(e.latlng, radius).addTo(map);

  let data = {
    id: userId,
    coords: [{
      lat: e.latitude,
      lng: e.longitude,
      acr: e.accuracy
    }]
  }
  socket.emit("send", data);
}

let errors = {
  1: "Geolocation Permission Denied",
  2: "Network Error",
  3: "Connection Timeout"
};

function onLocationError(error) {
  // confirm dialog
  alertify.confirm("We could not get your current location, reason : " + errors[error.code] + "! Would you like to reload the page and try again?", function () {
      // user clicked "ok"
      location.reload();
  }, function() {
      // user clicked "cancel"
      alertify.log("Please Note You Might Be Getting Stale Data!");
  });

  console.log(
    'code: '    + error.code    + '\n' +
    'message: ' + error.message + '\n',
    'Geo-Location Error'
  );
}

map.on('locationfound', onLocationFound);
map.on('locationerror', onLocationError);
map.on('moveend', function(e) {
  let bounds = map.getBounds();

  let sw = bounds.getSouthWest();
  let ne = bounds.getNorthEast();

  let sw_latitude = sw.lat;
  let sw_longitude = sw.lng;

  let ne_latitude = ne.lat;
  let ne_longitude = ne.lng;

  let zoom = map.getZoom();

  if( zoom < 16 ){
    // remove all the current layers
    ridersMarkers.clearLayers();
    // Show Toast to zoom in
    alertify.log("Please zoom in to view nearby riders");
  } else {
    // To Do - Query DB for nearby rides
  }
});

map.stopLocate();

// Start Mobility App function
let startApp = function (){
  // check whether browser supports geolocation api
  if (navigator.geolocation) {
    map.locate({
      watch:              true, 
      setView:            true, 
      maxZoom:            16,
      timeout:            60000,  
      maximumAge:         60000, 
      enableHighAccuracy: false 
    });
  } else {
    alertify.alert("Sorry, your browser does not support geolocation!");
  }
}

// Simulate Rider Authorization by asking for hypothetical ID
let userId;
let defaultUserId = "9999";
alertify
  .defaultValue("9999")
  .prompt("Please Enter A Hypothetical User ID : ",
    function (val, ev) {
      ev.preventDefault();
      alertify.success("You've clicked OK and typed: " + val);
      userId = val;

      startApp();
    }, function(ev) {
      ev.preventDefault();
      alertify.error("You've clicked Cancel, default ID 9999 used");
      userId = defaultUserId;

      startApp();
    }
  );

The business logic is a bit longer than the HTML, but it still clocks in under 200 lines of heavily commented JavaScript.

The express server.js code is 50 lines of JavaScript bliss.

let express = require('express')
let amqplib = require('amqplib')
let app = express()
let server = require('http').createServer(app)
let io = require('socket.io')(server)

app.use(express.static('public'))
app.use(express.static('node_modules/leaflet/dist'))
app.use(express.static('node_modules/alertify.js/dist'))
app.use(express.static('node_modules/socket.io-client/dist'))

let open = amqplib.connect('amqp://localhost')

// Create the RabbitMQ Exchange, Queues and Bindings
open.then(function(conn) {
  return conn.createChannel()
}).then(function(ch) {
  return ch.assertExchange('geolocation-exchange', 'fanout').then(function(geolocationExchange) {
    ch.assertQueue('analytics-queue').then(function(analyticsQueue) {
      ch.bindQueue(analyticsQueue.queue, geolocationExchange.exchange)
    })
    ch.assertQueue('map-queue').then(function(mapQueue) {
      ch.bindQueue(mapQueue.queue, geolocationExchange.exchange)
    })
    ch.assertQueue('redis-queue').then(function(redisQueue) {
      ch.bindQueue(redisQueue.queue, geolocationExchange.exchange)
    })
  })
}).catch(console.warn)

io.sockets.on('connection', function (socket) {
  socket.on('send', function (data) {
    socket.broadcast.emit('send', data);
    // Send to data to RabbitMQ
    open.then(function(conn) {
      return conn.createChannel()
    }).then(function(ch) {
      return ch.assertExchange('geolocation-exchange', 'fanout').then(function(geolocationExchange) {
        ch.publish(geolocationExchange.exchange, '', Buffer.from(JSON.stringify(data)))
      })
    }).catch(console.warn)
  })
  socket.on('receive', function (data) {
    socket.broadcast.emit('receive', data);
  })
})

server.listen(80, '127.0.0.1', function() {
  console.log('Mobility App running on Localhost')
})

It starts by simply requiring some necessary libraries, then specifying some folders that host static assets for our frontend. We then establish a RabbitMQ connection and specify an exchange as a fanout type. Next, we create some queues and bind them to the created exchange. We then wait for a socket.io server connection to materialize then listen for the 'send' and 'receive' from all clients. The 'send' events from our frontend will be forwarded to our RabbitMQ broker. Lastly, we bind an express server to port 80 (Node might need sudo permissions if on linux).

The last part of our simple mobility app, are the RabbitMQ consumers. We'll start with the 'map-queue' consumer. It basically consumes messages from the 'map-queue' and forwards it back to our socket.io server to distribute to frontend clients.

consumers/map.js

let amqplib = require('amqplib')
let io = require('socket.io-client')
let socket = io.connect('http://127.0.0.1:80', { reconnect: true });

let queue = 'map-queue'
let open = amqplib.connect('amqp://localhost')

// Update Client Map Consumer
open.then(function(conn) {
  return conn.createChannel();
}).then(function(ch) {
  return ch.assertQueue(queue).then(function() {
    return ch.consume(queue, function(msg) {
      if (msg !== null) {
        console.log(msg.content.toString());
        // Tell socket.io server to update clients with new geolocation details
        let data = msg.content.toString()
        socket.emit('receive', JSON.parse(data));
        ch.ack(msg);
      }
    });
  });
}).catch(console.warn);

Just 23 lines and we're done! We'll similarly build consumers for the 'analytics-queue' and the 'redis-queue'. This will simply log the data received from the queue for now.

consumers/analytics.js

let amqplib = require('amqplib')
let queue = 'analytics-queue'
let open = amqplib.connect('amqp://localhost')
// Log & Analytics Consumer
open.then(function(conn) {
  return conn.createChannel();
}).then(function(ch) {
  return ch.assertQueue(queue).then(function(ok) {
    return ch.consume(queue, function(msg) {
      if (msg !== null) {
        console.log(msg.content.toString());
        // To Do - Persist new Geolocation coordinates in Analytics store like Elasticsearch
        ch.ack(msg);
      }
    });
  });
}).catch(console.warn);

consumers/redis.js

let amqplib = require('amqplib')
let queue = 'redis-queue'
let open = amqplib.connect('amqp://localhost')
// Key-Value Store Consumer
open.then(function(conn) {
  return conn.createChannel();
}).then(function(ch) {
  return ch.assertQueue(queue).then(function(ok) {
    return ch.consume(queue, function(msg) {
      if (msg !== null) {
        console.log(msg.content.toString());
        // To Do - Persist new Geolocation Key-Value store like Redis
        ch.ack(msg);
      }
    });
  });
}).catch(console.warn);

Running the Demo

We're ready to view our simple mobility app in action now. Change to our project directory, open up 4 terminals and run these commands.

node server.js

node consumers\map.js

node consumers\analytics.js

node consumers\redis.js

Open a browser, preferably chromium based, and visit http://localhost

Your location is now show at the center of the map. Now, open a new browser tab and move around, or better yet mock movement by sending slightly new coordinates from the browser console.

var data = {
  id: 5678,
  coords: [{
    lat: -1.2355076,
    lng: 36.8850185,
    acr: 30
  }]
}
socket.emit("send", data);

Watch new popups appear on the map.

Monitor your console logs and see them logging that data. You can kill a consumer to mock server downtime, wait a minute, an hour or even a day and the messages will still be processed and in order.

Now you see how simple it is to enhance mobility apps with a powerful message queue like RabbitMQ. Never again should your users suffer because your analytics database is down, nor should your analytics database have to wait for Redis to load data from disk during restoration. You can download the source code of this sample ride sharing app from GitHub.

Community

Enhancing Shared Mobility Apps with RabbitMQ

The Rise and Rise of Shared Mobility

A Simple Frontend

Running the Demo

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

Message Service

ECS(Elastic Compute Service)