Contents

Visualise your data webp image

This post is a short story about how to use a JavaScript library to see what a backend developer cannot see from raw data.

Background

I’m currently working on a project where we are trying to solve the Vehicle Routing Problem (VRP in short) on scale, which is a more complicated variation of the Travelling Salesman Problem (TSP in short), which you are probably familiar with.

The project is about preparing the most optimal routes for couriers by utilizing a number of factors and different constraints like working time windows, service times at depots, unique skill requirements by a courier, and so on. These movable variables impact the calculation of the optimal route the most and can highly reorder the way the final route is computed. Yet the base factor for all the calculations is time & distance cost from point to point.

You are probably already imaging a Google Map with waypoints and routes between them, albeit it isn’t that easy.

The problem

The output from the computation is a JSON payload with routes for each courier that consists of number of stops with defined jobs to either pick up or deliver a package — you can even have multiple pickups and deliveries at the same location. And each location is identified by latitude and longitude coordinates. This makes it very hard to analyse the output by the human eye and to spot potential problems.

Also, as I’m working on the backend part of the project, I don’t have access to all the customer data — this is a SaaS platform with a plethora of clients. All I get is anonymous data with couriers’ IDs, orders’ IDs, and coordinates.

Any change to the logic, small adjustment of the service time behaviour can impact the final result a lot. It can produce unsustainable routes with a strange behaviour like leaving a depot, picking up a package and going back to the depot to pick up the rest of the orders.

Any manual analysis of the results is very time consuming and not only by decoding the coordinates but also because of a large volume of data — just imagine how big the DHL fleet of couriers is for example.

Solution

To solve the presented problem, I needed a tool to quickly visualise the output on a map with additional info like courier ID, coordinates of each location, and jobs to do in each stop. As I never built such a tool, I was looking for something simple to use and that’s how I found the Leaflet library.

Leaflet is a JavaScript library to create interactive maps, it has a large number of features, but from my perspective, I just needed two things: draw a line representing the calculated route and put markers on each stop with detailed info.

Implementation

I’m familiar with the JavaScript and NodeJS environment — maybe not an expert but I’m not scared to use such tools. Also, by having a support group of talented frontend developers at Softwaremill such a tech stack was an obvious choice :)

The tool consists of two pieces:

  • an http server to serve data and a view layer,
  • the view layer, which visualizes JSON data using the Leaflet library.

I put everything in one git repository without splitting the project into submodules — maybe in the future, it will make more sense to do it.

Let’s create a basic structure for the project:

$ mkdir map-viewer
$ cd map-viewer
$ npm init

Answer a few questions to set up a new NodeJS-based project. I just changed entry point from index.js into server.js for clarity.

The next step is to install a library that will be used to implement the HTTP server — ExpressJS is the right choice.

npm install -s express

Now I can implement my server.js to server JSON data and static HTML files:

const express = require('express')
const app = express()
const port = 8080
app.use('/', express.static('static'))
app.listen(port, () => {
  console.log(`Server is listening on ${port}`)
})

And basically, that’s it. I created a simple Express-based server, defined how to serve static files, and now it can be started with:

$ npm start
> map-viewer@1.0.0 start
> node server.js
Server is listening on 8080

Also, I created a ./static/ folder to serve all the static files I need on the frontend.

Viewer

Time to create a front part of the tool, the Map viewer. As you can read on the Leaflet webpage, you must use two files to start building your interactive map:

  • Leaflet CSS file in the head section of the html file,
  • and Leaflet JavaScript file after Leaflet’s CSS — this is very important to put the JavaScript next to the CSS.

The final thing is an anchor to tell Leaflet where to render your map, here is the full skeleton of the html file:

<html lang="en">
<head>
    <title>Map Viewer</title>
    <link rel="stylesheet" href="https://unpkg.com/leaflet@1.7.1/dist/leaflet.css"
          integrity="sha512-xodZBNTC5n17Xt2atTPuE1HxjVMSvLVW9ocqUKLsCC5CXdbqCmblAshOMAS6/keqq/sMZMZ19scR4PsZChSR7A=="
          crossorigin=""/>
    <!-- Make sure you put this AFTER Leaflet's CSS -->
    <script src="https://unpkg.com/leaflet@1.7.1/dist/leaflet.js"
            integrity="sha512-XQoYMqMTK8LvdxXYG3nZ448hOEQiglfqkJs1NOQV44cWnUrBc8PkAOcXy20w0vlaXaVUearIOBhiXZ5V3ynxwA=="
            crossorigin=""></script>

    <style>
        #map {
            height: 100%;
        }
    </style>
</head>
<body>

<div id="map"></div>

</body>
</html>

Now it’s time to implement the viewer, which will need data and logic to render the map. An example JSON output I will be using looks like this:

{
  "tours": [
    {
      "courier_id": 14,
      "stops": [
        {
          "id": 1,
          "lat": 52.0276,
          "lng": 8.554,
          "jobs": [
            {
              "id": 43,
              "type": "pickup"
            },
            {
              "id": 46,
              "type": "pickup"
            }
          ]
        },
        {
          "id": 2,
          "lat": 52.0365,
          "lng": 8.5725,
          "jobs": [
            {
              "id": 46,
              "type": "delivery"
            }
          ]
        },
        ...
      ]
    }
  ]
}

There can be many tours and many stops per each tour. I will put this payload in ./static/data.json to be accessible by the browser.

Let’s create a JavaScript file ./static/map-viewer.js that will be used to render the Map based on Leaflet and data. Remember to include JavaScript in the head section of the html document:

<script src="map-viewer.js"></script>

The first step is to define a function to download data from the server, the JSON payload presented above. This is the simplest way to do so:

function loadJSON(callback) {
  const request = new XMLHttpRequest();
  request.overrideMimeType('application/json');
  request.open('GET', 'data.json', true);
  request.onreadystatechange = function () {
    if (request.readyState === 4 && request.status === 200) {
      // Required use of an anonymous callback as .open 
      // will NOT return a value but simply returns undefined 
      // in asynchronous mode
      callback(request.responseText);
    }
  };
  request.send(null);
}

Note: Please remember that all the files are served from the ./static/ folder but as / endpoint from browser perspective.

Having data at hand, it’s time to define a function to render the map based on the data, here is the skeleton of the function:

function renderMap(response) {
  // create a base map using OSRM as a base layer

  // parse JSON output

  // render routes and stops

  // draw a line representing a given tour
}

Let’s implement each gap.

First, we must have a basic layer with a map. You can use the Mapbox service to get one, just create an account, then get a free access token which will be used by Leaflet to obtain the base layer:

// create a base map using OSRM as a base layer
const map = L.map('map');
L.tileLayer('https://api.mapbox.com/styles/v1/{id}/tiles/{z}/{x}/{y}?access_token={accessToken}', {
  attribution: 'Map data &copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors, Imagery © <a href="https://www.mapbox.com/">Mapbox</a>',
  maxZoom: 18,
  id: 'mapbox/streets-v11',
  tileSize: 512,
  zoomOffset: -1,
  accessToken: '<ACCESS TOKEN>'
}).addTo(map);

L.map('map'); references the anchor where to render the map, L.tileLayer constructs a base layer with a map from Mapbox. Yet this is not enough to see anything on the map. We need to put routes and stops on it, so let’s do that:

// parse JSON output
const jsonData = JSON.parse(response);
// render routes and stops
for (const tour of jsonData.tours) {
  const latLngs = [];
  for (const stop of tour.stops) {
    let lat = stop.lat;
    let lng = stop.lng;

    // create marker for the given stop
    L.marker([lat, lng])
      .addTo(map)
      .bindPopup();

    latLngs.push([lat, lng]);
  }

  // draw a line representing a given tour
  const polyline = L.polyline(latLngs).addTo(map);

  map.fitBounds(polyline.getBounds());
}

And now, finally, you can see your map, just call the function in the html document:

<div id="map"></div>

<script>
    loadJSON(renderMap);
</script>

Image: Rendered map

This is not the end

Currently, only basic info is presented like stops and schematic routes, yet it’s a good starting point to add more features. If you would like to add some details to each stop, just pass this information to .bindPopup call:

L.marker([lat, lng])
  .addTo(map)
  .bindPopup(`Courier: ${tour.courier_id}<br/>${lat},${lng}`);

Image: Waypoint with detailed info

Just examine the Leaflet documentation and examples to extend this simple application in the way you want to.

Conclusion

As you see, it wasn’t that hard to prepare everything and get a proper view of the data — this helped me a lot to understand if the produced result of the computation makes any sense and can be used by couriers to deliver orders.

It’s way easier to spot mistakes and analyse the results, without setting up the whole complicated environment. I can see the map almost the same as it be seen by an operator of the platform.

Post Scriptum: Stop the war

I’m writing this blog post during hard times when Putin’s army invaded Ukraine. The creator of the Leaflet library lives in Ukraine and all of us can support him and the people of Ukraine to stop this war. On the main page of the Leaflet library, you can read more details on how to help.

Slava Ukraini!

Blog Comments powered by Disqus.