Creating a node.js GeometryService : part 2

This is my second post on what I learned from creating an implementation of the geometryservice specification in node.js. The first post can be found here.

By default node.js doesn't reload your files when they have changed but while developing this can be very handy. The most easy to use tool for node.js on windows that monitors your files for changes, with an easy way to set the node.js debugging flag, is nodemonw. You can download nodemonw at https://github.com/cenanozen/nodemonw. Once you've downloaded the executable I suggest to copy it to your %Appdata%\npm directory or another directory thats in your path. To start nodemonw I now run the following command nodemonw --debug index.js. In the next section it will become clear why I added the --debug flag.

To be able to debug my node.js application I installed node-inspector (npm install -g node-inspector). As you can read in the readme of node-inspector it is really easy to get started with node-inspector. You just have to start node-inspector and then open http://127.0.0.1:8080/debug?port=5858 in your favorite WebKit based browser. On Wikipedia I found this list of WebKit based browsers. The most well known ones for Windows are Google Chrome and Safari. A screencast on node-inspector can be found here. There is also a node-inspector playlist on YouTube.

To be able to test parts of my GeometryService I used vows (npm install -g vows). Vows is an asynchronous behavior driven development framework. More info about it can be found on http://vowsjs.org/. I am now going to show a small part of the code from my GeometryService and some tests I wrote for this code. My directory structure for the code I'll show looks like this: lib/
-- datatransformer.js
test/
-- datatransformers.test.js
In datatransformer.js I started a function to convert an ESRI geometry JSON object to wkt based on its geometry type.

exports.esriGeoJsonToWKT = function esriGeoJsonToWKT (geometryType, geometry) {
  if(geometryType === "esriGeometryPoint") {
    return "POINT(" + geometry.x + " " + geometry.y + ")";
  }
}

And the content of datatransformer.test.js is:

var dt = require("../lib/datatransformer");
var vows = require('vows');
var assert = require('assert');

vows.describe('Esri geometry JSON to WKT').addBatch({
  'when converting esriGeometryPoint {"x":-117,"y":34}':{
    topic: function(){ 
      var geomType = 'esriGeometryPoint';
      var p = JSON.parse('{"x":-117,"y":34}');
      return dt.esriGeoJsonToWKT(geomType, p);
    },
    'we get "POINT(-117 34)"': function(topic){
      assert.equal(topic, "POINT(-117 34)");
    }
  },
  'but when converting esriGeometryPoint {"x":-117.01,"y":34.02}':{
    topic: function(){ 
      var geomType = 'esriGeometryPoint';
      var p = JSON.parse('{"x":-117.01,"y":34.02}');
      return dt.esriGeoJsonToWKT(geomType, p);
    },
    'we get "POINT(-117.01 34.02)"': function(topic){
      assert.equal(topic, "POINT(-117.01 34.02)");
    }
  }
}).exportTo(module);

As you can see I created two tests for converting point geometries from ESRI JSON to wkt. One for integer coordinates and one for decimal coordinates. The easiest way to run this tests with vows on windows was opening a commandline in the root directory of my node.js project and type vows --spec. This will run the tests it finds in the test and spec directories of your project. My output looks like this :

? Esri geometry JSON to WKT

when converting esriGeometryPoint {"x":-117,"y":34}
V we get "POINT(-117 "4)"
but when converting esriGeometryPoint {"x":-117.01,"y":34.02}
V we get "POINT(-117.01 34.02)"

V OK » 2 honored (0.007s)


On a side note if you ever encounter that node can't find any of your globally installed packages then it might help to add a new User Variable called NODE_PATH with as value %AppData%\npm\node_modules. That was it for this post more posts on this project will follow.

As I announced in this post, I open sourced the code of this project. It can be found in this bitbucket repository.

Creating a node.js GeometryService : part 1

Ever since I encountered the geoservices-rest-specification (pdf) I've been thinking about creating my own implementation. I also wanted wanted to try out node.js so I combined both ideas and started implementing a GeometryService in node.js. If you've never heard of node.js, this is the short introduction from the homepage of node.js:
Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices. The GeometryService is a web service that contains GIS related utility methods like project, intersect, buffer,... A sample server can be found here http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Geometry/GeometryServer.

I first downloaded and installed the windows installer of node.js version 0.65. After a reboot all locations where added to my path so that I can now start a new cmd window and type node. Note that when you install packages globally with the node package manager called npm (npm install <package> -g) npm (the node package manager) then the packages are installed in Windows 7 under %AppData%\npm (C:\Users\<username>\AppData\Roaming\npm). The full list of packages is located here search.npmjs.org.

To get started learning node I first read the node.js tutorial at nodebeginner.com. By following this tutorial I got a great insight in how to create your own webserver. At this moment I'm continuing to build my webservice based on this code but I plan to use some framework like express or journey in a later stage. This is not very urgent because I'll first focus on the JSON output of the GeometryService and thus won't need to output any html.

For my first version of the GeometryService I decided to use PostGIS as my geometry processor. I know this introduces an extra overhead. But I think this is the easiest way to get something up and running in a short period of time. If you don't want to bother with installing PostgreSQL and PostGIS I suggest you to download the Community edition of the Open Geo Suite. To be able to connect to the database I installed node-postgres with following command line: npm install pg -g. If you're on windows and get build errors you might try to add a file called true.cmd to the directory where node.exe resides (e.g. C:\Program Files (x86)\nodejs) with as content exit 0. Below is a short snippet on how I connect to the database. Make sure to replace all placeholders in the connection string.

var pg = require("pg");

function executeSQL(sql, parameters, resultCallback){
  var connectionString = "pg://username:password@host:port/databasename";
  pg.connect(connectionString, function(err, client) {
    client.query(sql,parameters, function(err, result) {
      // TODO add error handling
      console.log(result);
      resultCallback(result);
    });
  });
}

That was it for today. In the next part of the series I'll write about debugging node.js and about my progress on the GeometryService.

As I announced in this post, I open sourced the code of this project. It and can be found in this bitbucket repository.

C# REPL

Until recently when you wanted to test small snippets of C# code you had to create a small console application or use the immediate window while debugging or use something like LinqPad.

But a few weeks ago Microsoft finally announced a solution for this called Roslyn. The goal of Roslyn is to provide an API for the compiler. This is still just a CTP but it is very promising. One of the features of Roslyn is an interactive window for C# also called a REPL (read eval print loop). Note that some C# features like Linq query expressions, events and the dynamic and async keywords have not been implemented yet. More information on the Roslyn CTP can be found here Introducing the Roslyn CTP and here http://msdn.com/roslyn.

Tip of the day:
When you want to reevaluate or edit a previous entry then use ALT+Up and ALT+Down.

Bonus tip:
You might also wanna try out Jash. This is a javascript shell that can be opened on any website with a bookmarklet. It features some code completion and is very practical for trying out things you`re unsure about.

Performance of Checked vs Unchecked

If you're ever in doubt on using the checked keyword in C# or the /checked compiler directive because it might hurt your performance then don't worry anymore.

I did some integer additions on my portable in a checked and unchecked context and the performance decrease was only about 3-5% on operations of about 5 seconds. Don't forget that checked and unchecked only have an effect on integers. Float and double never check for overflows and the decimal always does. More info on this feature can be found on msdn and in this Code Project article

This is the code I used to benchmark the performance of a checked vs unchecked context. I also tested subtraction, multiplication and division with similar results. Pay great attention when using the checked context. Only inline operations are checked/unchecked, code in function calls and anonymous functions isn't.

using System;
using System.Diagnostics;

namespace GISSolved.TestConsole
{
    class Program
    {
        static void Main(string[] args)
        {
            GC.Collect(3, GCCollectionMode.Forced);
            var checkedResult = TimeIt(true);
            Console.WriteLine("Checked : {0}", checkedResult);
            GC.Collect(3, GCCollectionMode.Forced);
            var uncheckedResult = TimeIt(false);
            Console.WriteLine("Unchecked : {0}", uncheckedResult);
            Console.WriteLine("Difference : {0:0}%", (1 - ((double)checkedResult / (double)uncheckedResult)) * 100);
            Console.ReadLine();
        }
        
        static long TimeIt(bool isCheckedCalculation)
        {
            var s = new Stopwatch();
            s.Start();

            Int32 a = 123, b = 123;
            if (isCheckedCalculation)
            {
                checked
                {
                    for (int i = 0; i < 900000000; i++)
                    {
                        a = i + b + a;
                        a = 125;
                    }
                    // extra check to make sure that overflow exceptions are thrown
                    try
                    {   
                        Int32 x = a + Int32.MaxValue;
                    }
                    catch (OverflowException)
                    {
                        Console.WriteLine("Int32 Overflow");
                    }
                }
            }
            else 
            {
                unchecked
                {
                    for (int i = 0; i < 900000000; i++)
                    {
                        a = i + b + a;
                        a = 125;
                    }
                    Int32 x = a + Int32.MaxValue;
                }
            }
            s.Stop();
            return s.ElapsedMilliseconds;
        }
    }
}