Dynamics CRM and the Internet of Things - part 2

by Lucas Alexander

This is the second post in a five-part series on how I integrated a Raspberry Pi with Microsoft Dynamics CRM to recognize contacts using automobile license plates. As I mentioned in the first post of the series, the solution architecture I used is applicable to any Dynamics CRM + Internet of Things (IoT) integration. In today's post I will show how I set up my Raspberry Pi to handle the basic license plate capture and parsing operations.

Tooling

For this demonstration I am using the following tools:

  1. Raspberry Pi 2 Model B running Raspbian (Any Linux distribution that runs on the Raspberry Pi should work with the code I'm showing, but the configuration steps later might be different.)
  2. Logitech HD Webcam C615 to take photos
  3. OpenALPR to recognize license plates
  4. Node.js to provide a web-based interface to OpenALPR (and post to Socket.IO in the streaming interface)

OpenALPR and Node.js both run on Windows, so I think it should be possible to create a comparable solution using Windows 10 for IoT on a Raspberry Pi 2. Also, if you don't have a Pi or comparable IoT device, you can try this out on any system where you can run the software.

Basic Raspberry Pi configuration

Configuring the Raspberry Pi is beyond the scope of today's post, but as long as you have a Raspberry Pi running Raspbian with network access you should be good to go. My Pi is on my LAN with an IP address of 192.168.1.112, and all ports are open.

Camera

I am using a Logitech HD Webcam C615 because I just happened to have one available. Theoretically any Raspberry Pi-compatible webcam or the dedicated Raspberry Pi camera module should work. I am using the fswebcam package to interact with my webcam, and I followed the instructions here to get it working. As you'll see from the captured image at the end of this post, the quality of the images I'm capturing leaves something to be desired, but things seem to be working well enough that I haven't explored different configuration options to improve the quality.

OpenALPR

OpenALPR is the tool that parses images to find license plate numbers. You can download it from GitHub, and follow the directions in the "Easy Way" section here to build and install it. I also should be possible to use Docker as described here, but I haven't tried it myself.

Once you have OpenALPR installed, you can test that it's working using sample images like so:

wget http://plates.openalpr.com/ea7the.jpg
alpr -c us ea7the.jpg

wget http://plates.openalpr.com/h786poj.jpg
alpr -c eu h786poj.jpg

OpenALPR can also be run as a daemon that continually checks a camera stream and outputs detected license plates, but I chose not to use it in this case because the open source version never stops. If you put a license plate in front of the webcam, the open source OpenALPR daemon will just keep returning the parsed license plate number over and over again. I believe the commercial version includes a motion detector feature, but to put together this sample on the cheap, I didn't explore it.

Node.js

Having put together a few proof-of-concept interfaces using Node.js in the past, it was the first thing that came to mind to create a web-based interface to OpenALPR. You could use a variety of other options like Python, Perl, PHP or C# if you prefer, but Node.js made setting up this part of the sample extremely easy.

First you need to make sure Node.js is installed:

sudo apt-get install nodejs

Then you need to install the following modules with npm:

  1. node-uuid
  2. express
  3. socket.io

Node.js code

Here is the complete Node.js code to support all the scenarios in this series:

var http = require('http');
var express = require('express'),
    app = module.exports.app = express();
var server = http.createServer(app);
var io = require('socket.io').listen(server, {log:false, origins:'*:*'})
var uuid = require('node-uuid');
var sys = require('sys'),
    exec = require('child_process').exec;

//allow clients to directly view the images in the captures directory
app.use('/captures', express.static('captures'));

//route for the home page
app.get('/', function (req, res) {
	res.send('home page');
});

//route to handle a client calling node to check a plage
app.get('/check_plate', function (req, res) {
	//generate a guid to use in the captured image file name
	var uuid1 = uuid.v1();
	
	//tell the webcam to take a picture and store it in the captures directory using the guid as the name
	exec('fswebcam -r 1280x720 --no-banner --quiet ./captures/' + uuid1 + '.jpg',
	  function (error, stdout, stderr) {
		if (error !== null) {
		  //log any errors
		  console.log('exec error: ' + error);
		}
	});

	//now that we have a picture saved, execute parse it with openalpr and return the results as json (the -j switch) 
	exec('alpr -j ./captures/' + uuid1 + '.jpg',
	  function (error, stdout, stderr) {
		//create a json object based on the alpr output
		var plateOutput = JSON.parse(stdout.toString());
		
		//add an "image" attribute to the alpr json that has a path to the captured image
		//this is so the client can view the license plage picture to verify alpr parsed it correctly
		plateOutput.image = '/captures/' + uuid1 + '.jpg';
		
		//set some headers to deal with CORS
		res.header("Access-Control-Allow-Origin", "*");
		res.header("Access-Control-Allow-Headers", "X-Requested-With");
		
		//send the json back to the client
		res.json(plateOutput);
		
		//log the response from alpr
		console.log('alpr response: ' + stdout.toString());
		
		if (error !== null) {
		  //log any errors
		  console.log('exec error: ' + error);
		}
	});
});

//route to handle a request for a license plate capture to be written to a socket.io interface
//basically the same as the non-streaming interface except the output gets written somewhere different
app.get('/check_plate_stream', function (req, res) {
	//generate a guid to use in the captured image file name
	var uuid1 = uuid.v1();
	
	//tell the webcam to take a picture and store it in the captures directory using the guid as the name
	exec('fswebcam -r 1280x720 --no-banner --quiet ./captures/' + uuid1 + '.jpg',
	  function (error, stdout, stderr) {
		if (error !== null) {
		  //log any errors
		  console.log('exec error: ' + error);
		}
	});
	
	//now that we have a picture saved, execute parse it with openalpr and return the results as json (the -j switch) 
	exec('alpr -j ./captures/' + uuid1 + '.jpg',
	  function (error, stdout, stderr) {
		//create a json object based on the alpr output
		var plateOutput = JSON.parse(stdout.toString());
		
		//add an "image" attribute to the alpr json that has a path to the captured image
		//this is so the client can view the license plage picture to verify alpr parsed it correctly
		plateOutput.image = '/captures/' + uuid1 + '.jpg';

		//write the json to the socket.io interface
		io.emit('message', plateOutput);

		//return a response to the caller that the message was sent
		res.send('message sent');

		//log the response from alpr
		console.log('alpr response: ' + stdout.toString());
		
		if (error !== null) {
		  //log any errors
		  console.log('exec error: ' + error);
		}
	});
});

//start the server listening on port 3000
server.listen(3000, function () {
	console.log('App listening');
});

If you look at the "check_plate" route starting at line 19, you can see the script does the following things:

  1. Generate a new GUID to use in the captured image name.
  2. Take a picture with the webcam and save it to the "captures" directory.
  3. Execute OpenALPR to parse the image and return the results in a JSON object.
  4. Modify the JSON object to include the image path
  5. Return the JSON object to the client.

Trying it out

To try out the code, do the following:

  1. Save the Node.js script from above as app.js.
  2. Upload it to a directory on your Pi.
  3. Create a directory called "captures" under that directory.
  4. Start the application using the following command: nodejs app.js.
  5. Navigate to the URL for the Node.js application from a web browser.

Here's a picture of my testing "studio." The Pi and webcam are sitting on a TV tray in front of an old license plate propped up on a couch in my office.

When I call Node.js from my browser, this is what I see.

This is the JSON response formatted for easier reading.

{
	"version":2,
	"data_type":"alpr_results",
	"epoch_time":1450713975002,
	"img_width":1280,
	"img_height":720,
	"processing_time_ms":2567.216553,
	"regions_of_interest":[],
	"results":[
	{
		"plate":"43C32Y3",
		"confidence":91.097946,
		"matches_template":0,
		"plate_index":0,
		"region":"",
		"region_confidence":0,
		"processing_time_ms":269.342682,
		"requested_topn":10,
		"coordinates":[
		{
			"x":414,
			"y":315
		},
		{
			"x":812,
			"y":319
		},
		{
			"x":812,
			"y":516
		},
		{
			"x":414,
			"y":511
		}],
		"candidates":[
		{
			"plate":"43C32Y3",
			"confidence":91.097946,
			"matches_template":0
		},
		{
			"plate":"43G32Y3",
			"confidence":81.515587,
			"matches_template":0
		},
		{
			"plate":"43C3ZY3",
			"confidence":81.408203,
			"matches_template":0
		},
		{
			"plate":"43C32YS",
			"confidence":80.856506,
			"matches_template":0
		},
		{
			"plate":"43C32Y",
			"confidence":79.70826,
			"matches_template":0
		},
		{
			"plate":"43G3ZY3",
			"confidence":71.825851,
			"matches_template":0
		},
		{
			"plate":"43G32YS",
			"confidence":71.274155,
			"matches_template":0
		},
		{
			"plate":"43C3ZYS",
			"confidence":71.166771,
			"matches_template":0
		},
		{
			"plate":"43G32Y",
			"confidence":70.1259,
			"matches_template":0
		},
		{
			"plate":"43C3ZY",
			"confidence":70.018524,
			"matches_template":0
		}]
	}],
	"image":"/captures/be8da820-a7fc-11e5-a63a-7bb250f23f1c.jpg"
}

Here is the Node.js output from the Raspberry Pi command line.

This is the actual image the webcam captured. The white balance is horrible, but the plate number is clear enough for OpenALPR to recognize.

That's it for now. In my next post, I'll show how to trigger the license plate recognition functionality from a custom assembly in Dynamics CRM.