Creating a near real-time streaming interface for Dynamics CRM with Node.js

Welcome to a four-part series about creating a near real-time streaming interface for Microsoft Dynamics CRM using Node.js and Socket.IO.

In today’s post I will present an overview of how a near real-time streaming interface can be used with Dynamics CRM, and I will discuss the solution approach
In the second post, I will show how to create the Node.js component of the solution to process messages received from Dynamics CRM and send notifications to connected clients.
In the third post, I will show plug-in code to send messages from CRM to the Node.js application.
Finally in the last post, I will show how to configure a client to receive and process notifications from the Node.js application, and I’ll also discuss some general considerations related to this approach.

If you don’t want to wait that long to get to the good stuff, I’ve already posted all the code for the entire solution to GitHub here, but I encourage you to at least finish this first post before you start experimenting on your own.

What does near real-time streaming mean?

This is actually somewhat difficult to explain directly, so I think it’s better to contrast it with two frequently used approaches for integrating Dynamics CRM with external systems - batch files and web services.

Imagine you need to get data from a Dynamics CRM organization to an external system. If you want to get a list of all cases created or updated in the previous 24 hours on a nightly basis, a typical approach would be to create a process that exports CRM data to a flat file and places it somewhere the external system can pick it up and process it. Conversely, if your external system needs to get a real-time notification whenever a case is created or modified, you’d want to set up a synchronous web service call from CRM (using a plug-in or workflow).

Now think about what you do if you don’t need real-time synchronization, but you do still need something that operates closer to real time than a batch file. You could use an asynchronous workflow to trigger an outbound web service call, but this has a couple of potential drawbacks. First, the Dynamics CRM asynchronous service might introduce unacceptable latency into the update process. Second, if you have multiple clients that want to receive updates, you have to tell your code to make multiple web service calls (this is a potential problem with synchronous web service calls, too). On the other hand, a streaming API will allow for multiple clients to subscribe to receive notifications as they are processed. This processing time that occurs between the data updates in CRM and the delivery of the notifications to the connected clients are what make this a near real-time interface.

The video below shows an example of the interface at work. In the Dynamics CRM UI on the left, I am making updates to a contact record. These updates cause a plug-in to post messages to the Node.js application. On the right side of the screen, I have a web page and a console application connected to the Node.js application via Socket.IO. When the Node.js application receives a message from the Dynamics CRM plug-in, it sends that message to any clients connected to the socket.

Why Node.js?

Now that I’ve explained exactly what it is I’ll be showing how to build, the next point to discuss is why I chose to use Node.js. But maybe the better question with which to start is “what is Node.js?”

According to Wikipedia:

Node.js is an open source, cross-platform runtime environment for server-side and networking applications. Node.js applications are written in JavaScript, and can be run within the Node.js runtime on OS X, Microsoft Windows, Linux and FreeBSD.

Node.js provides an event-driven architecture and a non-blocking I/O API that optimizes an application's throughput and scalability. These technologies are commonly used for real-time applications.

As a result of its underlying architecture, a Node.js application is perfectly suited to delivering low-latency notifications to connected clients and because it’s written in JavaScript, there’s not a challenging learning curve to get started with it.

Wrapping up

That’s all for today. I hope I’ve piqued your interest in using Node.js for near real-time notifications from Dynamics CRM. Next time I’ll show how to build the Node.js component of the solution.

A version of this post was originally published on the HP Enterprise Services Application Services blog.