Building a custom Dynamics 365 data interface with OpenFaaS

by Lucas Alexander

Over the past several months, I've been doing a lot of work with OpenFaaS in my spare time, and in today's post I will show how you can use it to easily build and deploy a custom web service interface for data in a Dynamics 365 Customer Engagement online tenant.

OpenFaaS

If you're not familiar with OpenFaaS, it's basically a serverless functions platform like Azure Functions or AWS Lambda, but you run it on Kubernetes or Docker Swarm on your own servers or in the cloud. What I particularly like about OpenFaaS compared to the various commercial serverless platforms is that in addition to offering more control over how/where it's deployed, OpenFaaS supports a wider variety of languages for writing serverless functions.

OpenFaaS (Functions as a Service) is a framework for building serverless functions with Docker and Kubernetes which has first class support for metrics. Any process can be packaged as a function enabling you to consume a range of web events without repetitive boiler-plate coding.

To follow along with the samples in this post, you'll need access to a cluster with OpenFaaS deployed, so if you don't already have one, now would be an excellent time to look at the OpenFaaS deployment docs or maybe even work through the hands-on workshop. I've also previously written about how to securely deploy OpenFaaS on a free Google Cloud VM with Docker Swarm or on an Azure Kubernetes Service cluster.

Preparing to build the interface function

As soon as you have OpenFaaS running, it's time to look at the actual custom interface function.

My demo C# function does the following:

  1. Parse a JSON object sent in the client request for an access key and optional query filter
  2. Validate the client-supplied access key to authorize or reject the request
  3. Retrieve a Dynamics 365 OAuth access token using my Azure AD OAuth 2 helper microservice
  4. Execute a query for contacts against the Dynamics 365 Web API
  5. Return the Web API query results to the client in an array as part of a JSON object

Because the OpenFaaS function uses my OAuth helper microservice instead of requesting an OAuth access token directly from Azure Active Directory, you need to deploy that microservice to your cluster before moving forward.

If you're using Kubernetes, you can create the deployment and corresponding service using the following YAML. You'll need to set the RESOURCE environment variable to the FQDN for your Dynamics 365 CE organization, but you can leave the CLIENTID and TOKEN_ENDPOINT values alone. (While I used to think you needed to register a separate client application for every Dynamics 365 org to use OAuth authentication, I recently learned via a Twitter conversation that there is a "universal" CRM client id you can use instead.)

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: azuread-oauth2-helper
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: azuread-oauth2-helper
    spec:
      containers:
      - name: azuread-oauth2-helper
        image: lucasalexander/azuread-oauth2-helper
        ports:
        - containerPort: 5000
        env:
        - name: RESOURCE
          value: "https://XXXXXXXX.crm.dynamics.com"
        - name: CLIENTID
          value: "2ad88395-b77d-4561-9441-d0e40824f9bc"
        - name: TOKEN_ENDPOINT
          value: "https://login.microsoftonline.com/common/oauth2/token"
---
apiVersion: v1
kind: Service
metadata:
  name: azuread-oauth2-helper
spec:
  ports:
  - port: 5000
  selector:
    app: azuread-oauth2-helper

Once you've deployed the microservice, here's the definition for a Kubernetes ingress. In this case my microservice is accessible on the same host as OpenFaaS (akskube.alexanderdevelopment.net), and it is secured with the same Let's Encrypt certificate. You'll want to update your configuration with the appropriate values for your specific situation.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: azuread-oauth2-helper-ingress
  annotations:
    kubernetes.io/tls-acme: "true"
    certmanager.k8s.io/issuer: letsencrypt-production
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  tls:
  - hosts:
    - akskube.alexanderdevelopment.net
    secretName: faas-letsencrypt-production
  rules:
  - host: akskube.alexanderdevelopment.net
    http:
      paths:
      - path: /oauthhelper
        backend:
          serviceName: azuread-oauth2-helper
          servicePort: 5000

After the OAuth helper microservice is deployed, you should validate that you can get a token returned for a valid username/password combination. Here's what that looks like in Postman.

Building the interface function

If you've made it to this point, building and deploying the function is easy!

First the function gets its configuration data from environment variables that are set when the function is deployed. If you were actually using this function in production, it would be better to store sensitive values like the access key and the Dynamics 365 password as secrets, but I've used environment variables here to keep this overview as simple as possible.

//get configuration from env variables        
var username = Environment.GetEnvironmentVariable("USERNAME");
var userpassword = Environment.GetEnvironmentVariable("USERPASS");
var tokenendpoint = Environment.GetEnvironmentVariable("TOKENENDPOINT");
var accesskey = Environment.GetEnvironmentVariable("ACCESSKEY");
var crmwebapi = Environment.GetEnvironmentVariable("CRMAPI");

After the function gets its configuration data, it deserializes the client request using Json.Net to extract a client-supplied access key and an optional query filter. The client-supplied key is validated against the stored key value, and if they don't match, an error response is returned.

var queryrequest = JsonConvert.DeserializeObject<QueryRequest>(input);

if(accesskey!=queryrequest.AccessKey)
{
    JObject outputobject = new JObject();
    outputobject.Add("error", "Invalid access key");
    Console.WriteLine(outputobject.ToString());
    return;
}

After the access key is validated, the function then makes a request to the authentication helper microservice to get an access token.

var token = GetToken(username, userpassword, tokenendpoint);

...
...
...

string GetToken(string username, string userpassword, string tokenendpoint){
    try
    {
        JObject tokencredentials = new JObject();
        tokencredentials.Add("username", username);
        tokencredentials.Add("password",userpassword);
        var reqcontent = new StringContent(tokencredentials.ToString(), Encoding.UTF8, "application/json");
        var result = _client.PostAsync(tokenendpoint, reqcontent).Result;
        var tokenobj = JsonConvert.DeserializeObject<Newtonsoft.Json.Linq.JObject>(
            result.Content.ReadAsStringAsync().Result);
        var token = tokenobj["accesstoken"];
        return token.ToString();
    }
    catch(Exception ex)
    {
        return string.Format("Error: {0}", ex.Message);
    }
}

Once the token is returned from the microservice, the function executes the Web API query. The query is just a hardcoded OData query in the form of /contacts?$select=fullname,contactid plus any filter supplied by the client. The function expects that the filter will also be provided in supported Dynamics 365 OData format like startswith(fullname,'y').

var crmreq = new HttpRequestMessage(HttpMethod.Get, crmwebapi + crmwebapiquery);
crmreq.Headers.Add("Authorization", "Bearer " + token);
crmreq.Headers.Add("OData-MaxVersion", "4.0");
crmreq.Headers.Add("OData-Version", "4.0");
crmreq.Headers.Add("Prefer", "odata.maxpagesize=500");
crmreq.Headers.Add("Prefer", "odata.include-annotations=OData.Community.Display.V1.FormattedValue");
crmreq.Content = new StringContent(string.Empty.ToString(), Encoding.UTF8, "application/json");
var crmres = _client.SendAsync(crmreq).Result;

var crmresponse = crmres.Content.ReadAsStringAsync().Result;

var crmresponseobj = JsonConvert.DeserializeObject<Newtonsoft.Json.Linq.JObject>(crmresponse);

Finally results are returned to the client in an array as part of a JSON object.

JArray outputarray = new JArray();
foreach(var row in crmresponseobj["value"].Children())
{
    JObject record = new JObject();
    record.Add("id", row["contactid"]);
    record.Add("fullname", row["fullname"]);
    outputarray.Add(record);
}
JObject outputobject = new JObject();
outputobject.Add("contacts", outputarray);
Console.WriteLine(outputobject.ToString());

Here's the complete function.

using System;
using System.Text;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Collections.Generic;

namespace Function
{
    public class FunctionHandler
    {
        private static HttpClient _client = new HttpClient();

        public void Handle(string input) {
            //get configuration from env variables        
            var username = Environment.GetEnvironmentVariable("USERNAME");
            var userpassword = Environment.GetEnvironmentVariable("USERPASS");
            var tokenendpoint = Environment.GetEnvironmentVariable("TOKENENDPOINT");
            var accesskey = Environment.GetEnvironmentVariable("ACCESSKEY");
            var crmwebapi = Environment.GetEnvironmentVariable("CRMAPI");
            
            //deserialize the client request
            var queryrequest = JsonConvert.DeserializeObject<QueryRequest>(input);
            
            //validate the client access key
            if(accesskey!=queryrequest.AccessKey)
            {
                JObject outputobject = new JObject();
                outputobject.Add("error", "Invalid access key");
                Console.WriteLine(outputobject.ToString());
                return;
            }

            //get the oauth token
            var token = GetToken(username, userpassword, tokenendpoint);
            
            if(!token.ToUpper().StartsWith("ERROR:"))
            {
                //set the base odata query
                var crmwebapiquery = "/contacts?$select=fullname,contactid";
                
                //add a filter if the client included one in the request
                if(!string.IsNullOrEmpty(queryrequest.Filter))
                    crmwebapiquery+="&$filter="+queryrequest.Filter;
                try
                {
                    //make the request to d365
                    var crmreq = new HttpRequestMessage(HttpMethod.Get, crmwebapi + crmwebapiquery);
                    crmreq.Headers.Add("Authorization", "Bearer " + token);
                    crmreq.Headers.Add("OData-MaxVersion", "4.0");
                    crmreq.Headers.Add("OData-Version", "4.0");
                    crmreq.Headers.Add("Prefer", "odata.maxpagesize=500");
                    crmreq.Headers.Add("Prefer", "odata.include-annotations=OData.Community.Display.V1.FormattedValue");
                    crmreq.Content = new StringContent(string.Empty.ToString(), Encoding.UTF8, "application/json");
                    var crmres = _client.SendAsync(crmreq).Result;
                    
                    //handle the d365 response
                    var crmresponse = crmres.Content.ReadAsStringAsync().Result;

                    var crmresponseobj = JsonConvert.DeserializeObject<Newtonsoft.Json.Linq.JObject>(crmresponse);
                    
                    try
                    {
                        //build the function response
                        JArray outputarray = new JArray();
                        foreach(var row in crmresponseobj["value"].Children())
                        {
                            JObject record = new JObject();
                            record.Add("id", row["contactid"]);
                            record.Add("fullname", row["fullname"]);
                            outputarray.Add(record);
                        }
                        JObject outputobject = new JObject();
                        outputobject.Add("contacts", outputarray);
                        
                        //return the response to the client
                        Console.WriteLine(outputobject.ToString());
                    }
                    catch(Exception ex)
                    {
                        JObject outputobject = new JObject();
                        outputobject.Add("error", string.Format("Could not parse query response: {0}", ex.Message));
                        Console.WriteLine(outputobject.ToString());
                    }
                }
                catch(Exception ex)
                {
                    JObject outputobject = new JObject();
                    outputobject.Add("error", string.Format("Could not query data: {0}", ex.Message));
                    Console.WriteLine(outputobject.ToString());
                }
            }
            else
            {
                JObject outputobject = new JObject();
                outputobject.Add("error", "Could not get token");
                Console.WriteLine(outputobject.ToString());
            }
        }

        string GetToken(string username, string userpassword, string tokenendpoint){
            try
            {
                JObject tokencredentials = new JObject();
                tokencredentials.Add("username", username);
                tokencredentials.Add("password",userpassword);
                var reqcontent = new StringContent(tokencredentials.ToString(), Encoding.UTF8, "application/json");
                var result = _client.PostAsync(tokenendpoint, reqcontent).Result;
                var tokenobj = JsonConvert.DeserializeObject<Newtonsoft.Json.Linq.JObject>(
                    result.Content.ReadAsStringAsync().Result);
                var token = tokenobj["accesstoken"];
                return token.ToString();
            }
            catch(Exception ex)
            {
                return string.Format("Error: {0}", ex.Message);
            }
        }
    }

    public class QueryRequest
    {
        public string AccessKey {get;set;}
        public string Filter{get;set;}
    }
}

Because the function relies on Json.Net, you need to add a reference to it in your .csproj file before you build the function.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>
  <PropertyGroup>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="newtonsoft.json" Version="11.0.2" />
  </ItemGroup>
</Project>

Here is my function definition YAML file with enviroment variables included. You will need to update them with your appropriate values, and you will also need to change the image name if you're building your own function instead of just deploying mine from Docker Hub.

provider:
  name: faas
  gateway: http://localhost:8080

functions:
  demo-crm-function:
    lang: csharp
    handler: ./demo-crm-function
    image: lucasalexander/faas-demo-crm-function
    environment:
      USERNAME: XXXXXX@XXXXXX.onmicrosoft.com
      USERPASS: XXXXXX
      TOKENENDPOINT: https://akskube.alexanderdevelopment.net/oauthhelper/requesttoken
      CRMAPI: https://lucastest20.api.crm.dynamics.com/api/data/v9.0
      ACCESSKEY: MYACCESSKEY

Once the function is deployed, you can execute it either through the OpenFaaS admin UI or another tool that makes HTTP requests like Curl or Postman. Here's what an unfiltered query in Postman looks like for a Dynamics 365 org with sample data installed.

And here's a query with a filter included.

Wrapping up

Once I got OpenFaaS running, writing and deploying the actual function only took about an hour. Obviously writing a more complex data interface to support real-world requirements would take longer, but using a serverless functions platform like OpenFaaS is definitely a significant accelerator for custom Dynamics 365 integration development.

What do you think about this approach? Are you using serverless functions with your Dynamics 365 projects? What do you think about OpenFaaS vs Azure Functions or AWS Lambda? Let us know in the comments!