For the last several months I've been working on an enterprise Dynamics CRM project where one of our goals is to minimize the amount of custom code we write by using North52's Business Process Activities. I had not been exposed to North52 before working on this project, but I have been pleasantly surprised with how much it has allowed our mostly functional resources to achieve without needing technical assistance.

While looking through North52's documentation a while back, I noticed it could be used to call a REST web service. This got me thinking about how I could rework my Sentiment analysis in Dynamics CRM using Azure Text Analytics sample using North52 instead of a custom workflow activity.

I found I was able to replace the custom workflow activity with a North52 Process Genie. It executes a Smart Flow to call the Azure Machine Learning Text Analytics API and then return the result to the calling CRM dialog.

Here's the formula I used in my Process Genie:

SmartFlow(
  SetVar('jsoninput', CreateJObject( 
     CreateJProperty('Inputs', 
           CreateJArray(CreateJObject(
                           CreateJProperty('Id', '1'),  
                           CreateJProperty('Text', [account.texttoanalyze]) 
                              )
                        )
                     )
               )
          ),

  CallRestAPI(
      SetRequestBaseURL('https://api.datamarket.azure.com/data.ashx/amla/text-analytics/v1'),
      SetRequestResource('/GetSentimentBatch'),
      SetRequestDetails('POST'),
      SetRequestHeaders(),
      SetRequestParams('RawContentTextJSON',GetVar('jsoninput')),
      SetRequestAuthenticationBasic('AccountKey','YOUR_AZURE_ML_API_KEY_HERE'),
      SetRequestFiles(),
      SetRequestExpected('OK'),
      SetRequestActionPass(SetVar('result', GetVarJsonValue('SentimentBatch{0}.Score'))),
      SetRequestActionFail(SetVar('result', 'ERROR' + GetVarJsonValue('Errors{0}.Message')))
    ), 

   SmartFlowReturn(GetVar('result'))
)

Here's a screenshot of my CRM dialog:

To execute the Process Genie, you use a North52 N52 Process Genie step like this:

The Formula ShortCode value is the short code of the North52 Process Genie. The Formula Parameter Xml value contains the text to analyze from the dialog input in the format expected by the formula:

<account><texttoanalyze>{Response Text(Get text)}</texttoanalyze></account>

If were going to use this in real solution, I would not hardcore the Azure ML API key directly in the formula. Other than that I think this is a production-ready approach.

What do you think? Would you consider using something like this as an alternative to writing your own custom code?

Earlier this year I wrote a post that showed how to perform sentiment analysis in Dynamics CRM using Microsoft Azure Text Analytics. Azure Text Analytics makes it incredibly easy to use sentiment analysis (with English text only), but the full Azure Machine Learning offering is much more powerful. In today's post I will show how to create a custom predictive web service in Azure ML and make predictions with it in Dynamics CRM.

Why am I doing this?

One of the exciting announcements about Dynamics CRM 2016 is that it includes some sort of integration with Azure ML, so what's the point of this blog post? Actually, here are two reasons:

1. It's not clear (as of late November 2016) what capabilities the standard CRM-Azure ML integration will have. The approach demonstrated here allows for total customization.
2. The approach below can be used right now in any version of CRM (online or on-premise) from CRM 2011 onward.

The demo data set

For this demonstration I am using data from the AdventureWorks data warehouse sample database to build a model to predict whether a contact in CRM is likely to be a bicycle buyer. I created a flat file of contacts that includes several independent variables and a yes/no flag for whether they purchased a bicycle. That sample file is included in my solution source linked at the bottom of this post.

You can also download the full AdventureWorks database from CodePlex here.

Getting started with Azure ML

As I was trying to get the hang of Azure ML, I found the Bluewater SQL blog tremendously helpful. If you're just getting started with Azure ML, I suggest you take a look at both of these posts before continuing. (These two posts also use the AdventureWorks data warehouse sample database, though I had already decided to use it for my demo before I found this site. I guess the AdventureWorks data is an obvious choice for this sort of thing.)

1. https://bluewatersql.wordpress.com/2014/07/31/azure-machine-learning-first-look/
2. https://bluewatersql.wordpress.com/2014/08/01/azure-machine-learning-a-deeper-look/

Creating the predictive web service

Now that the introductory bits are out of the way, here's an overview of how I set up my predictive web service in Azure ML. I've also shared my work in the Cortana Analytics Gallery, so if you just want to see how the experiments are set up without going through all the steps yourself, skip to the end of this post for the gallery links.

1. Export your data set from CRM. I use a simple KingswaySoft SSIS package to do this. It's included in my sample project files.
2. Upload your data set to Azure ML Studio. I am using the Azure ML free tier, which requires me to manually upload the data set for the initial upload and any subsequent updates. If I were using the standard tier, it would be possible to upload the data set to Azure blob storage as part of my SSIS package. This would be useful for periodically retraining the model with updated data in the future.
3. Create and run a new experiment to classify bike buyers. I basically followed the steps outlined in the two Bluewater SQL posts, so take a look at those if you want to see a more detailed explanation.
4. Save the trained model from this experiment to use in your predictive web service.
5. Create a new experiment that will serve as the foundation for the predictive web service
6. Add the following items to the new experiment canvas:
7. Data set
8. Trained model from the earlier experiment
9. Score model component
10. Project columns transformation between the data set and score model component to exclude the lpa_bikerbuyername column
11. Project columns transformation after score model component to only include score labels
12. Run the experiment.
13. Create a predictive web service.
14. Run the experiment.
15. Deploy the web service.
16. Open the web service and copy the API key to use later.
17. Open the request/response page.
18. Copy the post URL to use later. Leave off the "&details=true" at the end.
19. Scroll down to find the JSON request/response samples and copy them for review later.

Making predictions from CRM

Once your predictive web service is set up, using it in Dynamics CRM is as simple as posting a JSON request to the web service and then parsing the JSON response. A few years ago, I wrote a code sample showing an easy way to make JSON requests from CRM custom assemblies that serves as the foundation for the workflow activity I'm using in this demo. There are only a few changes required to that sample:

1. Add an authorization header that includes the bearer API key you copied earlier.
2. Add input/output parameters that match the inputs and outputs for your predictive web service. I am also passing the API key and service endpoint as input parameters instead of hardcoding them.
3. Modify the JSON request/response classes to allow for serialization to/deserialization from the request and response messages. Because of how Azure ML passes the inputs/outputs, these are actually fairly generic, so you can probably use mine with no or minimal changes.
4. Send the correct request parameters and correctly handle the response parameters.

To demonstrate the custom workflow activity, I created a sample CRM dialog.

Here's where I supply the input parameters.

Here's what the dialog looks like when I run it from a contact record.

The code

You can get all the code for this demonstration from my GitHub repository here. The sample code includes the following:

1. CRM solution containing:
2. Contact entity with custom fields added and a system view for data export for Azure ML
3. Sample dialog (you would need to update the parameters with your own API key and endpoint details)
4. Compiled custom workflow activity to call the web service
5. Contact data ready for import to CRM
6. SSIS package solution for data export
7. Custom workflow activity source code

I've also shared my training and prediction experiments in the Cortana Analytics Gallery. Here are the links where you can open and import them into your own workspace:

1. Training experiment
2. Prediction experiment

What do you think about this approach? Does it seem like it would be useful to you in a real-world situation? Let me know in the comments!

Last year I created a proof-of-concept solution that showed how to integrate Dynamics CRM with HP Haven OnDemand (then called HP IDOL OnDemand) to perform sentiment analysis and index records to support "find similar" queries. While I was working through the AzureCon challenge a few weeks ago, I thought it would be an interesting exercise to update my sentiment analysis code to work with the Text Analytics offering from the Microsoft Azure Marketplace.

The approach

As with my Haven OnDemand solution, the approach I'm using with Azure relies on a custom workflow activity that does the following:

1. Parse a supplied text input and strip any HTML tags using a helper function.
2. Create a JSON sentiment analysis request and post it to Azure Text Analytics with an HttpWebRequest.
3. Deserialize the JSON respsonse returned by Azure Text Analytics to a custom class object using a DataContractJsonSerializer.
4. Return the sentiment score to the calling process.

There are two main differences with the my Azure Text Analytics solution:

1. The Azure service only returns a sentiment score, so this custom workflow activity doesn't return a positive/negative string value.
2. Instead of embedding an access key in the custom workflow activity code, I've made it a parameter, which means you can take the solution straight from GitHub and start using it in your CRM organization as soon as you sign up for the Text Analytics service.

The solution in action

Here's a sample dialog I've created to demonstrate the use of the custom workflow activity.

Here's how the process works for the sample text "I hate you."

Wrapping it up on a more positive note, here's the same dialog with "I love you" instead.

As you can see, the score for "I hate you" is about .06, and the score for "I love you" is about .91, which makes sense as scores closer to one are more positive, and scores closer to zero are more negative.

The code

You can download all the custom code and a CRM solution extract from my Crm-Sample-Code repository on GitHub. Let me know what you think in the comments!