In this final (unless I need to expand on the scenarios) part of the story, Flow will be used to take the information garnered from the user and create a work order.
Calling a Sub-flow
The original flow was all about Alexa, but what about other voice assistants? Flow is no different that any other programming languages and as such we can use the same concepts to writing decent flows, one of them being re-usability. If the code to write the Work order and book it is generic, it can be re-used when Google or Siri is connected to the application.
To this end, the Flow starts with another HTTP trigger, which is called from the previous flow.
Just like when connection Flow to Alexa, the URL is required in the child to update the caller. Create a new Flow, using the When a HTTP request is received. As content to the JSON Schema, add just enough to get you going and use the output to send an email so that the schema coming from the parent flow can be seen. This is a repart of the logic used to start our parent flow.
Once saved, the URL will be generated in the trigger step. Use this to call the child Flow from the parent. This is the HTTP action. The Method is a POST, the URI is the trigger URL defined in the child Flow. No headers are required. In the body, add in the things that are already known, just to ensure repeat calls to D365 are made. The Intent is also passed in, which has all the details about what the user wanted.
Now ready for testing, ensure both Flows are in Test mode and trigger Alexa. After a little delay, an email should be sent in the child Flow detailing enough information to create a Work Item.
Use this email to populate the JSON as previously, easily creating the schema that is required.
Creating a Work Order
Next, get the Account the contact is associated with. This is done with a call to the D365 Instance using the Common Data Service Connector.
A Work Order needs some fields to be set before it can be save, Work Order Type is one of them. This could be hard coded, but Alexa has supplied the intent. To match the Work order type with the intention in Alexa, a field on the Work Order Type was added, Alexa Intent, which is searched for in the CDS List Records action.
To make the Flow easier to manage and reduce the dual looping, the Work Order Type is returned to a variable
Once the data for the new Work Order is available, create the Work Order using the CDS Create Record connector.
Most of these are obvious, but the one that is not is the Work Order Number. In Field Service, this is automated, with a prefix and a number range. As this doesn’t work in Flow, a work number is generated using a random number, using an expression.
A few other parts of the work order are populated, helping the service manager to match the time as appropriate.
In this post I will be linking Alexa to D365 and returning some information back to the end user.
Receiving information from Alexa
Alexa interacts with the outside world with a HTTPS request. Once Alexa has determined that it understands the user and they have asked for something of your skill, it posts to the web service you have configured.
That API Guy had a great post where he links his Alexa to his O365 email account and has Alexa read out his new email. This article steps through linking Alexa and Flow, and it showed me how simple that part of the integration is. Microsoft has done the hard work by certifying it’s connections, you just need to create one.
Flow provides several methods of subscribing to HTTP events, but the one we are interested in is at the bottom
The URL is generated for you, and is the bit you need to post into Alexa Skill configuration once we have created the Flow. In the JSON schema, we just want a stub for now. The schema is defined by Alexa and how you configure the skill. It is best to get the connection up and running and use the initial call to substitute the schema later.
Every Flow needs at least one action to be able to save, so to aid testing and understand the JSON sent by Alexa, the first step is to send an email to myself with the content of the call.
Saving the flow and go back to the trigger
A specific trigger URL for our Flow is now generated.
Back in Alexa
In Alexa, on the left in the Skill is Endpoints. Alexa allows different endpoints to the skill depending on the Alexa region as well as a fall back. As this is a POC, using the default is appropriate.
The important part of this is the drop down below the URL copied from Flow, this needs to be the second option “My development endpoint is a sub-domain of a domain that has a wild card certificate from a certificate authority“. Basically, Microsoft has certified all their Flow endpoints with wild card certificates, allowing Amazon to trust that it is genuine.
One saved, Build your skill again. I found every time I touched any configuration or indeed anything in Alexa, I needed to rebuild.
Ready to test. Alexa allows you to test your connection via the Test tab within your skill.
You have to select Development from the drop down. You can also use this interface to check Production skills.
A word of warning, everytime you build, to test that new build, I found I had to re-select Development in this window by toggling between “Off” and “Development”.
Back in Flow, test your flow by confirming that you will conduct the action.
In the Test panel in Alexa, enter a phrase with your Invocation and Utterance in Alexa entry and if Alexa understands it is for your invocation, your flow should be triggered! Alexa will complain, as our simple flow hasn’t returned anything to it. We’ll worry about that later.
In our simple Flow, I used email as the action, as can be seen below.
This is the raw JSON output of the Alexa skill and this is used to tell Flow what to expect. This way it will provide the rest of the Flow with properties that are more appropriate.
Back in the Flow trigger, select Use sample payload to generate schema, which presents the text entry, where you paste in the email body.
Flow does the hard work now and presents you with a schema for what Alexa is sending your Flow
Authenticating the user
Taking triggers from Alexa, hence a user is all well and good, but they expect a rapid response. Alexa Skills are public domain once you publish it, anyone can enable your skill, hence a little work needs to be done to understand who is calling our skill and whether Big Energy Co supports them.
Which User is it?
The request from Alexa does include a unique reference of the user, but normally businesses like Big Energy Co work on email addresses, which you can get from Alexa, but the user has to give you permission. This can be pre-approved when the end user installs the skill or you can ask for approval.
Alexa needs to be told that your skill would like this permission. This allows Alexa to request this permission for you when your skill is enabled by the user
On the left hand menu, there is a Permissions link, where our skill asks for the email address
To ask Alexa for an users email address is a seperate web call with properties from original message sent from Alexa in the trigger.
Alexa provides each session an access token so that Flow can ask for more information from Alexa in the context of the users session, location, names, email etc. As this is a globally distributed system, the api End point can vary depending on where the user started their session. The URI at the end asks for the email address of the user of the session.
Alexa may refuse to return the email address, because the end user has not given permission for our skill to share this information. If the call to get the email was a success, it means that Alexa has returned the value and the Flow can continue
Without the permission, the call fails and so the Flow captures this. Remember to set a run after for this step to ensure it still runs on failure. Otherwise the flow will fail not so gracefully.
Alexa provides standard permission request functionality, the Flow responds to the original call with this detail in a JSON formatted string, with the permissions the Skill wants listed. Finally, for this failure, let the Flow terminate successfully. The Flow can’t continue without the information.
Does Big Energy Co support the user?
It is not enough that the user has asked us for help, they need to be known to Big Energy, as a contact. Querying D365 for this data is the next step
Using the D365 connector, query the contact entity for the email address that Alexa has given us.
A conditional flow checks to see if the return from the Retrieve Contact step has a single record using an expression below
Responding to the user, quickly
Like any Alexa Skill, the user expects an immediate response. Flow in itself is quick, but when it comes to updating or creating records, it can take seconds. The timeout for your response to Alexa is 10 seconds, which isn’t a lot when you want to look up several things to create the appropriate work order.
To get around this, respond to the user once you know you have all the information you need to create the appropriate records in D365. Here, the Flow responds if the contact is in our database. In production, you could do some more checks or just assume that if the contact is known, the logic could create an opportunity to sell them a contract if nothing else. Equally, terminate gracefully after responding that Big Energy Co doesnt know the customer, prompting them to ring the service desk.
In the switch statement, a response to the user is built up. Specific, personalised responses using the data you have retrieved is essential to give the customer an understanding that you have received the request and will respond.
The responses are short and to the point but personalised to the customer and what they asked for with some expressions to add more information if they told Alexa.
This snippet adds after “request for a Service” what the user asked for a service on, relying on the JSON formatted values.
if(equals(triggerBody()?['request']?['intent']?['slots']?['device']?['value'], ''), '', concat(' for your ',triggerBody()?['request']?['intent']?['slots']?['device']?['value']))
This snippet adds to the end a sentence including the date that the user asked for.
if(equals(triggerBody()?['request']?['intent']?['slots']?['date']?['value'],''),'',concat( 'We will endeavour to send an engineer on ',triggerBody()?['request']?['intent']?['slots']?['date']?['value']))
Finally, for the Alexa response part, the Flow returns a response to the user. This combines the body with a little standard text. This is what Alexa will say. You can also add a response to the screen for those devices able to do that.
The final part of this flow goes on and creates the work order. I seperated out the flows using a sub flow, which is discussed in the next part of the blog.
As we all now have smart devices in our home, linking them to business applications could be a key differentiator between winners and the also rans. This series of posts will demonstrate how I connected Alexa to D365 Field service.
Our generic big energy company is diversifying into support and maintenance of home energy products, boilers, central heating, plumbing, electricals and numerous other aspects of a consumer’s home life. A client will ring up, tell the support desk that they have an issue with the appliance and the employee will book a suitable engineer in to come out to their home. This is all done via Field Service in D365. Big Energy also have scheduled servicing of Boilers on an annual basis.
What the CTO wants to do is embrace the in home virtual assistant to promote Big Energy as a forward thinking organisation which is at the forefront of technology to allow Big Energy’s customers to book an engineers visit via their smart device. Her expectation is that Alexa, Google Home or Siri will allow a service call to be booked without talking to the support desk. This will allow meaningful interactions with their customers 24/7.
The proof of concept will start with the front runner in the war of virtual assistance, mostly because I have one.
If you have read my introduction to LUIS in Alexa Skill concepts are a mirror of the concepts introduced in LUIS. Alexa has Utterances & Intents. Entities are called Intent Slots for Alexa. Alexa has also got another concept, that being the Invocation.
First off, get yourself an Amazon account & sign up for the developer program at developer.amazon.com. This is all free. People can buy skills, but not sure Big Energy’s customers would think this is appropriate.
Use the Create Skill button, enter a name & select a language. I also choose Custom & Provision your own to allow us to send data to Flow which is hosting our integration.
Then I select Start from scratch, none of the others seem to match our scenario
An Invocation is the starting point and is the differentiator between you and every other Alexa skill out there. It is effectively a name for your skill, what the user would say to call your skill. I have slightly altered my invocation to put some spaces in there so it is more natural to the end user.
Like in LUIS, intents are a category or what the user is looking for or asking. You can have many intents per skill. In our scenario around home appliances the user can ask for a service, a repair or an emergency as a starting point. Let’s start with a request for service.
Utterances are samples to train Alex to understand the intent in natural language. Add as many utterances as you like as samples of how a person would ask for a service or repair, but ensure they are different.
In my sample utterances above you can see I have added Slots. Slots are the same as entities in LUIS, data that the user gives us when they are saying their utterance in addition to the type of thing they want.
Each Slot has a type, and I have used the default Amazon types except for device, which is a custom one.
Amazon has 43 of it’s own list types or 6 built in types for numbers & dates, but devices are not in the list. I want to know what type of thing the engineer is going out to fix, not sure I need to be that specific, but would be good to know if I need to send an electrician, gas engineer or plumber. I have added my own Slot Type, called it device and I now list the values I expect.
You also should also enter synonyms, not everyone calls it a telly, just us northeners.
Once you have entered enough intents & utterances, time to build and test. A nice message lets you know when it is done building. Ready for the utterance profiler or testing
In the top right, is a pop out to test your utterances. Enter an utterance you haven’t used before to check you are getting the expected results.
The text I wrote is in blue, it has decided that this is a service intent, with a device of boiler & the day of wednesday.
You can carry on fine tuning your utterances, intents and slots to get the most accurate model. Like any language understanding, this will be an ongoing model.
This is Alexa done, we have configured everything we need to in Alexa, apart from our link to Flow. This needs a bit of pre-work in Flow to activate, in the next post.