January 25, 2017

Launch a Script Using Alexa Voice Commands

In a previous post, I showed how you can build a smart mirror with an Alexa voice assistant on board. The MagicMirror software and Alexa voice assistant were both hosted on a Raspberry Pi, but unfortunately there was no obvious way to get Alexa to control the smart mirror, or deliver commands to the Raspberry Pi.

I have now found a solution that is free, reliable, and very flexible. This is done by writing an Alexa Skill that adds a message to a cloud hosted queue based on your voice command. The Raspberry Pi repeatedly checks this queue for new messages, and runs customizable behaviour based on message contents. This is not limited to smart mirror applications, or Raspberry Pis. It can be used to launch any script you want on any platform that will connect to Amazon's SQS.

Here is a demonstration, and high level overview of how it works:



and a follow up demonstrating an extension of this idea:



In this tutorial I will focus on just using this to simply turn the smart mirror on and off. Adding your own scripts should then be fairly straight forward,

The steps will be as follows:
  1. Create a Queue using the Amazon Simple Queue Service (SQS)
  2. Write some python code to read and write to this queue
  3. Write a Lambda Function that posts messages to the queue
  4. Write an Alexa Skill that calls the Lambda Function
  5. Schedule a task on your Raspberry Pi to read queue messages and take appropriate action.

Creating a Queue


Creating an Amazon SQS queue is very straightforward. Just go to the SQS website, create an Amazon AWS account if needed, and click "Create New Queue". Name it whatever you want, and stick with the default settings. Personal use will also be infrequent enough that we will remain in the free tier.

Once the Queue is create, you simply need to note the url. This can be done by clicking on the queue in your SQS console, selecting the details tab, and reading the URL: property.



You will also need to create an amazon IAM user to get an api access key, and access secret for accessing the Queue. That can be done by visiting the IAM page, and clicking the "Users" tab, followed by "Add User".

Then you click the new user's name, and find the tab "Security Credentials" which will have a "Create access key" button. Click this button. This will open a dialogue window with the option to display access secret, and download csv. Download this file, or copy paste the access key and access secret into a text file.

Reading and Writing to Queue with Python


The only package you'll need beyond basic python is called boto3, so you will need to run $> python -m pip install boto3 to make sure this is installed. 

Add your access key and secret, as well as queue url, and the region (which will be a substring in your queue url, such as "us-east-1") to the following code example.


This sets up your credentials, adds the message "test" to your queue, and then reads a message from the queue and deletes the message. This should print the word "test". Read this code over to ensure you understand what is going on. If you need some python brushing up, checkout my introduction post.

This code is just to define the behaviour needed. It will be split between the Lambda function, and the Raspberry Pi scheduled task. We will be using the "post_message" behaviour in the Alexa Skill, and the "read_message" behaviour running on the Raspberry Pi.

Write a Lambda Function to Post Queue Messages


The Lambda Function is where all the "thinking" for the Alexa Skill takes place. To create one of these, navigate to the AWS Lambda Page and click "Create A Lambda Function". A series of drop down menus will appear. Select runtime python2.7 and select the blank project template.

Now you need to configure a trigger. There will be an empty gray dotted box image linking to the Lambda Function. Click the dotted box, and choose "Alexa Skills" as the trigger.


Click next, and enter any function name and description. It is also recommended that you create a custom role under the Role dropdown, called "lambda_basic_execution" with the default rights. For more info on roles, you can follow this Amazon guide.

Then Select Python2.7 from the runtime drop down menu once again. This will display a code box with a single python function called lambda_handler. This is where the code triggered by an Alexa phrase will be placed. There is also an event parameter, which is where voice keyword behaviour is implemented (e.g. "on" or "off"). 

We will take some snippits from the demo code we wrote to write to the queue, and add some helper functions taken from AWS Lambda tutorial code for building a Lambda Function. You can copy paste the code below, but you will need to ensure you have added your api key, secret, location, and queue URL once again here. 



Now click "Next" and "Create Function". Now under the Lambda -> Functions dashboard, you should see your function. If you click test, and select "Alexa Intent - Answer" from the dropdown, this should return a JSON response with "text: unknown". This will show the function is working as expected.

Now it's time to connect it to Alexa Skills.

Writing an Alexa Skill 


To write an Alexa Skill, you have to navigate to the Alexa Developer portal. In here, you will click "Get Started" under the Alexa Skills Kit. Then click "Add a New Skill" and give your skill a name.

Under the invocation field, this will determine how you start this skills. Mine is called "magic mirror", and I have to say "Alexa, ask magic mirror (varying commands)" to invoke. Name yours whatever you want. 

Clicking next will bring you to an intent scheme page. This defines all the different actions within your skill, and this information is passed to the Lambda Function as well. The code in the Lambda is tied to these Intents. So if you use something other than "MirrorOn" or "MirrorOff", you will need to update the corresponding section in the Lambda.


Note the intents MirrorOn and MirrorOff have a multiple sample utterances. These provide the context on what logic the Lambda should execute, with multiple ways to achieve the same thing. To dive deeper on utterances, here is a great blog post detailing how to cover as many phrases as possible.

Next you will have to add the Lambda Amazon Resource Name (arn) from your Lambda function to link the two in your skill configuration.

This can be found under the AWS Lambda Dashboard when you click Function, and on a function name. It will appear in the top right corner beside the bold ARN. Copy and paste that into the Alexa Skills text box after specifying your region.


Locating the Lambda ARN

Link Lambda ARN to Alexa Skill


Clicking next will bring you to a test page. Simply type your invoking phrase in the "utterance" box, and this should run your lambda function. Pay close attention to your utterance. If you have followed this tutorial verbatim, "turn on" or "turn off" should complete successfully, providing a JSON response with the "text:" parameter set to either on or off.

If you have the same account registered under the Alexa app settings, this skill should now be enabled automatically. You can confirm by visiting this link, and scrolling to find your skill.

This indicates the Lambda Function has successfully added an "on" or "off" message to the queue. Wonderful! Almost finished.

Reading the Queue from target system


In this case, I want my Raspberry Pi hosting the smart mirror software to check this queue for additional commands. This is done by using the python queue code developed earlier for reading messages, and running custom behaviour based on the message content.

The script reads from the queue and run the on or off actions is as follows. Once again, don't forget to add your own API, URL, and location information:



Now you can add a cronjob to run this script every minute. This script is using long polling for the queue message, which means it waits for 20s for a message to arrive. This is repeated until 1 minute has elapsed, and then cron relaunches the job. This keeps your SQS requests well in the free tier, but responds immediately to messages.

This can be added as a cronjob with the following commands:

$> crontab -e

Modifying this textfile to add the following line (with a newline break):
* * * * * python3.4 /home/pi/alexa_skill/check_queue.py

The tv on and off scripts can be found  here.

Extended Functionality


This has applications far beyond simply starting and stopping a TV. Anything you can write a script for can now be triggered using custom Alexa Voice Commands. Considering the Raspberry Pi is equipped with GPIO ports, this means you can use voice commands to spin motors, unlock doors, turn on lights, and much much more.

If you would like to write your own skill, you need to add an additional intent to the Alexa Developer Skill configuration. Then in the Lambda function, add an if statement to handle the new intent by posting a different message. Finally, on your target platform, in the queue checking script, add an extra if statement to handle the new keyword.


And I haven't even started thinking the use of IFTTT!

33 comments:

onelilfizzle said...

Awesome writeup! Might I recommend Sonus: https://github.com/evancohen/sonus

Ben Eagan said...

onelilfizzle, looks interesting, but I'm having a little trouble connecting the dots. Would this achieve the same as I've done here, but with the everything except the voice processing occurring locally?

Might have to give this a try.

Unknown said...

Well, I followed your fantastic directions, but now I can't figure out how to add my new skill to my own Echo.... :|

Ben Eagan said...

Hey Unknown, you have to have your device registered under the same account you wrote the Alexa skill under. Checkout alexa.amazon.com, click skills, then look for "Your skills" at the top to check you've got it enabled.

Anonymous said...

Modifying this textfile to add the following line (with a newline break):
* * * * * python3.4 /home/pi/alexa_skill/check_queue.py

Hi sir may I know what is this check_queue.py
Thanks! :)

Regards,
Derrick

Derrick Zhen Yu said...

Hi ben,
the command seems to be taking a random 5-30 secs to take place.
Is there any way to improve this?
Thanks in adv!

Ben Eagan said...

Hi Derrick, you can change the duration of the "sleep" command in the python check_queue script to check more frequently, but this could put you over the "free" threshold SQS threshold and end up costing a couple dollars a month. You're probably not going to get more responsive than 2-5s though just based on queue refresh rate.

Derrick Zhen Yu said...

Hi Ben,
A huge thank for your detailed reply.
I believe I have followed everything as you have done, and it is really taking a random 5-30 seconds.
Could it be the delay over the cloud? As in I am using the East US setting but I am located at Singapore.
Thanks!

Payli Tuzu said...

NICE POST NICE BLOG
โกลเด้นสล็อต
บาคาร่า

Anonymous said...

Amazing. For Mirror On and Mirror off, do you just tell it to put the raspberry pi to sleep? Or to put a black screensaver? If the raspberry pi were off, Alexa wouldnt work so "mirror on" wouldn't work.

Anonymous said...

Thank you for sharing your wonderful implementation. Curious as to why you use SQS instead of MQTT?

Anonymous said...

where exactly was queue_service.py used?

Ben Eagan said...

It's just educational to show what you want done but isn't run in the final implementation. Its functionality gets split between the lambda_function.py and check_queue.py files. Hope that makes sense.

Anonymous said...

Thank you Ben, it's makes sense now.

Roy Kolk said...

if got a question. if i ask alexa to turn de mirror off she says i cant help you with that. if i run a test on the lambda page with off than my screen goes off. what do i need to do to fix it??

Ben Eagan said...

Hey Roy, I had some trouble until I realized I had to say specifically "Alexa, ask (appname) turn on / turn off". I think the lambda test has an assumed utterance that comes before starting the app, you have to include that bit too. Does that make sense?

Roy Kolk said...

no sorry i dont understand that. i did exacly like the discription and named it the same.. if i say alexa ask magic mirror off. i hear a beep and than noting. can you explain it in noobie plz? im from the netherlands so its hard to understand

Roy Kolk said...

i that it was some problem on the pi, but i think thats not the problem because he turns the screen off then you type it on the test right?
greets roy

Ben Eagan said...

Sorry Roy, but this is almost impossible to diagnose from the info you have given me. If it is working when you type the command and not working through your alexa device, it's probably related to your utterances. Otherwise, try to isolate what piece is not working and we can take it from there.

Roy Kolk said...

hey ben!
thanks. im gonna try it tomorrow

Barrett Ranney said...

Hi ben, great project, been following along and so far, have been able to replicate. When I run the test after creating the function in lambda I get the following:

{
"errorMessage": "Handler 'handler' missing on module 'index'"
}

I even tried to copy and paste the above code for the lambda function only editing the sections in relation to keys ids and urls. I'm not seeing where I made my error, any chance your able to point me in the right direction?

Ben Eagan said...

Hi Barrett, I'm not sure what this would. Do you get anything else in your error message (line numberm etc)? Have you successfully linked the lambda ARN to your skill?

baker397 said...

Just a cool thought, not sure how useful it is to you. I saw an article a while back about creating a "Smart Power Strip" using a raspberry pi. This could be useful in a network closet type scenario. Have your firewall and network switches on the smart power strip and create scripts to cut and restore power to individual outlets. Then create triggers to run each individual script based around keywords. For instance "Alexa, ask network closet reboot switch 3". I would never trust anything like this for an enterprise environment but could make a cool addition to advanced home networks.

Ben Eagan said...

Hey baker397, thanks for the great suggestion! I've actually needed that on many occasions.

Barrett Ranney said...
This comment has been removed by the author.
Barrett Ranney said...

As a follow up, I found my error in Lambda, as far as I can tell. Corrected it, and received a success for testing. Moving on to Alexa Skills, where if I do a text for mirror off I get something other than a endpoint could not be reached. I have to say, this has taught me quite a bit and I'm grateful you put this up for others to learn from.

HD said...

I love your instruction - exactly what I needed.

can you give me a hint where would I need to change the code if I want alexa to say something else than on/off ?

E.g. "Turning TV off"

Ben Eagan said...

Thanks HD, checkout the "message" being passed in the Alexa lambda_handler to build a response.

Pooja Sisal said...

Hello Ben, I have some trouble selecting the blueprints and testing my lambda function. Following are my issues:
1. As you said select Run-time: python 2.7 and then select a blank project template, I can't find a blank template in there. Instead I tried selecting and going ahead with the 'alexa-skills-kit-color-expert' but still no success.
2. I entered my function name and the description and also selected the run-time but I do not get the lambda_handler code box displayed.
3. When I go ahead and try to test the function choosing the Alexa Intent-Answer, I obtain an error saying 'Execution result:failed(logs)'.
Would appreciate an immediate reply as I'm working on it. Thank You

kal said...

There's a small error in line 21 of the sample code in section "Reading and Writing to Queue with Python". The argument 'url' should be changed to 'queue_url'.

Ben Eagan said...

Fixed, thanks kal!

Pooja Sisal said...

Waiting for your reply Ben! I need it ASAP.

Ben Eagan said...

Pooja, it has been a while since I wrote this, so it's possible the skill web dashboard has changed. I would recommend trying another Skill specific tutorial if these instructions aren't working for you.

Post a Comment