Using Amazon Echo to Control Lights and Temperature

By @zachfeldman
Written on Dec 28, 2014


Coming off of their popular Fire TV Stick, Amazon is venturing further into the home with something in a completely new category. Amazon Echo’s marketing page states that it is, “Always Ready, Connected, and Fast. Just Ask.” The service it provides is one of knowledge, music (which is limited to what you have on Prime Music, “Tune In”, and “iHeartRadio”), and adding items to your to-do list/buying them on Amazon.

The feature on Amazon Echo that most enthralled me upon receiving mine was the voice recognition. I noticed since getting it that the array of 7 microphones was providing voice translation, many times at a distance, with really impressive accuracy, more accuracy then I’d seen in Siri, Google’s voice recognition, or any comparable services.

I wanted to tap into this transcribing, but I wasn’t sure how. My first step was to try Wireshark to see if I could intercept the transmissions back and forth, but the traffic was encrypted. Instead, I went with a far less elegant solution, for now.

On the Echo web app (echo.amazon.com), you can go to the settings for an individual Echo unit and see its search history. Bingo! The following code allowed me to fire an event whenever a new search was added to the page, with its contents. Also, if I use the “stop” keyword after any command, Alexa basically ignores me. Thanks to Amazon for requiring jQuery:

1
2
3
4
5
6
7
8
var lastCommand; $(document).ajaxComplete(function(){ command = $(".dd-title.d-dialog-title").first().text() if(lastCommand != command){ $.get("http://localhost:4567/command?q="+command) lastCommand = command; } })

I had to find a way to get a headless browser or a server running that could inject this JS on a page and keep it alive, waiting for requests. I’m still working on this with a Raspberry Pi and some other hardware, but for now, I have a proof of concept running on my computer.

The Sinatra application I’ve built allows natural language parsing to turn on and off the lighting in my apartment selectively, as well as control the lights. This is a really rough beta but if you follow the following steps, basically anyone can get it to work with the same setup.

1) Clone the repository my code is in: alexa-home

2) Open up echo.amazon.com and sign in with your credentials. Then, go to the Settings -> History page and inject the JS code above by opening up the JavaScript Inspector (Command + Option + J), pasting, and hitting enter. You can also find the necessary code in script.js in the repository.

3) Set environmental variables for your Nest login information if you plan on using that feature (NEST_EMAIL, NEST_PASS). Press the button on the Hue module to make sure it’s ready to accept incoming connections.

4) Boot up the Sinatra server in the repository. You’ll need ruby installed and you’ll have to have the right dependencies (bundle install in the directory if you use bundler). To boot up the server, use ruby app.rb.

5) Start making requests to your Amazon Echo and hoping they get answered by your proxy! Some examples:

1
2
3
alexa turn on all the lights stop alexa turn off all the lights stop alexa set nest to seventy eight stop

Obviously this is just the beginning as far as what can be accomplished! I’d love to see some contributions to the GitHub for it for more advanced functionality and I’d like even more to see an official API from Amazon so we don’t have to play this game of cat and mouse.

Basically, I’m a fan of anything that makes it seem more like we’re living in Star Trek.

Here’s the demo video again:





Sign up for our e-mail list to hear about new posts.