Arduino Home Automation using Acoustic Connectivity powered by Chirp

Recently I got intrigued with all the new Home Automation products in the market and hence I purchased a Google Home Mini and few other smart devices to control lights, fans, AC and what not. While it did work and everything was fun to use, I got a bit troubled with having all these wireless connectivities happening around me. My bedroom alone had 5 smart devices, and including my laptop and phone, it will be a total of 7 wireless devices around me staying always connected to my router. I was worried that all these Wi-Fi signals might be bad for our health and this research paper on Sciencedirect added evidence to my thoughts. This is when I started thinking about an alternative wireless home automation solution that does not need Wi-Fi / Bluetooth to operate.

 

A Novel approach for Home Automation Devices using Chirp

We have built an IR Remote controlled Home automation project earlier which can work without Wi-Fi or Bluetooth, but it is no longer cool and I wanted to control the devices with voice. I was looking for a technique for the Google home mini to directly communicate with the smart devices, so basically I need a Machine to Machine wireless communication method without using Wi-Fi or BLE.

 

This is when I remembered the Interview with Dan Jones, CTO of Chirp in which he introduced Chirp as “Chirp is a way to transmit information using sound waves. In contrast with Wi-Fi or Bluetooth which uses Radio frequencies, Chirp encodes data in tones that can be played (transmitted) using any computer speaker and received through any computer microphone without the need of having any additional hardware like RF chips. This enables Chirp to be used on any consumer device that has a speaker and microphone in it, like mobile phones, Laptops, PA systems, etc and can transmit information even through YouTube stream or TV broadcast.” This means that we can use our Google home mini to talk directly (using chirp) to our smart device. All we need is a microphone and a processor inside these smart devices to decode the chirp audio data to perform required actions. Chirp has also recently announced the integration of data-over-sound for Arduino, this means we should be able to use the new Arduino nano 33 BLE sense board  to run Chirp SDK for our project.

 

So in this project, we will build an Acoustic Home Automation Device in which we will be able to control this smart device without using any wireless connections like Wi-Fi or Bluetooth. We will use Actions for Google and Dialogflow to build a test application for our Google Assistant to make it play encoded audio (Chirp). This audio will then be captured by the Arduino nano BLE sense board running the Chirp Arduino Program, based on the message encoded in the data the Arduino board could perform any action like toggling an AC load controlling the RGB LED, etc. It might sound like a lot, but it is relatively simple and the complete instructions are given below. So let’s get started.

 

Playing Chirp Audio with Google Assistant

The easiest way to build voice-controlled home automation is to use IFTTT with Google Assistant for Home Automation. But, at the time of writing this article Chrip does not have an Applet on IFTTT, so we will use the native Actions for Google platform. There is a good chance that Chirp has provided its own applet in IFTTT at the time you are reading this article so do check that out. If not let's proceed with Actions for Google to build a Google Assistant Application.

 

Step 1: Open the Dialogflow console. You might have to sign-up if you have not done it yet. Then click on “Create New Agent” as shown below

 

Step 2: Give your new agent a name and then click on create. I have named my agent as “Circuitdgest_Automation”

 

Step 3: Now you will be taken to the intents section. This is where you train your new agent for various intents. In our case we will write two intents, one is to turn on the light and the other is to turn off the light. You can write any number of intents based on the applications and commands that have to be understood by your Agent. By default, you will have the welcome intent, but let’s create a new one to turn on the light by clicking on “Create intent” and I will name this intent Lights on

In the new page click on “Add Training Phrases” and enter the command that has to be understood by your assistant in this case “Turn on the light”. You can write any number of training phrases that convey the meaning of the same intent.

 

Step 4: Now scroll down and click on “Add Response”. This is where we tell the assistant how to respond back if this particular intent is triggered. By default, you can enter text response here, but we need audio to be played our assistant, so click on the “+” symbol near the “Default” tab and select Google assistant and then clock “Add responses” and select “Simple Responses”. Inside the simple response text box type in this code

<speak>
<audio src = "INSERT YOUR LINK">
</audio>
Lights Turned ON
</speak>

 

It is a simple SSML code to play audio from a URL and then say “Lights Turned ON”. This audio should be the encoded audio from Chirp and it should be uploaded jovo.tech.

 

Getting encoded audio from chirp: 

This encoded audio can be obtained from the Chrip Android application. Install the app and type in the message to be encoded and click on the Send button to listen to the encoded audio

Here I have typed “Lights on”. Similarly, you can type any message and click on the Send button to check the encoded audio. We can then save this audio file (mp3 format) using any sound recorder. I have recorded two such audio files with the encoded message “Lights on” and “Lights off”. You can download them from here if you want to save time. Once you are ready with the audio files go to jovo tech audio converter to upload your mp3 files and get a link for it.

 

Note that this is a testing tool and your files will be live only for 24 hours. If you need it to be working for a long time you have to use your own web server like Google fire-base like this sample firebase Google assistant interface. Now that you have got the link your response on dialog flow should look something like this

Make sure you click on save to save this intent.

 

Step 5:  Similarly, create another new intent turn off the lights. Again enter your training phrase and also your response. This time insert the link of the other audio file and ask it to say Lights turned off as shown below.

Again make sure you save this as well.

 

Step 6: Now, on the left side click on the integrations tab and select Google assistant. This will help us test the application using Google assistant. Select the Google Assistant Application and click on “TEST”

 

Step 7: This will take some time after which you will get the test screen as shown below. On the left corner, you can see a text box where you can type in your test commands and see how the application responds. You can also find some debugging applications on the left side of the screen.

When you type “turn on reading lights” it should play the chirp tone and say lights turned on, similar it should also for “Turn off reading lights”. Once it works here, you can try it on any google assistant device like your phone or google home mini that connected to your Gmail ID. Just say talk to my test app and start controlling the device.

 

Preparing your Arduino Nano 33

Now on the hardware side, things have gotten pretty easy thanks to the Arduino Nano 33 BLE Sense board. Because the board has an in-built microphone and chirp libraries ready to deploy directly from your Arduino IDE. If you are completely new to this board then it is recommended that you read this “Getting started with Arduino Nano 33 BLE” Article to understand more on this board. The circuit diagram to connect the relay module with nano 33 for home automation is shown below

I have used Fritzing to draw the circuit diagram and since the nano 33 board is not yet available I have used the standard nano, but both have almost the same pinout. Again do remember that nano 33 operates on 3.3V so to connect a 5V relay you can use the Vin pin on the board. Since the nano 33 BLE sense board already has an onboard microphone we need not worry about it. But make sure your relay module can also work on 3.3V signal voltage because the GPIO pins of Nano 33 only get 3.3V hen high.

 

Programming Arduino using Chirp:

The complete Arduino code can be found at the bottom of this page. But the code also requires credentials.h header file inside your project directory, so you can download the complete project folder using the link below. Make sure you have installed chirp SDK to your Arduino IDE for this program to compile.

Arduino Chirp Communication ZIP File Download

In order to use Chirp with Arduino, you have to create an account on the developer site of Chirp to get your credentials. Also, make sure to select the configuration as 16khz-mono-embedded as shown below. Make note of the key, secret and config details because we have to use them in our Arduino program.

 

Now open you Arduino program and move to the credentials.h tab and paste the corresponding values here. These credentials will vary from person to person and should be shared.

 

The main program is the same as the example program, except inside the onReceivedCallback() we will add a small program to compare the obtained data with a pre-defined text. We have initialized a string variable called payload which will compare the actual value with a string and toggle the digital pin as required. We have also toggled the on-board LED light for acknowledgment.

String payload = String(data);
    //Serial.println(payload);
    if (payload == "lights on")
    {
      digitalWrite(13,HIGH);
      digitalWrite(2,HIGH);
      Serial.println("Arduino has turned on the lights");
    }
    if (payload == "lights off")
    {
      digitalWrite(13,LOW);
      digitalWrite(2,LOW);
       Serial.println("Arduino has turned off lights");
    }

Make sure the strings that you compare are the same as the data encoded in your audio file. You can also open the Arduino Serial monitor for debugging or use the Chirp Android application. Once this set-up is ready you can begin testing your project with any device running google assistant that is linked to your Gmail ID. The complete working can be found in the video given at the end of this page. Also, find the complete code below or it can be downloaded from here.

By the time I finished my project I got really excited by the possible applications of Chirp and was able to figure out many other ways in which chirp and Arduino can be used. What will you build with Chirp? Let me know in the comment section, also if you have any questions please use the forums to post them and I will try my best in answering them all.