How to Use APDS9960 RGB and Gesture Sensor with Arduino

Today most of the phones come with gesture control feature to open or close any app, start music, attend calls etc. This is a very handy feature to save time and it also looks cool to control any device with gestures. We previously used accelerometer to build gesture controlled robot and gesture controlled Air mouse. But today we learn to interface a gesture sensor APDS9960 with Arduino. This sensor also has a RGB sensor to detect colors, which will be also used in this tutorial. So you don’t need to use separate sensors for gesture and color detection, although a dedicated sensor for color detection is available- TCS3200 color sensor which we have already used with Arduino to build a color sorter machine.

 

Required Components

1. Arduino UNO
2. APDS9960 RGB and Gesture Sensor
3. 16x2 LCD
4. DPDT Switch
5. 100K pot and 10K resistor
6. Jumper Cables

 

Introduction to APDS-9960 Digital Proximity RGB & Gesture Sensor

APDS9960 is a multifunction sensor. It can detect gestures, ambient light and RGB values in light. This sensor can also be used as a proximity sensor and is mostly used in smartphones, to disable the touchscreen while attending a call.

 

This sensor consists four photodiodes. These photodiodes detect the reflected IR energy which is transmitted by an on-board LED. So whenever any gesture is performed then this IR energy gets obstructed and reflects back to the sensor, now the sensor detects the information (direction, velocity) about the gesture and converts it into digital information. This sensor can be used to measures the distance of obstacle by detecting reflected IR light. It has UV and IR blocking filters for sensing RGB colors and it produce 16-bit data for each color.

The pin-out of APDS-9960 sensor is shown below. This sensor works on I²C communication protocol. It consumes 1µA current and powered by 3.3V so be careful and do not connect it with 5V pin. The INT pin here is interrupt pin, which is used to drive the I²C communication. And VL pin is optional power pin for the on-board LED if the PS jumper is not connected. If the PS jumper is closed then you only need to power VCC pin, it will provide power to both - the module and the IR LED.

 

Circuit Diagram

Connections for APDS960 with Arduino are very simple. We will use a DPDT button to switch between the two modes RGB Sensing and Gesture Sensing. Firstly the I2C communication pins SDA and SCL of APDS9960 are connected to Arduino pin A4 and A5 respectively. As stated earlier the operating voltage for the sensor is 3.3v so, VCC and GND of APDS9960 are connected to 3.3V and GND of Arduino. The interrupt pin (INT) of APDS9960 is connected to D2 pin of Arduino.

For LCD, data pins (D4-D7) are connected to digital pins D6-D3 of Arduino and RS and EN pins are connected to D6 and D7 of Arduino. V0 of LCD is connected to pot and a 100K pot is used to control the brightness of LCD. For the DPDT buttons we’ve used only 3 pins. The second pin is connected to D7 pin of Arduino for input and the other two are connected to GND and VCC followed by a 10K resistor.

 

Programming Arduino for Gesture and Color Sensing

The programming part is simple and easy and the complete program with a demo video is given at the end of this tutorial.

First we need to install a library made by Sparkfun. To install this library navigate to Sketch->Include Library->Manage Libraries.

 

Now in search bar type “Sparkfun APDS9960” and click on the install button when you see the library.

 

And we are ready to go. Let’s get started.

So firstly we’ve to include all the required header files. The first header file LiquidCrystal.h is used for LCD functions. Second header file Wire.h is used for I²C communication and last one SparkFun_APDS996.h is used for the APDS9960 sensor.

#include <LiquidCrystal.h>
#include <Wire.h>
#include <SparkFun_APDS9960.h>

 

Now in next lines we’ve defined the pins for button and LCD.

const int buttonPin = 7;
const int rs = 12, en = 11, d4 = 6, d5 = 5, d6 = 4, d7 = 3;
LiquidCrystal lcd(rs, en, d4, d5, d6, d7);

 

In the next portion, we’ve defined a macro for the interrupt pin which is connected on digital pin 2 and one variable buttonState for the current state of button and isr_flag for the interrupt service routine.

#define APDS9960_INT 2
int buttonState;
int isr_flag = 0;

 

Next an object is created for the SparkFun_APDS9960, so that we can access the gesture movements and fetch the RGB values.

SparkFun_APDS9960 apds = SparkFun_APDS9960();
uint16_t ambient_light = 0;
uint16_t red_light = 0;
uint16_t green_light = 0;
uint16_t blue_light = 0;

 

In setup function, the first line is to fetch the value from button (low/high) and second & third line defines interrupt and button pin as input. apds.init() initializes the APDS9960 sensor and the lcd.begin(16,2) initializes the LCD.

void setup()
{
  buttonState = digitalRead(buttonPin);
  pinMode(APDS9960_INT, INPUT);
  pinMode(buttonPin, INPUT);
  apds.init();
  lcd.begin(16, 2);
}

 

In loop function the first line gets the values from button and stores it in buttonState variable defined earlier. Now in next lines we are checking the values from button, if it is high then we enable the light sensor and if it is low then initialize gesture sensor.

The attachInterrupt() is a function used for external interrupt which in this case is sensor’s interrupt. The first argument in this function is interrupt number. In Arduino UNO, there are two interrupt pins digital pins - 2 and 3 denoted by INT.0 and INT.1. And we’ve connected it to pin 2 so we’ve written 0 there. The second argument calls the function interruptRoutine() which is defined later. The last argument is FALLING so that it will trigger the interrupt when the pin goes from high to low. Learn more about Arduino Interrupts here.

void loop()
{
  buttonState = digitalRead(buttonPin);
  if (buttonState == HIGH)
  {
   apds.enableLightSensor(true);
  }

 

In the next portion, we check for button pin. If it is high then start process for RGB sensor. Then check if the light sensor is reading values or not. If it is not able to read the values then in that case, print that "Error reading light values". And if it can read values then, compare the values of the three colors and whichever is highest, print that color to the LCD.

if (buttonState == HIGH)
  {
    if (  !apds.readAmbientLight(ambient_light) ||
          !apds.readRedLight(red_light) ||
          !apds.readGreenLight(green_light) ||
          !apds.readBlueLight(blue_light) ) {
      lcd.print("Error reading light values");
    } else {
      if (red_light > green_light)
      {
        if (red_light > blue_light)
        {
          lcd.print("Red");
          delay (1000);
          lcd.clear();
        }
…….
………..

 

In next lines again check for button pin, and if it is low the process the Gesture sensor. Then check for isr_flag and if it is 1 then a function detachInterrupt() is called. This function is used to turn off the interrupt. The next line calls the handleGesture() which is function defined later. In the further next lines define isr_flag to zero and attach the interrupt.

  else if (buttonState == LOW)
  {
    if ( isr_flag == 1 )
    {
      detachInterrupt(0);
      handleGesture();
      isr_flag = 0;
      attachInterrupt(0, interruptRoutine, FALLING);
    }
  }

 

The next is interruptRoutine() function. This function is used to turn the isr_flag variable 1, so that the interrupt service can be initialized.

void interruptRoutine().
{
  isr_flag = 1;
}

 

The handleGesture() function is defined in the next part. This function firstly checks for the availability of gesture sensor. If it is available then it reads the gesture values and checks which gesture it is (UP, DOWN, RIGHT, LEFT, FAR, NEAR) and prints the corresponding values to LCD.

void handleGesture()
{
  if ( apds.isGestureAvailable() )
  {
    switch ( apds.readGesture() )
    {
      case DIR_UP:
        lcd.print("UP");
        delay (1000);
        lcd.clear();
        break;
      case DIR_DOWN:
        lcd.print("DOWN");
        delay (1000);
        lcd.clear();
        break;
      case DIR_LEFT:
        lcd.print("LEFT");
        delay (1000);
        lcd.clear();
        break;
      case DIR_RIGHT:
        lcd.print("RIGHT");
        delay (1000);
        lcd.clear();
        break;
      case DIR_NEAR:
        lcd.print("NEAR");
        delay (1000);
        lcd.clear();
        break;
      case DIR_FAR:
        lcd.print("FAR");
        delay (1000);
        lcd.clear();
        break;
      default:
        lcd.print("NONE");
        delay (1000);
        lcd.clear();
    }
  }
}

 

Finally, upload the code to Arduino and wait for the sensor to initialize. Now while the button is turned OFF means it is in gesture mode. So try moving your hands in left, right, up, down directions. For far gesture, keep your hand at a distance of 2-4 inches from sensor for 2-3 seconds and remove it. And for near gesture keep your hand far from sensor then take it near and remove it.

 

Now turn the button ON to put it into color sensing mode and take red, blue and green objects one by one near to the sensor. It will print the color of the object.