SmellsLikeML

Project: PoochPak

1/2/18

Smart Home Security, IoT, Make, K-9 Rescue/Security

I found a hackster.io contest challenging makers to use the Hologram Nova to push the limits of cellular IoT. When the nova USB modem arrives, you will have the device, instructions, SIM cards, 2 antennae, and a small clear plastic case.

Inspired by the work of others to push AI onto the Raspberry Pi, I wanted to incorporate a camera for computer vision. My partner and I brainstormed until deciding to embed wearable sensors into a harness for our dog Sweetpea.









We considered a variety of use cases: home security, lost pets, search and rescue where introducing environmental and audio/visual sensors while maintaining connectivity through a small modem can be powerful.

In automation, we often focus on engineering humans-in-the-loop to take advantage of the cognitive strengths of computers and the human mind. But dogs have a set of skills making them very well suited to certain tasks that humans cannot effectively perform. PoochPak introduces the sensors to relate that experience in terms of metrics we are interested in.

Challenges

The challenges of small embedded systems, like the Raspberry Pi, include:

This repo helps to use YOLO weights to perform object recognition on a pi. Here, we modify the repo to send notifications using Hologram's Cloud rather than telegram-cli. Furthermore, we make use of this list to capture images of dogs and cars as well as people. But all of this can be compute intensive. We need to limit access to these resource hogs until necessary.

Powering a pi running an infrared camera and processing images with a Keras YOLO model will quickly drain the battery. But here we can get creative with the information available to us.

Suppose, for instance, we were interested in developing a dog-harness mounted security camera, streaming data to a local network as part of a home monitoring system.

Home Monitoring

Low power sensors like a heart rate monitor or microphone can be actively monitoring until triggered to take readings from resource intensive sensors like a camera.

Likewise, detecting certain objects can escalate to sending a text message while recording an image. Here, a dog's mobility and curiosity facilitate information gathering from a different vantage. This is still a weakness of home security systems reliant on stationary cameras.

Pet Rescue

While the cost of implanting an RFID chip in your pet is negligible, the logistics of finding your pet depend on somebody with the hand held chip reader helping. With cellular location and visual cues, you are more likely to safely recover your pet.

Security

This furry deployment mode may lend well to infosec applications. Adding a USB adapter while perhaps swapping the harness for something more benign makes war driving a walk in the park.

Furthermore, the canine shows an astonishing work capacity in security and tracking. Dogs can move fluidly through terrains and through spaces that humans awkwardly navigate.

But unless your dog's name is Lassie, you may have no idea how something terrible happened. With PoochPak, you can take a look through the eyes of your watchful companion.

Overview

First, let's consider the vision for how we might integrate temperature, heart rate sensors, infrared cameras and accelerometers with the Raspberry Pi Zero and a Hologram Nova USB modem.

Overall, we position the biometric sensors under the front of the pack (dog's shoulder).

A tiny temperature sensor

Above, a heart rate sensor, below we see the analog-to-digital converter which powers the heart rate sensor, as well as the accelerometer.

We make sensor readings available over a simple server.

From here, we can GET data and render simple logic to determine if/how notifications or logging is appropriate.

For example, we might:

Here is a simple class we might use to send a text notification or get approximate location values. In addition to installing the python SDK and enabling devices, we must get a phone number and device key to use the Hologram Cloud.


from Hologram.HologramCloud import HologramCloud

class Hologrammer(object):
    def __init__(self):
        from config import DEVICEKEY
        credentials = {'device_key': DEVICEKEY}
        self.hologram = HologramCloud(credentials, 
                                      network='cellular')

    def get_geo(self):
        l = self.hologram.network.location
        if l:
            return {'lat': l.latitude, 
                    'lon': l.longitude, 
                    'time': l.time}
        else:
            return

    def msg_send(self, msg, geo_stamp=True):
        if geo_stamp:
            lat, lon, tm = self.get_geo()
            msg += ' {} {} {}'.format(lat, lon, tm)
        self.hologram.sendMessage(msg, 
                                 topics=['poochpak'], 
                                 timeout=20)
        return

One natural trigger for security/rescue applications would be to record video when a person is identified but this requires computer vision. If you've never seen YOLO in action, check it out!. They've created a very fast object recognition model that we can run in Tensorflow/Keras.

Pete Warden shows how we can put these powerful frameworks on even the Raspberry Pi Zero! Likewise, since we want to build OpenCV for a Zero, we'll use Adrian Rosebrock's encyclopedic site.

Finally, we'll borrow much of PiSimo's repo to load YOLO model weights, apply openCV for motion detection, run a picam server. We process video feeds with YOLO object recognition, sending text messages with Hologram instead of using telegram-cli.

Check out the project website and the repo.