In this article, I will go through an application which monitors your incoming packages at your doorstep using AI.
As we are mostly home, due to the current pandemic situation, we shop online for a lot of items from different e-commerce websites and the packages get delivered at our doorsteps. How cool would it be that, if you get notified instantly when packages get delivered to or removed from your doorstep? This is exactly what I will be covering in this article.
My motivation for this project was 2-fold. Firstly, the idea came up when alwaysAI asked if I would like to speak and do a demo at a webinar of theirs on accelerated AI at the edge. Secondly, I received the second place in the OpenCV Spatial AI competition where I worked on a parcel classification and dimensioning problem. I was well versed working with computer vision on cardboard packages. More details about that are here.
I used alwaysAI low code platform for making the computer vision application and Balena for deployment. Balena provides a software platform that helps developers build, deploy, and manage the code that runs on connected devices.
The architecture is pretty simple. We have a setup with a Raspberry Pi 4 that is connected to a webcam as well as an Intel NCS2. The Pi is connected to an MQTT broker and Balena cloud using Wi-Fi. The Raspberry Pi runs a Balena OS image, which includes a component called balena-engine. This engine orchestrates the application containers like microservices’, making them independent and self-contained.
The overall flow of how to build this type of application is shown as below.
For this project, I trained a custom object detection model that detected packages. The first step in this process was to create my dataset. I gathered a lot of images of packages how they would look at my doorstep under different conditions like lighting, distance, time of the day, angles, combinations, stacked, etc. The…