Easy Nákup
Easy Nákup (eng. "easy purchase") was a winning project that we built at Hack Košice 2019, a hackathon in Slovakia. In this post, I will describe the project itself into more detail; to find out more about the hackathon you can read this post.
The main challenge we were were trying to figure out was how to help the Košice region, a region in eastern Slovakia. We understood that the region has many small and remote villages sparsely located far from bigger cities or towns. Coming from similar regions we quickly identified some a potential issue of delivering food, medicine, and other supplies to these villages. This can be especially problematic for the elderly or for people without cars. In such cases, their families need to drive them to the shops or deliver their supplies for them, which might not always be possible due to work, family, etc.
An obvious solution seems to be online shopping. However, large vendors like Amazon often do not deliver in such areas, at least not for a reasonable price.
Therefore, we came up and designed a system which allows anyone, and especially people who are not comfortable with modern technology, to issue an order to their preferred (local) supplier.
How it works
The system consists out of two main components: a shopping list scanning device and an app for the vendors.
A shopping list scanning device is capable of converting hand-written shopping lists into digital text. Thus, people can simply write what they need on a piece of paper, put this piece of paper into a scanner and that's it. The scanner sends a digitalised shopping list to the user's preferred vendor which is usually a local supermarket or other smaller shops, rather than a big online retailer. Such device can be installed at a local block of flats or some other facility that is very handy for the locals.
The scanner needs to know which user is scanning their shopping list. We figured that if people have problems with using technology in the first place, it would be unlikely that usernames and passwords would work as an authentication mechanism. Hence, we opted to use facial recognition which is easy for anyone to use. Only after the user is recognised, the system can start scanning and converting the list. An alternative option could also be using a card or an RFID tag that people could just hang on their keys.
As mentioned, the digitalised shopping list is sent to one of the, usually smaller, local vendors. Usually, they do not have their own online shopping platform or any supporting system at their disposal because these tend to be expensive to develop. Therefore, we created a web app, where local sellers can register and keep track of their orders. The idea is that the scanner sends a digitalised shopping list to these vendors who can then specify prices for each item (which can be pre-set) and put together a bag of the requested supplies. To confirm the order, they can also send an automatic SMS receipt. In the app, they can also see a map with all the locations they have to deliver to and the most efficient root to all of them. The delivery can happen at the end of the day or even a few days later when it is sensible for the vendor to drive to all those people. The payment can be in cash or card directly to the vendor when the supplies are delivered.
This way we could help local vendors to participate in a smaller delivery system without having to implement their own APIs which are very costly. By simply creating an account with our web application, they can already start participating.
How we built it
The shopping list scanning device was built with a DragonBoard 410c with a webcam attached to it for facial recognition and list scanning. The facial recognition was running entirely on the DragonBoard 410c and was powered by OpenCV library. For converting the shopping lists we utilised Optical Character Recognition (OCR) offered by Google Cloud.
We used a browser based Graphical User Interface (GUI) for the DragonBoard 410c where people could see the instructions and the progress. This was built with simple HTML and JavaScript.
The main server, which was the main link between the scanning device and the vendors app, was built with NodeJs and MongooseDB. All connections between all parts of the system were managed through Socket.IO.
You can see more details in our repository or on Devpost.
Demo
Despite having only 24 hours to come up, design and build a prototype of the system, we managed to make all the core functionality working: facial recognition accurately identified registered users, shopping lists were correctly transcribed and linked to the right user, the vendors could see the new order on their screen as soon as it was created and confirmed it, which triggered an SMS message to be sent.
You can see all of this working on the video below. The video was created after 30 hours without sleep, almost 24 hours of hacking and 7 presentations in front of the hackathon judges – which explains the production quality. At the end of the video, you can see our team: Danilo Del Busso, Alvaro Rausell Guiard and me.