Person Follower
General Description
The Person Follower is a real-time tracking system designed to detect and follow individuals in video streams. It utilizes YOLOv8 for object detection and either BoTSORT or ByteTrack algorithms for multi-object tracking with optional ReID (re-identification) features. The system processes data from RealSense depth cameras to accurately locate and track persons, calculating their position, distance, and angle relative to the camera. The component is containerized with Docker for easy deployment and offers both tracker and follower services with RESTful APIs for integration with robotic systems.
| Resource | Link |
|---|---|
| Source code | https://gitc.piap.lukasiewicz.gov.pl/ai-prism/wp4/ai-based-perception-modules/person-follower |
| Demo Video | KEBA%20AGV%20FollowMe/demo/followme_demo_keba_session3.mp4 |
Contact
The following table includes contact information of the main developers in charge of the component:
| Name | Organisation | |
|---|---|---|
| Dorin Clisu | dorin.clisu@nttdata.com |  |
| Iulia Farcas | iulia.farcas@nttdata.com | |
License
Proprietary.
Technical Foundations
Integrated and Open Source Components
Overview
The Person Follower component integrates several open-source libraries to provide robust person detection and tracking functionality. It primarily uses YOLOv8 for object detection coupled with BoTSORT/ByteTrack for tracking. The system leverages PyTorch and TorchVision for deep learning capabilities, FastAPI for API endpoints, OpenCV for image processing, and Hailo for hardware acceleration. Additional libraries include data-middleware for messaging, pandas for data analysis, and Runner-with-API for service management.
Pre-existing Components
YOLOv8 (Ultralytics)
Source
YOLOv8 is developed by Ultralytics. https://github.com/ultralytics/ultralytics
Description
YOLOv8 is a state-of-the-art object detection model that provides fast and accurate detection of objects in images and video streams.
Modifications
None, except fine-tuning a model on MOT17 dataset.
Purpose in AI-PRISM
Provides the core person detection capability needed for the tracking and following functionality.
License
AGPL-3.0 License - https://github.com/ultralytics/ultralytics/blob/main/LICENSE
Bot-SORT
Source
Bot-SORT: https://gitc.piap.lukasiewicz.gov.pl/ai-prism/libraries/bot-sort.git
Original Bot-SORT: https://github.com/NirAharon/BoT-SORT
Description
BoTSORT is a tracking algorithm that maintain identity consistency across video frames.
Modifications
Some modifications were required to make it run in a containerized environment, and also to accelerate the inference using Hailo AI.
Purpose in AI-PRISM
Provides consistent tracking of individual persons across frames, enabling the following functionality.
License
MIT License (for the original ByteTrack)
FastAPI
Source
FastAPI: https://github.com/tiangolo/fastapi
Description
FastAPI is a modern, high-performance web framework for building APIs with Python.
Modifications
None.
Purpose in AI-PRISM
Provides the API interface for the tracker and follower components, enabling integration with other systems.
License
MIT License - https://github.com/tiangolo/fastapi/blob/master/LICENSE
OpenCV
Source
OpenCV: https://github.com/opencv/opencv
Description
OpenCV (Open Source Computer Vision Library) is an open-source computer vision and machine learning software library.
Modifications
None.
Purpose in AI-PRISM
Used for image processing, visualization, and camera integration.
License
Apache 2.0 License - https://github.com/opencv/opencv/blob/master/LICENSE
PyTorch & TorchVision
Source
PyTorch: https://github.com/pytorch/pytorch TorchVision: https://github.com/pytorch/vision
Description
PyTorch is an open-source machine learning framework with TorchVision providing datasets, model architectures, and image transformations.
Modifications
None.
Purpose in AI-PRISM
Provides the deep learning backbone for the object detection and tracking models.
License
BSD License - https://github.com/pytorch/pytorch/blob/master/LICENSE
How to install
Every AI-PRISM component is installed using the Cluster management service. During the installation process, the user needs to configure a set of high-level parameters.
How to use
The Person Follower system has two main components: the tracker and follower services, with a ZMQ data pipeline and REST API for controlling the behavior. See http://localhost:8000/docs for the API documentation.