RealSense data publisher
General Description
The RealSense ZMQ Publisher is a service that interfaces with multiple Intel RealSense depth cameras, provides configuration management via a REST API, and publishes camera data (color and depth frames) through ZeroMQ sockets. It enables applications to access calibrated depth sensor data without direct hardware interaction. The service manages camera discovery, configuration, and streaming while providing a clean separation between hardware and consumer applications through the publish-subscribe pattern. It supports both live camera streams and recorded bag file playback with configurable resolution and frame rates.
| Resource | Link |
|---|---|
| Source code | https://gitc.piap.lukasiewicz.gov.pl/ai-prism/wp3/ambient-digitalization/realsense-zmq |
| Demo Video |
Contact
The following table includes contact information of the main developers in charge of the component:
| Name | Organisation | |
|---|---|---|
| Dorin Clisu | dorin.clisu@nttdata.com |  |
License
Proprietary.
Technical Foundations
Integrated and Open Source Components
Overview
The RealSense ZMQ Publisher integrates several open-source components to provide its functionality. It relies on Intel's pyrealsense2 for camera interfacing, FastAPI for the REST API, ZeroMQ for message passing, and AnyIO for asynchronous operations. These components are orchestrated to provide a robust and flexible service for depth camera data streaming.
Pre-existing Components
PyRealSense2
Source
PyRealSense2 is developed by Intel as part of the RealSense SDK. https://github.com/IntelRealSense/librealsense
Description
PyRealSense2 provides Python bindings for the Intel RealSense SDK, allowing applications to interface with Intel RealSense depth cameras. It enables access to depth, color, and infrared streams and provides tools for camera calibration and point cloud generation.
Modifications
No modifications were made to PyRealSense2 itself. It's used via the AI-PRISM library (python-realsense) that is a compilation / build wrapper for ARM64 architecture.
Purpose in AI-PRISM
PyRealSense2 is used to interface with Intel RealSense cameras, enabling the application to capture and process depth and color data, which is crucial for digitizing the physical environment in the AI-PRISM ecosystem.
License
Apache License 2.0. https://github.com/IntelRealSense/librealsense/blob/master/LICENSE
FastAPI
Source
FastAPI is an open-source web framework. https://github.com/tiangolo/fastapi
Description
FastAPI is a modern, high-performance web framework for building APIs with Python based on standard Python type hints. It provides automatic interactive documentation and validation for API requests.
Modifications
None.
Purpose in AI-PRISM
FastAPI provides the REST API that allows users and other systems to configure and monitor the RealSense cameras, making the component easily integrable into the wider AI-PRISM ecosystem.
License
MIT License. https://github.com/tiangolo/fastapi/blob/master/LICENSE
AnyIO
Source
AnyIO is an asynchronous I/O library. https://github.com/agronholm/anyio
Description
AnyIO is a high-level asynchronous I/O library that works with multiple asynchronous event loop implementations, giving developers flexibility while maintaining a consistent API.
Modifications
None.
Purpose in AI-PRISM
AnyIO is used to manage concurrent operations in the RealSense ZMQ Publisher, enabling efficient handling of multiple cameras and various I/O operations simultaneously.
License
MIT License. https://github.com/agronholm/anyio/blob/master/LICENSE
Uvicorn
Source
Uvicorn is an ASGI web server. https://github.com/encode/uvicorn
Description
Uvicorn is a lightning-fast ASGI server implementation, using uvloop and httptools, that serves as the web server for FastAPI applications.
Modifications
None.
Purpose in AI-PRISM
Uvicorn serves the REST API created with FastAPI, allowing for configuration and monitoring of the RealSense ZMQ Publisher.
License
BSD 3-Clause License. https://github.com/encode/uvicorn/blob/master/LICENSE.md
How to install
Every AI-PRISM component is installed using the Cluster management service. During the installation process, the user needs to configure a set of high-level parameters.
How to use
The component should ideally be deployed independently on an embedded computer such as raspberry pi, thus emulating the functionality and usage philosophy of an IP camera. By knowing the ip address of the embedded computer, once can read frame data from the camera using the data-middleware library. For example:
from data_middleware.pubsub.zeromq import ZMQSubscriber
from data_middleware.models.realsense import RealsenseData
sub = ZMQSubscriber('tcp://192.168.1.42:5000', ['color/D415/0'])
info, _data = sub.receive_raw()
data = RealsenseData.parse_multipart(_data)
print(info)
print(data)