This article is also available in PDF format
1.1 About This Document
This document presents a step-by-step guide for how to enable Intel® RealSense™ depth cameras to be networked over an ethernet or Wi-Fi connection, as depicted in Figure 1. It describes an open-source reference design that is meant to be easy to replicate with off-the-shelf components and free software. The presented architecture is intended for rapid prototyping and evaluation and does not currently offer the maturity and full feature set needed to serve as a production solution. There are several excellent alternative 3rd-party products available for those who are interested in mature industrialized Intel® RealSense™ cameras with ethernet interfaces  . The goal of this project is to offer an alternative solution that allows users to network existing Intel® RealSense™ depth cameras D415, D435 and D435i (and in future more Intel® RealSense™ cameras), with nearly seamless integration with the Intel® RealSense™ SDK 2.0 and Intel® RealSense™ Viewer, versions 2.34 and above.
We will be covering the basic architecture, the reference design parts, and the source code, as well as examples of how to use it. We will then discuss the expected performance, as well as avenues for future improvements.
USB and Ethernet standards offer different advantages, as highlighted in Table 1:
480 Mbps for USB2
100 Mbps or
Up to 5 meters
Up to 100 meters
Requires special PoE equipment and step-down from ~50V to 5V
Sensitive to cable movement and electromagnetic interference from lightning or industrial equipment
Sufficiently robust for most industrial applications
While the standard high-speed USB interface of the Intel® RealSense™ cameras serves most applications very well, many usages require an interface that allows for longer cable lengths, remote access via network, and robustness to repeated disconnection / reconnection to different PC clients. In particular, these requirements are prevalent in robotics and industrial applications.
|9-inch 90-degree Type-A to USB Type-C Cable||Right-angle short cable is recommended for this project to save space when using with provided case||Visit|
|Standard CAT5 Ethernet Cable||Visit|
|microSD Card||SD-Card is needed to operate the Raspberry Pi. 16GB and up is recommended||Visit|
|Raspberry Pi 4 Model B 4GB||We are using Raspberry Pi 4 in this project due to its versatility and popularity. This guide can be applied with minor modifications to other compute boards ||Visit|
|Intel® RealSense™ Camera D415 / D435 / D435i||Provided case and software were designed to work out-of-the-box with D415, D435 and D435i cameras||Visit|
|Raspberry Pi PoE Hat (optional)||PoE Hat is required in order to enable Power-over-Ethernet capability. Without it, the project will have the same functionality but will require external power source via Raspberry Pi USB-C connector||Visit|
|6 x M5 5mm Screws|
|Custom 3D printed case (optional)||We provide custom Raspberry Pi case schematics with mounting points for Intel® RealSense™ Camera. The case is ready for print and can be ordered online via one of commercial 3D-printing services||Visit|
Figure 3. List of components
Power-over-Ethernet is not supported by most consumer routers, so in order to utilize PoE capability compatible network switch is required.
The size and cost of this project can be significantly reduced by using camera modules (example link) and replacing USB connector by soldering USB wires directly to the PCB. To simplify assembly this guide assumes off-the-shelf Intel® RealSense™ Cameras and requires no soldering skills
Raspberry Pi 4 requires microSD-card with compatible software to work.
Latest official software is available at raspberrypi.org/
Intel® RealSense™ SDK can be installed using the following instructions.
As reference, a pre-built image is available (download link).
After downloading and extracting the image, use balenaEtcher to flash it to the SD-card:
For initial configuration, Raspberry Pi can be connected to a monitor, keyboard and mouse. Alternatively, it is possible to connect the device directly to a laptop / desktop PC via Raspberry Pi USB Type-C port :
For this method to work the cable must be USB Type-A to USB Type-C. Type-C to Type-A to Type-C will also work is some cases, but Type-C to Type-C will not work due to Pi’s USB hardware limitation
After about 30 seconds, host computer should detect new network named “PI4 USB Device”. It is then possible to establish remote terminal session to the device:
Initial password and username are realsense.
First things first
Start by running passwd command to change the default password from realsense to a new strong password
Raspberry Pi 4 offers number of networking solutions – first and foremost there is the gigabit ethernet port, but there is also Wi-Fi and limited network over USB that we used for initial configuration. In this guide we’ll be setting-up ethernet connection with a static IP address, but it is worth noting there are other ways to utilize the project.
While logged-in to the Pi via secure shell, enter:
sudo nano /etc/dhcpcd.conf
After entering your password, navigate to “Example static IP configuration” section:
Uncomment ip_address, routers and domain_name_servers lines.
Enter new intended static IP address in ip_address line. The line must end with /24, do not remove it. Enter the IP address of your network router under routers and domain_name_servers.
To find out your router IP, on Linux type-in ip route | grep default in a new terminal window. On Windows, run ipconfig and look for Default Gateway, and on a Mac – run netstat -nr | grep default
Press Control + X, Y to save changes.
Restart the system (using sudo reboot) to apply the new configuration.
Click Add Source > Add Network Device. Enter camera IP address:
Click Ok, and toggle Depth and RGB sensors on:
The device should work similar to standard Intel® RealSense™ camera, with depth, infrared and color streams available, standard sensor controls, point-cloud and texture mapping in the 3D view. For list of software limitations refer to section 3.5.
Intel® RealSense™ SDK 2.34 has introduced two new software components –
- realsense2-net module
The module exports single function rs2_create_net_device and encapsulates all dependencies required for working with network cameras
- rs-server tool
This application will connect to first available Intel RealSense camera and listen for incoming connection requests
No network code is compiled into core realsense2 module, limiting code size and security risks for customers who don’t need this functionality
Intel® RealSense™ SDK 2.0 is open-source and cross-platform software. The Network module follows the same guidelines and can be built from source as well.
In order to build realsense2-net module and the rs-server tool,
add -DBUILD_NETWORK_DEVICE=ON to the list of CMake flags when configuring the project. When building for the Raspberry Pi
-DFORCE_RSUSB_BACKEND=ON flag is also strongly recommended.
At the moment, rs-server can be only compiled on Linux but realsense2-net works on Windows, Linux and Mac OS. Integration with python, Android and other wrappers in librealsense ecosystem will be provided in future releases based on feedback from the community.
The project uses industry standard RTP protocol for video streaming.
In addition, RTSP protocol is used for camera control. Networking is handled by Live555 – open-source cross-platform library for low-power multimedia streaming. Depth compression is done using LZ4 , color and infrared streams are compressed using libjpeg-turbo open-source library .
Existing SDK examples and Apps will not work with a network device out of the box since network connection needs to be explicitly established.
That said, modifying existing code to work with a network camera is easy:
- Add #include <librealsense2-net/rs_net.hpp> to the code
- Inside CMakeLists.txt add realsense2-net to target_link_libraries
- Create network device from IP and add it to a context:
rs2::context ctx; dev.add_to(ctx);
- Pass context to pipeline constructor:
No additional code changes should be nessesary. From that point the SDK should treat network camera like any other Intel® RealSense™ camera.
This section outlines the considerations for key design decisions in this project.
RTP/RTSP protocols are used for streaming and control .
ROS ecosystem also offers good networking capabilities and may be a better choice if the client is also using ROS . ROS is, however, less portable, requires larger download and does not integrate easily into existing SDK applications.
Another significant alternative is GigE Vision – a well known industry format for network cameras . RTP/RTSP was chosen because of availability of lightweight, permissively open-sourced and portable implementation.
3.5.2 USB2 vs USB3
Different types of USB interface offer different advantages with USB3 bandwidth being significantly higher allowing for higher resolutions, and USB2 cables being more widespread and less sensitive to elegtromagnetic interference 
Depth data is compressed using LZ4 algorithm. Depth does not follow some of the assumptions of common lossy compression algorithm and using these algorithms on depth data can have destructive effects on 3D applications. This is why lossless algorithm was chosen .
Color and Infrared data are compressed using JPEG algorithm. While video compression algorithms such as MPEG can offer better compression rates, they are sensitive to frame drops . In addition, ffmpeg is industry standard for MPEG compression, but it is not as permissively licensed as other SDK components .
At the moment, the following features are not yet available for network device:
- D400 Advanced Mode API
- Dynamic and Self Calibration
- Firmware Update API
- IMU streaming support
- Integration with SDK Wrappers
In addition, to better match ethernet bandwidth limitation rs-server is only exposing the following camera modes:
Raw depth data
6, 15, 30
6, 15, 30, 60
Left and right infrared
6, 15, 30
6, 15, 30, 60
Color stream from left imager
6, 15, 30
6, 15, 30, 60
Color stream from RGB camera
6, 15, 30
6, 15, 30, 60
Table 2: Table of supported camera modes
Finally, current design is not addressing camera discovery, authentication and data encryption. For list of known issues please refer to SDK release notes 
This section assumes that the camera is configured with static IP 10.0.0.99
Restart the device:
ssh [email protected]
Restart camera service (rs-server):
ssh [email protected]
sudo systemctl restart rs-server.service
These are the steps required to update firmware of a remote camera:
- Download latest firmware from dev.intelrealsense.com/docs/firmware-releases
- Extract firmware image file
- Open secure shell session to the device:
ssh [email protected]
- Copy firmware binary from host to remote machine:
scp Signed_Image_UVC_5_12_3_0.bin [email protected]:/home/realsense/Signed_Image_UVC_5_12_3_0.bin
- Use rs-fw-update tool to update camera firmware:
rs-fw-update -f Signed_Image_UVC_5_11_6_250.bin
Updating server software requires building new version of the SDK from source and installing it on the device. See compilation instructions
Running the following commands can significantly improve network camera performance:
echo 8388608 > /proc/sys/net/core/wmem_default
echo 8388608 > /proc/sys/net/core/rmem_default
Raspberry Pi image comes pre-configured with these settings; however, we recommend also running this on the client side as well.
All measurements were captured using the following setup:
• Intel® RealSense™ Depth Camera D435
• Raspberry Pi 4 Model B 4GB running rs-server
• Intel® Core™ i7-8550U @ 1.8GHz running Ubuntu 16.04
• 3000x2000 60Hz Monitor
• Gigabit ethernet (Raspberry Pi, host PC and network switch)
rs-latency-tool is designed to estimate camera latency without special equipment. It can offer only an upper-bound on the actual camera latency. In practice results also depend on host performance and monitor refresh rate
Results vary from configuration to configuration, but overall show that the ethernet stack is adding on average between 1 to 1½ frames of extra latency:
RGB 640x480 30 FPS
112ms / 3.3 frames
160ms / 4.8 frames
RGB 1280x720 6 FPS
320ms / 1.9 frames
330ms / 2 frames
IR 640x480 30 FPS
93ms / 2.7 frames
130ms / 3.9 frames
IR 1280x720 6 FPS
166ms / 1 frame
330ms / 1.9 frames
Table 3: Table of latency measurements over Ethernet and USB
RGB sensor is rolling-shutter and by default actual FPS is controlled by auto-exposure algorithm. To reduce these effects, measurements were taken with manual exposure of 44ms and gain 93.
The following results show data from example runs of the system at various configurations with both Depth and RGB streams enabled. Device was restarted between runs.
• Power – power consumption in Watts
• CPU% - percent of overall system CPU utilization, over all cores, counting both kernel and userspace load
• Temperature – temperature of the CPU in degrees Celsius
• FPS – average FPS across 5 seconds intervals
• Jitter – number of instances per second where frame did not arrive within 5ms range of expected time
This guide presents steps needed to assemble and operate network depth-camera based on Intel® RealSense™ technology. We’ve reviewed design considerations, software architecture and basic performance results.
Streaming 3D content over ethernet is still active area of research. The following steps can further benefit this project:
• Leveraging better compression algorithms will allow more USB3 camera modes to be reliably supported over ethernet. Novel software and hardware 3D compression schemes are being continuously published and call for closer evaluation   .
• Designing and manufacturing IP65 enclosure with an M12 connector could greatly improve applicability for industrial use-cases .
• Computational resources of the Raspberry Pi can be utilized to perform additional post-processing tasks. Depth decimation can further reduce network bandwidth, while spatial and temporal filtering can improve depth quality. Furthermore, Intel® OpenVINO™ toolkit can be combined with Intel® RealSense™ SDK to run AI inference on the edge .
• RTP protocol allows for multi-client broadcasting. This opens new possibilities for sharing 3D data with wider audience over the Internet, for education and research purposes.
• Additional areas of research – pairing, camera discovery , authentication, data encryption and PTP time synchronization.
- LIPSedge™ AE400 Industrial 3D Camera
- FRAMOS Industrial Depth Camera D435e
- Aeon Up-Board and Power-over-Ethernet Hat
- Pi4 USB-C Gadget by Benjamin Hardill
- live555 Media Server
- LZ4 - Extremely fast compression algorithm
- libjpeg-turbo – optimized JPEG compression implementation
- Intel® RealSense™ Depth Camera over Ethernet
- CompTIA A+ Certification Guide – on differences between USB2 and USB3
- ROS Intel RealSense WIKI – using multiple cameras on two machines
- Wikipedia - GigE Vision
- JPEG and MPEG compression
- FFmpeg License and Legal Considerations
- KINOVA Gen3 Robots
- Pudu Robotics and Intel RealSense Technology
- Anybotics ANYmal C autonomous legged robot
- HEBI Robotics
- Mouser Electronics - M12 versus RJ45 Ethernet connection systems
- Depth Post-Processing for Intel® RealSense™ D400 Depth Cameras
- Intel® Distribution of OpenVINO™ toolkit with the Intel® RealSense™ Viewer
- Fast Lossless Depth Image Compression by Andrew D. Wilson (video)
- Intel RealSense D400 depth stream encoding to HEVC Main10 by Bartosz Meglicki
- Real-time decoding and AR playback of the emerging MPEG video-based point cloud compression standard
- Point Cloud Compression in MPEG - IEEE International Conference on Image Processing
- PTP/IP Adapter Design and Connectivity Techniques for Legacy Imaging Appliances
- Internet Protocols for Real-Time Multimedia Communication
- Intel® RealSense™ SDK 2.0 Release Notes
Updated about 1 year ago