Intel® RealSense™ Developer Documentation

custom html

Calibration Tools User Guide for Intel® RealSense™ D400 Series

Revision 2.6.4.0

Also available in PDF format.

Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.
Intel technologies may require enabled hardware, specific software, or services activation. Check with your system manufacturer or retailer.
The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps.
Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or visit https://www.intel.com/content/www/us/en/design/resource-design-center.html .
Intel and the Intel logo Xeon, Core trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.

1. Introduction

1.1 Purpose and Scope of this Document

This document is a comprehensive user guide for calibration tools released as part of Intel® RealSense™ D400 Series Calibration Tools and API software release. The usage of calibration Tool API are separately covered in document “Intel® RealSense™ D400 Series Calibration Tools API programmer’s guide”

1.2 Components

The Intel® RealSense™ D400 Series Calibration Tools and API software release package includes:

  • Calibration Tools including Dynamic Calibrator
  • Calibration API libraries, headers, and sample application
  • Intel® RealSense™ Dynamic Target phone application

1.3 Class 1 Laser and Caution

The Intel® RealSense™ D400 series is classified as a Class 1 Laser Product under the EN/IEC 60825-1, Edition 3 (2014) internationally and IEC60825-1, Edition 2 (2007) in the US.
This product complies with US FDA performance standards under 21 CFR 1040.10 for laser products except for deviations pursuant to Laser Notice No. 50 dated June 24, 2007.

Caution

Use of controls or adjustments or performance of procedures other than those specified herein may result in hazardous radiation exposure.

Do not power on the product if any external damage was observed.
There are no service/maintenance, modification, or disassembly procedures for the stereo module and infrared projector. The system integrator must either notify Intel or return modules before any failure analysis is performed.

  • Do not attempt to open any portion of this laser product.
  • Invisible laser radiation when opened. Avoid direct exposure to beam.
  • There are no user serviceable parts with this laser product.
  • Modification or service of the stereo module, specifically the infrared projector, may cause the emissions to exceed Class 1.
  • No magnifying optical elements, such as eye loupes and magnifiers, are allowed.
  • Do not try to update camera firmware that is not officially released for specific camera module SKU and revision.

1.4 Hardware Requirements

Table 1-1. Supported Intel® RealSense™ Depth Modules and Depth Cameras

No.
Depth Module/Depth Cameras

1

Intel® RealSense™ Depth Module D400

2

Intel® RealSense™ Depth Module D410

3

Intel® RealSense™ Depth Module D415

4

Intel® RealSense™ Depth Camera D415

5

Intel® RealSense™ Depth Module D420

6

Intel® RealSense™ Depth Module D430

7

Intel® RealSense™ Depth Camera D435

The host processor connection to the camera is through USB 3.1 Gen 1.

Figure 1 1. Intel® RealSense™ Depth Camera D435 (Depth Module Integrated)

Figure 1 1. Intel® RealSense™ Depth Camera D435 (Depth Module Integrated)

Figure 1 2. Intel® RealSense™ Depth Module D410 with Vision Processor D4 Card (Non-integrated Bare Depth Module)

Figure 1 2. Intel® RealSense™ Depth Module D410 with Vision Processor D4 Card (Non-integrated Bare Depth Module)

1.5 Software Requirements

Table 1-3. Software Requirements

Software
Version

Intel® RealSense™ camera firmware

5.9.9.0 or higher

Linux*

64-bit host system supporting Ubuntu* version 16.04

Linux* Kernel

4.4.0 and later

Windows*

Windows 8.1 64-bit, Windows 10 64-bit

2. Overview

This chapter provides an overview of the calibration parameters and dynamic calibration process.

2.1 Calibration Parameters

Dynamic Calibration is optimizing extrinsic parameters, i.e., they refer to calibration done in the field at the user environment with minimal or no user intervention. They are ONLY extrinsic parameters (translation and rotation) of the camera image with regard to the main axis system (the axis between the left and right). Intrinsic parameters such as distortion, field of view, principal point are not dynamically calibrated.
Dynamic calibration is run under the assumption that it is the re-calibration of the depth modules/cameras after factory calibration, or at least that the nominal parameters are known.

The left camera is the reference camera and is located at world origin. RGB parameters only apply to depth modules/cameras with RGB sensor for color, e.g., D415 and D435.

Intrinsic includes

  • Focal length - specified as [fx; fy] in pixels for left, right, and RGB cameras
  • Principal point - specified as [px; py] in pixels for left, right, and RGB cameras
  • Distortion - specified as Brown's distortion model [k1; k2; p1; p2; k3] for left, right, and RGB cameras

Extrinsic includes

  • RotationLeftRight - rotation from right camera coordinate system to left camera coordinate system, specified as a 3x3 rotation matrix
  • TranslationLeftRight - translation from right camera coordinate system to left camera coordinate system, specified as a 3x1 vector in millimeters
  • RotationLeftRGB - rotation from RGB camera coordinate system to left camera coordinate system, specified as a 3x3 rotation matrix
  • TranslationLeftRGB - translation from RGB camera coordinate system to left camera coordinate system, specified as a 3x1 vector in millimeters

2.2 Types of Dynamic Calibration

Different types of dynamic calibrations are supported for Intel® RealSense™ D400 series

  1. Rectification calibration: aligning the epipolar line to enable the depth pipeline to work correctly and reduce the holes in the depth image
  2. Depth scale calibration: aligning the depth frame due to changes in position of the optical elements

Dynamic Calibration Tool API supports these algorithms in two distinctive operating modes: targeted and target-less. The Dynamic Calibrator supports only targeted calibration.

Targeted calibration is the recommended approach because it supports both rectification and depth scale calibrations and will give more accurate results than rectification only calibration done in target-less calibration.

2.2.1 Target Dynamic Calibration (Depth Scale Calibration)

In targeted mode, Dynamic Calibration API supports depth scale calibration and a target is required. The target is predefined and can be displayed on a smartphone through a phone app. A simplified flow is summarized as following:

  1. Take the images from L and R camera, including the depth stream (in real-time)
  2. Detect the target on the smartphone in both images
  3. Similarly to target-less calibration, user moves the phone so that it covered most of the image, repeating steps 1-2
  4. Once done, user just keeps taking images by positioning the phone anywhere in the image but must move the phone every time
  5. After taking 15 images in step 4, the process is complete
  6. The process checks for rectification error (absolute Y difference) but also compares the measured pattern size with ground truth

The advantage of targeted calibration is that it’s accurate and consistent. It calibrates left/right depth as well as RGB (on devices with RGB). The disadvantage is that it requires calibration target which means it cannot be used in cases where calibrating with a target is not feasible

2.2.2 Target-less Dynamic Calibration (Rectification Calibration)

In target-less mode, Dynamic Calibration API supports rectification calibration without the need of any target. Its basic flow is summarized as following:

  1. Take the images from L and R camera (in real-time)
  2. Extract features from the images
  3. Match the features between L and R camera
  4. The image is binned into 6x8 grid. Check whether each bin contains enough corresponding points.
  5. The user sees the panel status and moves the device around (bins without features are blue).
  6. Steps 1-6 are iterated until all bins have enough feature points
  7. Check rectification error (absolute Y difference). If too large, run a solver to optimize extrinsic parameters to minimize it.

Note

Targeted calibration is likely to give more accurate results and is therefore the recommended approach.

The advantage of target-less calibration is simplicity, no calibration target is needed. It fits with user cases where calibrating with a target may not be feasible. The disadvantage is that it calibrates left/right depth but not RGB and generally less accurate and less consistent than targeted calibration.

2.3 Depth Quality Tool

The Intel® RealSense™ SDK includes a Depth Quality Tool that can be used to test the quality of the Intel® RealSense™ D400 series depth module/cameras if it is suspected that the module has gone out of calibration. Refer to the SDK release page at https://github.com/IntelRealSense/librealsense/releases to download Depth Quality Tool.

3. Installation

The Intel® RealSense™ Dynamic Calibrator is supported on Linux (Ubuntu 16.04), Windows 8.1 64-bit and Windows 10 64-bit.

3.1 Installation on Linux (Ubuntu 16.04)

To open a terminal window, hit (Ctrl + Alt + T).

A working Internet connection is required for installing the 3rd party dependent libraries and patching the kernel. For systems behind a firewall or proxy server, proxy will also need to be configured. Please consult with your network administrator for details.

3.1.1 Install 3rd-party dependencies

  1. Ensure apt-get is up to date:
sudo apt-get update
  1. Install libusb-1.0:
sudo apt-get install libusb-dev libusb-1.0-0-dev
  1. Install ibglfw:
sudo apt-get install libglfw3 libglfw3-dev
  1. Install freeglut:
sudo apt-get install freeglut3 freeglut3-dev
  1. Install libpng12-dev:

On Ubuntu 16.04 and earlier, run the following command:

sudo apt-get install libpng12-dev

On Ubuntu 18.04, run the commands below:

wget -q -O /tmp/libpng12.deb http://mirrors.kernel.org/ubuntu/pool/main/libp/libpng/libpng12-0_1.2.54-1ubuntu1_amd64.deb && sudo dpkg -i /tmp/libpng12.deb && rm /tmp/libpng12.deb

3.1.2 Calibration Tools and API Package Installation

Calibration Tool API provides installation packages in dpkg format for Debian OS and its derivatives. The packages and their respective content are listed below:

Table 3-1: Calibration Tools and API Packages

Name
Content

librscalibrationtool

Intel® RealSense™ Dynamic Calibrator and other related tools

librscalibrationtool

Developer package including Calibration API and example sources

3.1.2.1 Debian Package Installation through Amazon AWS

The Debian packages are available on Amazon AWS and can be installed with the following steps.

  • Add Intel server to the list of repositories:
echo 'deb http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo xenial main' | sudo tee /etc/apt/sources.list.d/realsense-public.list

It is recommended to backup /etc/apt/sources.list.d/realsense-public.list file in case of an upgrade.

  • Register the server's public key:
sudo apt-key adv --keyserver keys.gnupg.net --recv-key 6F3EFCDE
  • Refresh the list of repositories and packages available:
sudo apt-get update 
  • Install the librscalibrationtool package which includes Intel® RealSense™ Dynamic Calibrator:
sudo apt-get install librscalibrationtool
  • (Optional) Developers shall also install librscalibrationapi package:
    This is optional. Regular user does not require this package. Only users who would like to develop third party calibration apps or customize the calibration apps for their specific usages need to install this package.
sudo apt-get install librscalibrationapi 

3.1.3 Checking Package Installation

After the debian package install finishes, check for files installed. Under debian convention, the executables are installed under /usr/bin, library files under /usr/lib, and other files including sample code under /usr/share/doc.

For example, for Calibration Tool, librscalibrationtool, the files are installed as below:

Using a terminal, navigate to the directory where the dynamic calibration tools release files are installed. For example:

cd /usr/share/doc/librscalibrationtool

For Calibration API, librscalibrationapi, if installed, the files are installed as below:

Using a terminal, navigate to the directory with the dynamic calibration API release files are installed. For example,

cd /usr/share/doc/librscalibrationapi

3.1.4 Video4Linux backend

  1. Ensure no cameras are presently plugged into the system.
  2. Patch and insert modified kernel drivers for Intel® RealSense™ Camera.
    Intel® RealSense™ SDK 2.0 provides dpkg package librealsense2-dkms for the kernel rules and kernel drivers. Please follow instructions in the following link for Ubuntu OS version supported and package installation details:
    https://github.com/IntelRealSense/librealsense/blob/development/doc/distribution_linux.md
    Please note generally on supported platforms, only the Realsense DKMS kernel driver package librealsense2-dkms is required for Dynamic Calibration. The rules are included in librealsense2-udev-rules which librealsense2-dkms depends and will be installed automatically.
sudo apt-get install librealsense2-dkms

3.1.5 Test Installation

Now check dynamic calibration tool version by running the following command:

The calibration tool executables are located under /usr/bin.

For example,
To print the tool version to screen:

/usr/bin/Intel.Realsense.DynamicCalibrator -v

To print device information to screen, connect a camera device to the system and execute the following command:

/usr/bin/Intel.Realsense.DynamicCalibrator -list

If you see the correct version information and device data printed on screen, congratulations you have the right setup and ready to use the calibration tools.

3.2 Installation on Windows

The Intel® RealSense™ Dynamic Calibration Tool runs on Windows 8.1 and Windows 10.

3.2.1 Install 3rd-party dependencies

Windows dynamic calibration package requires the following installed on the host system:
Microsoft Visual C++ 2015 Redistributable.

The install package installs this library automatically. So normally user will not need to install this separately.

3.2.2 Package installation

The release includes two install packages for Windows:

  • Calibration Tool: CalibrationTool-2.6.4.0-Setup.exe
  • Calibration API: CalibrationAPI-2.6.4.0-Setup.exe

3.2.2.1 Calibration Tool Windows Installer Package

The installer package compresses all contents into a single installer file, for example, CalibrationTool-2.6.4.0-Setup.exe. Simply click it to install in your system.
By default, it installs under C:\CalibrationToolAPI\2.6.4.0.

The final installation should look like following directory structure:

The installed directory contains two versions of the Dynamic Calibrator tool, both share same functionality but different user interfaces:

  • Intel.Realsense.DynamicCalibrator.exe with a graphical user interface
  • Intel.Realsense.DynamicCalibratorCLI.exe with a command line interface with command line options
    It also includes Intel.Realsense.CustomRW.exe tool which allows user to dump the calibration data from device, write calibration parameters to device, and reset the device calibration to default gold settings.
    The installer will create Dynamic Calibrator icon link to DynamicCalibrator.exe at the system desktop.

Double click the icon to start using the tool with its graphical user interface.
Of course, if command line interface preferred, users can always navigate to the install directory using a windows terminal.
For example,

cd CalibrationToolAPI
cd 2.6.4.0

3.2.2.2 Calibration API Windows Installer Package

This is optional. Regular user does not require this package. Only users who would like to develop third party calibration apps for the D400 devices or customize the calibration apps for their specific usages need to install this package.
The installer package compresses all contents into a single installer file, for example, CalibrationAPI-2.6.3.0-Setup.exe. Simply click it to install in your system.
By default, it installs under C:\CalibrationToolAPI\2.6.4.0.

By default, it installs to the same folder as Calibration Tool package. The following additional directories/files are added on top of Calibration Tool package:

4. Calibration with Dynamic Calibrator

Your camera might be out of calibration if you see the following symptoms:

  • Reduced depth density on objects in the operating range (might still get depth on vertical lines).
  • Flat surfaces look "wobbly", i.e. there is more deviation from the flatness than usual.
  • Measuring the physical distances to objects are not within about 3% of what they should be.

Once you determined the camera is out of calibration, the camera needs to be re-calibrated to correct the changes that might have occurred between the left and right depth sensors.

Dynamic Calibrator is under the bin directory. To support both regular users who prefer graphical user interface (GUI) and OEM/ODMs who prefer command line interface (CLI) for ease of automation, the tool supports both GUI and CLI interfaces.

  • On Linux, a single executable Intel.Realsense.DynamicCalibrator supports both GUI and CLI interfaces.
  • On Windows, two separate executables, both share same functionality but different user interfaces, are provided:
    • Intel.Realsense.DynamicCalibrator.exe with a graphical user interface
    • Intel.Realsense.DynamicCalibratorCLI.exe with a command line interface with command line options

4.1 Dynamic Calibrator Graphical User Interface (GUI)

The Graphical User Interface (GUI) is provided for easier use. The app detects all Intel RealSense D400 series devices on the system and lists them in a drop down box on the top left corner.

User selects one of the device from the list to calibrate:

Details on the device including product id, device name, serial number, and FW version displayed:

The calibration process requires a special target, either printed or phone target. Please refer to Appendix for target setup details.

By default, the app handles everything automatically. However, a few advanced settings are provided for experienced users who prefer to make custom choices.

For targeted calibration, user can customize AE (auto exposure) setpoint. Indoor and Outdoor are auto options that let the app to handle it automatically. The app will adjust automatically to the environment light conditions so that calibration target can be found without over exposure. The Manual option allow user to set a fixed setpoint value. This is useful in very complex environment where the app may experience difficulty to sweep through the setpoints automatically. The value set in such cases is completely dependent on the environment lighting, for example, 1000 - 2000 should be sufficient for indoor usage and 3000 – 4000 might work better outdoor. The exact value in user case can be found through trial and error.

User can also adjust the timeout period. User can adjust it in case an extended period is required in difficult situations:

The last step is to start calibration by clicking the “Start Calibration” button:

User can always stop/cancel calibration and return to main interface at any time by clicking the “Stop Calibration” button”:

During calibration, the UI displays the elapsed time in the top right corner, and status feedback on the lower left corner, and either the number of frames or target distance on the lower right corner:

Rectification Phase

Rectification Phase

Scale Phase

Scale Phase

4.2 Dynamic Calibrator Command Line Interface (CLI)

A number of command line options are supported in Dynamic Calibrator command line interface.

Table 4-1: Dynamic Calibrator Command Line Options

Option
Description

-help
-?

Display list of command line options.

-show
-list

Display list of connected cameras with device name, serial number and firmware version information.

-laser <0 or 1>
-l <0 or 1>

Target-less calibration only. Force turn on or off laser in target-less calibration.

-dump <0 or 1>
-d <0 or 1>

For debugging use. Dump calibration images and other data.
0 to dump only frames used in calibration and results to files.
1 to dump every frames received.

-setpoint <0 to 4095>
-s <0 to 4095>

Force AE setpoint (reference point). For dark environment, choose a low setpoint, 200, for example. For normal office lighting, a setpoint around 800 is appropriate. For outdoor bright sunlight environment, a higher setpoint is required, for example, 1200 or higher.

-version
-v

Show Calibration Tool API version info

‑serial < camera serial number>
-sn <camera serial number>

If multiple cameras connected to the current system, choose one of the camera devices to calibrate by specifying its serial number.

-timeout
‑t <Time out duration in seconds>

Customize time out duration. By default, calibration session will time out in 180 seconds for target-less and 300 seconds for targeted calibration.

-cli

Run app in CLI mode

-AE
-a

Disable auto exposure

-DE <Depth exposure value>
-de <Depth exposure value>

Set Depth Exposure

-RE <RGB exposure value>
-re <RGB exposure value>

Set RGB Exposure

-rgbOnly
-rgb

Calibrate RGB only, for debug purpose

The app source code is contained in the Sample folder.

4.3 Connect camera to host system

The Intel® RealSense™ camera to be calibrated should be connected to the system where dynamic calibrator software will run. This is usually done through a USB cable (for peripherals). On systems with camera integrated, it’s already connected, so no extra step is required.

4.4 Multiple camera support

If multiple cameras connected to the current system, dynamic calibrator allows user to list all camera devices on the system and choose one of the camera device to calibrate. Only a single camera device can be calibrated at a time.

4.4.1 List all camera devices with Graphical User Interface (GUI)

The app detects all Intel® RealSense™ D400 Series devices on the system and lists them in a drop down box on the top left corner.

User selects one of the device from the list to calibrate and details on the device including product ID, device name, serial number, and Firmware version is displayed:

4.4.2 List all camera devices with Command Line Interface (CLI)

To list all devices, use the –show or –list option in DynamicCalibrator.

For example:

/usr/bin/Intel.Realsense.DynamicCalibrator –show

Device Name                   Serial Number       Firmware Version
Intel RealSense 410           718212020356        05.08.09.00
Intel RealSense 430           722312071055        05.08.09.00
cd bin
Intel.Realsense.DynamicCalibrator.exe –show

Device Name                   Serial Number       Firmware Version
Intel RealSense 410           718212020356        05.08.09.00
Intel RealSense 430           722312071055        05.08.09.00

4.4.3 Perform calibration on selected device with Command Line Interface (CLI)

Then choose one of the camera devices to calibrate by specifying its serial number though the –sn option in DynamicCalibrator app.

For example,

/usr/bin/Intel.Realsense.DynamicCalibrator –sn 722312071055
Intel.Realsense.DynamicCalibratorCLI.exe –sn 722312071055

4.4.4 Stop Calibration

After calibration is started, user can stop the process at any time by doing any one of the following:
a) ESC key – press ESC key anytime in both GUI and CLI mode will stop the calibration process.
b) Or, in GUI mode, press the “Stop Calibration” button at top left corner of the calibration window.

c) Or, in CLI mode, press the “Q” or “q” key will quit the process
d) Or, simply close the calibration window

4.5 Targeted Calibration

4.5.1 Setup Calibration Target

Calibration Tools API supports printed target on letter sized paper, target displayed on iPhones and android phones. Please refer to details in Appendix INTEL® REALSENSE™ DYNAMIC CALIBRATION TARGET SETUP.

For example, a target displayed on iPhone 7 plus via Intel® RealSense™ Dynamic Target Tool app:

Figure 4-1. Calibration Target Displayed on Phone

The camera should be pointing to the target at a distance of approximately 50 cm to 100 cm. It should not be too close or too far. Due to various phone sizes, camera device models, and lighting conditions, exact distance cannot be specified. User will need to move the camera in the specified range to find a distance that works best. In most cases, a distance of around 70 cm is sufficient.

4.5.2 Running Dynamic Calibrator with targeted calibration

On Linux:

/usr/bin/Intel.Realsense.DynamicCalibrator

Be default, the application displays Graphical User Interface (GUI). It detects all Intel RealSense D400 Series devices on the system and lists them in a drop down box on the top left corner. Selects one of the device from the list to calibrate. Details of the device is displayed as well:

By default, “Use Target” is selected, so the tool will run in targeted calibration mode. This requires a special target setup described in previous section “Setup Calibration Target”. If you haven’t done that, please follow instructions and complete the target set up either on a printed target or on a phone target.

The “Show Demo” button will highlight the key steps of the calibration process. This is also detailed in Appendix “Calibration process Highlights”.

Next, start calibration by clicking the “Start Calibration” button:

If command line interface is preferred, user can display DynamicCalibrator command line options by running:

/usr/bin/ Intel.Realsense.DynamicCalibrator -help

And specify appropriate options on the command line. The GUI control panel is not displayed in command line mode. Calibration process starts immediately by default.

On Windows:
If the application is installed through an installer, an app icon “Dynamic Calibrator” should be already created on your desktop.

Double click the icon to start the app.

By default, the app starts with Graphical User Interface (GUI). It detects all Intel RealSense D400 devices on the system and lists them in a drop down box on the top left corner. Selects one of the device from the list to calibrate. Details on the device displayed as well:

This calibration process requires a special target setup described in previous section “Setup Calibration Target”. If you haven’t done that, please follow instructions and complete the target set up either a printed target or phone target.

Next to start calibration by clicking the “Start Calibration” button:

If command line interface is preferred, user can display DynamicCalibrator command line options by running:

cd bin
Intel.Realsense.DynamicCalibratorCLI.exe -help

And specify appropriate options on the command line. The GUI control panel is not displayed in command line mode. Calibration process starts immediately by default.

cd bin
Intel.Realsense.DynamicCalibratorCLI.exe

When DynamicCalibrator calibration process starts, it briefly displays a logo screen:

Figure 4-2. Dynamic Calibrator Initial Start Window

Figure 4-2. Dynamic Calibrator Initial Start Window

Followed by a warm up screen that indicates progress in getting the camera ready for calibration.

Figure 4-3. Dynamic Calibrator app warm up window

Figure 4-3. Dynamic Calibrator app warm up window

4.5.3 Auto Exposure Adjustment during Targeted Calibration

Throughout the targeted calibration process, exposure is adjusted automatically. The tool tries to detect and recognize the calibration target in all lighting conditions. When a target is not found, sometimes because the surrounding environment has a reflection on the phone screen or the target is too dark/bright to detect, it will adjust exposure automatically so that a target is detected. So from time to time, you may see the exposure changes from dark to bright and gradually reduces brightness. This is expected behavior.

In some cases, if the tool adjusted too bright or too dark, and the tool can’t make progress, you can point the camera away from the target and wait a couple seconds until the brightness comes to a normal level and then point back to the target.

4.5.4 Target calibration phases

Target calibration process consists of two phases – rectification and scale calibration in sequential order.

For the first phase, rectification, a block of be squares are present in middle part of the window:

Figure 4-4. Dynamic Calibrator Targeted Rectification Phase

Figure 4-4. Dynamic Calibrator Targeted Rectification Phase

Point the camera to the calibration target and move camera slowly to position target over the blue squares until all are cleared:

Figure 4-5. Dynamic Calibrator Targeted Rectification Phase (continued)

Figure 4-5. Dynamic Calibrator Targeted Rectification Phase (continued)

Figure 4-6. Dynamic Calibrator Targeted Rectification Phase (continued)

Figure 4-6. Dynamic Calibrator Targeted Rectification Phase (continued)

Once all blue squares are cleared, rectification phase is completed. Intermediate result is applied to the current stream.

Scale calibration phase will start immediately afterwards. Point the camera to the target and move slowly. A green progress bar will appear as it makes progress in scale calibration. Target images will be captured and analyzed and accepted target positions highlighted on screen. Total 15 successful target images will be accepted.

Figure 4-7. Dynamic Calibrator Scale Phase

Figure 4-7. Dynamic Calibrator Scale Phase

Figure 4-8. Dynamic Calibrator Scale Phase (continued)

Figure 4-8. Dynamic Calibrator Scale Phase (continued)

After the green bar extends to full, scale calibration completes.

On D435 and D415 devices, an extra step will be run to calibrate the RGB camera. The usage is similar as the depth scale phase.

Depends on what user interface (GUI or CLI) used, result will be either on the GUI status box or displayed in the command line window. Results will be updated to device automatically. Success or fail status will also be displayed along with the results.

Results update in GUI mode.

Results update in GUI mode.

Results update in CLI mode.

Results update in CLI mode.

This completes the targeted calibration process.

5. Handling Calibration Data with CustomRW Tool

While users calibrate devices with dynamic calibrator or through their own developed calibration tools based on Calibration API, there might be a need to read, write, and restore calibration data on the device. CustomRW Tool provides this ability to users. CustomRW Tool is included in the Calibration Tool package and is installed by default under C:\CalibrationToolAPI\2.5.2.0\bin on Windows and /usr/bin on Linux.

The tool supports three options:

5.1 Restore Calibration on Device

The -g option restores the calibration on device to default gold settings. Depending up on the type of calibration done on the device, the gold settings represents the original factory setting or the calibration setting from OEM Calibration if such a calibration is done on the device after factory calibration.

5.2 Read Calibration from Device

The tool supports -r option to read the calibration parameters from the device. By default, the parameters are printed on screen. If a file name is provided, the parameters can also be dumped into a file in XML format.

The calibration parameters are defined in section 2.1 “Calibration Parameters”. The XML file format is as below:

<?xml version="1.0"?>
<Config>
    <param name = "ResolutionLeftRight">
        <value>1920</value>
        <value>1080</value>
    </param>
    <param name = "FocalLengthLeft">
        <value>1374.95</value>
        <value>1375.3</value>
    </param>
    <param name = "PrincipalPointLeft">
        <value>969.179</value>
        <value>533.098</value>
    </param>
    <param name = "DistortionLeft">
        <value>0.130383</value>
        <value>-0.416597</value>
        <value>-0.00264688</value>
        <value>-0.000169552</value>
        <value>0.371485</value>
    </param>
    <param name = "FocalLengthRight">
        <value>1384.39</value>
        <value>1384.67</value>
    </param>
    <param name = "PrincipalPointRight">
        <value>956.174</value>
        <value>542.892</value>
    </param>
    <param name = "DistortionRight">
        <value>0.11961</value>
        <value>-0.372387</value>
        <value>-0.00278001</value>
        <value>0.000241054</value>
        <value>0.337878</value>
    </param>
    <param name = "RotationLeftRight">
        <value>0.999994</value>
        <value>0.00203031</value>
        <value>-0.00275904</value>
        <value>-0.00202979</value>
        <value>0.999998</value>
        <value>0.000190069</value>
        <value>0.00275942</value>
        <value>-0.000184468</value>
        <value>0.999996</value>
    </param>
    <param name = "TranslationLeftRight">
        <value>-55.1674</value>
        <value>0.0539346</value>
        <value>-0.182312</value>
    </param>
    <param name = "HasRGB">
        <value>1</value>
    </param>
      <param name = "ResolutionRGB">
        <value>1920</value>
        <value>1080</value>
    </param>
    <param name = "FocalLengthRGB">
        <value>1385.99</value>
        <value>1386.17</value>
    </param>
    <param name = "PrincipalPointRGB">
        <value>951.048</value>
        <value>530.789</value>
    </param>
    <param name = "DistortionRGB">
        <value>0</value>
        <value>0</value>
        <value>0</value>
        <value>0</value>
        <value>0</value>
    </param>
    <param name = "RotationLeftRGB">
        <value>0.999994</value>
        <value>0.00334682</value>
        <value>0.000991736</value>
        <value>-0.00334019</value>
        <value>0.999973</value>
        <value>-0.00660826</value>
        <value>-0.00101383</value>
        <value>0.0066049</value>
        <value>0.999978</value>
    </param>
    <param name = "TranslationLeftRGB">
        <value>14.9153</value>
        <value>-0.320785</value>
        <value>-0.523442</value>
    </param>
</Config>

5.3 Write Calibration to Device

User can come up with the calibration parameters from their own calibration algorithm or from previous backups of the calibration data in XML format and write it to the device through the -w option.

6. Troubleshooting

Table 6-1. Troubleshooting

Issue
Resolution

With targeted calibration on Linux, after rectification phase is successfully completed, sample app windows may become black

The most like cause is the kernel not improperly patched or os booted into the non-patched kernel. Please follow instructions to redo the patching in installation section Video4Linux backend.

By default, Ubuntu auto kernel update feature is enabled. So you may experience the same screen blacking issue again if your Ubuntu kernel is automatically updated.

To avoid this, you can choose to do one of the following:

Method 1 - Pin/hold your kernel
Examples:
Discover the current latest kernel currently installed. It’s the one with the biggest version number.

$ dpkg -l | grep linux-image
$ uname -r
4.4.0-79-generic
$ sudo apt-mark hold linux-image-4.4.0-79-generic
linux-image-4.4.0-79-generic set on hold.
$ sudo apt-mark hold linux-headers-4.4.0-79-generic
$ sudo apt-mark hold linux-image-extra-4.4.0-79-generic
linux-image-extra-4.4.0-79-generic set on hold.
$ dpkg --get-selections | grep linux-image-4.4.0-79-generic
linux-image-4.4.0-79-generic hold

Method 2 - Disable automatic updates on Ubuntu:

Open the Unity Dash
Search for ‘Software & Updates’
Select the ‘Updates’ tab.
Change ‘Automatically check for updates’ from ‘Daily’ to ‘Never‘.
This setting will stop the system from checking for ANY updates

In certain cases, when the device is badly out of calibration or negative effect of lighting conditions, dynamic calibration may not be able to find the right correction and complete the process.

Try to find a different scene or environment to run dynamic calibration again. If multiple try do not resolve the issue, the device may need to be recalibrated with technician calibration or factory calibration.

Devices calibrated through targeted dynamic calibration shows greater depth error than expected (i.e., more than 2%)

Follow target setup instruction to check the target physical sizes. Target size is critical to calibration accuracy.

With modules with old calibration tables, target calibration does not change calibration on device

A simple workaround is to perform a target-less dynamic calibration on the device first and then targeted calibration would have no issue. Additional calibration would have no such limitation.

7. Appendix A - Calibration Check

This section describes two methods to check the camera calibration. The first method is a quick and easy way to check the alignment of left and right cameras. The second method is to test the accuracy of depth measurement.

The two methods are explained below:

7.1 Quick Check

With the quick check, the user should view the depth image from the camera with a tool that can display depth, Intel® RealSense™ Viewer or Depth Quality Tool in Intel® RealSense™ SDK 2.0. https://github.com/IntelRealSense/librealsense/releases/latest

Point the camera to a flat surface such as a wall about 1 to 2 meters away (3 to 6 feet). Avoid black surfaces.

Visually inspect the depth image display of the wall. A lot of black dots or holes on the image is an indication of the camera being out of calibration.

Figure 7-1. Calibration Check – Camera in Calibration

Figure 7-1. Calibration Check – Camera in Calibration

Figure 7-2. Calibration Check – Camera Out of Calibration

Figure 7-2. Calibration Check – Camera Out of Calibration

7.2 Accuracy Check

This procedure should be used to check the accuracy of the camera. Place the camera in parallel to a flat wall and exactly two meter (2000 mm) away.

Once the camera is placed in its position, Use Intel® RealSense™Viewer or Depth Quality Tool to measure the absolute distance. For a flat surface at a distance of 2 meter the absolute distance should be within 2% or better at 2 meter (2000mm).

If the distance is not within the defined range, then the camera needs to be calibrated.

8. Appendix B - Calibration Process Highlights

8.1 Targeted Calibration

Targeted calibration requires setting up a special target either printed on paper or displayed on phone screen through "Intel RealSense Dynamic Target Tool" app on iPhones or Android phones. The process goes through multiple phases to correct various calibration errors on the device, including rectification and scale in depth and depth to RGB on devices supporting RGB. Targeted calibration should be used to calibrate all D400 Series devices if a target setup is permitted.

9. Appendix C - Intel® RealSense™ Dynamic Calibration Target Setup

9.1 Introduction

Dynamic calibration is used to re-calibrate the Intel® RealSense™ 3D camera. As part of the process, the algorithm requires a vertical stripped bar image as calibration target. The target image can be printed on 8.5x11 letter size paper or displayed on an Apple or Android phones through the Intel® RealSense™ Dynamic Target Tool phone apps.

This section provides instructions to setup the target for dynamic calibration.

9.2 Dynamic Calibration Flow

The figuration below shows the overall dynamic calibration process flow.

Figure 9-1. Dynamic calibration flow

Figure 9-1. Dynamic calibration flow

  1. A target image with multiple vertical bars and black and white blocks is presented on a mobile phone via the Intel® RealSense™ Dynamic Target Tool app or printed target.
    • Phone target - the target image is dynamically generated on the fly at runtime. The image will be exact match to screen resolution and its content rendered in such that the bar widths are exactly 10 mm (except the most left and right bars on the edges) and the vertical distance between the top black blocks and bottom black blocks also meet certain requirement.
    • Printed target - The target can be printed on standard 8.5”x11” paper and taped to a flat surface.
  2. The target (either a printed, or displayed on phone) is placed about 50-70 cm (for AWG modules, closer 30 – 80 cm) away in front of the Intel Realsense Camera as straight as possible.
  3. The calibration app acquires the target images and perform scale calibration on the Intel RealSense Camera.

An example of the fixed width bar target image is shown below. The actual image depends on scenario – printed, iPhone or Android phone, and screen sizes.

Figure 9-2. Target Image

Figure 9-2. Target Image

Targets printed on paper or displayed on phone/tablet screens with the Intel® RealSense™ Dynamic Target Tool app on Android and IOS phone devices are supported.

9.3 Printed Target Setup

  1. Print target on letter size pager
    • A print target image v2.1, print-target-fixed-width.pdf, is supplied in the dynamic calibration package under the target directory.
    • Please print with regular laser printer on 8.5” x 11” letter size paper, choose “actual size” in printer options, no scaling.
Figure 9-3. Target Print Options

Figure 9-3. Target Print Options

  • After the target is printed, please check the physical target image on the paper. The specific dimensions are marked around the image.
Figure 9-4. Target Verification

Figure 9-4. Target Verification

  • Target precision is critical to calibration accuracy. Please verify key features below with a ruler after printed.
    • Target size - overall image should be 68.4 mm wide (A) and 121.6 mm tall (B) (same size as iPhone 7 plus 5.5” display)
    • Bar size - the 5 vertical bars in the middle (3 black and 2 white, in the order, black-white-black-white-black) in equal spacing each 10 mm wide, total 50 mm wide (C), and the vertical bar length is 100.0 mm (D)
    • Setup the printed target on a flat surface so that the bars are vertical (use a roller or some other object, to ensure the target is flat on the surface. Tape the printed target on the flat surface after confirming that the target is flat).
Figure 9-5. Printed target setup

Figure 9-5. Printed target setup

  1. Place the camera about 50 – 100 cm away and point to the target. It should not be too close or too far. Due to various phone sizes, camera device models, and lighting conditions, exact distance cannot be specified. User will need to move the camera in the specified range to find a distance that works best. In most cases, a distance of around 70 cm is sufficient.

  2. On the host, start Intel RealSense Dynamic Calibrator app. Refer to section Calibration with Dynamic Calibrator App in this document for details.

On Linux:
/usr/bin/Intel.Realsense.DynamicCalibrator

On Windows:
Double-click the “Dynamic Calibrator” icon on Windows desktop to start the calibrator app.

9.4 Phone Target Setup on Apple iPhones Devices

For Apple iPhone devices, Intel® RealSense™ Dynamic Target Tool app is available on Apple App Store for iPhones with IOS 10.0 and later.

  • App full name: Intel® RealSense™ Dynamic Target Tool
  • App short name: Dynamic Target Tool
    Supported device include iPhone 5, 5s, 6, 6s, 6s+, 7, 7+. Latest iPhones (iPhone 8, iPhone 8 plus, and iPhone X) not yet supported, will be supported in future update. Please refer to appendix Validated Devices for Intel® RealSense™ Dynamic Target Tool Phone App for details.

9.4.1 Locate the app on Apple App Store

A few simple ways to find the app on Apple App Store:

9.4.2 Install the App on iPhone

Once the app is found on the app store, the following informational pages display

Press the download/install icon to install the app on your phone.

App installed and the Dynamic Target Tool icon appears on phone screen.

9.4.3 Running the app on phone

Press the app icon to start Intel® RealSense™ Dynamic Target Tool, a brief splash screen with Intel® RealSense™ logo should appear:

After a brief moment (~ 1 second), stripped bar target should display. This is the target chart for dynamic calibration. The bars are showed vertically. The width of the black bars should be equal and always 10 mm. The number of bars can be varying depends on size of the device screen. Please refer to section “Target Image Accuracy Check” in appendix “Intel® RealSense™ Dynamic Target Tool Phone App Technical Specification” for details.

There is hidden information page that can show the device display details. This is not normally needed, but a good information in case user need to verify. While in the Intel® RealSense™ Dynamic Target Tool app, swipe from left to right, or touch on the left edge will display this information page. Swipe again from right to left or touch again will dismiss the information page and return the full screen to target.

9.4.4 Perform Dynamic Calibration

Place the camera about 50 – 100 cm away and point to the phone target. It should not be too close or too far. Due to various phone sizes, camera device models, and lighting conditions, exact distance cannot be specified. User will need to move the camera in the specified range to find a distance that works best. In most cases, a distance of around 70 cm is sufficient.

On the host, start Intel RealSense Dynamic Calibrator app. Refer to section Calibration with Dynamic Calibrator App in this document for details.

On Linux:
/usr/bin/Intel.Realsense.DynamicCalibrator

On Windows:
Double click the “Dynamic Calibrator” icon on Windows desktop to start the calibrator app.

9.5 Phone Target Setup on Android Phone Devices

For Android phone devices, Intel® RealSense™ Dynamic Target Tool app is also available on Google Play Store for phones with Android 4.4 and later.

  • App full name: Intel® RealSense™ Dynamic Target Tool
  • App short name: Dynamic Target Tool
    Tested devices include Samsung Galaxy S8 plus, Samsung Galaxy S7, Google Nexus 5, LG G4, Notes 5, and Motorola XT1650. Please refer to appendix Validated Devices for Intel® RealSense™ Dynamic Target Tool Phone App for details.

9.5.1 Locate the app on Google Play Store

A few simple ways to find the app on Google Play Store:

10.5.2 Install the App on Android Phone

Once the app is found on the app store, the following informational pages display

Press the INSTALL button to install the app on your phone.

App installed and the Dynamic Target Tool icon appears on phone screen:

9.5.3 Running the app on phone

Press the app icon to start Intel® RealSense™ Dynamic Target Tool, a brief splash screen with Intel® RealSense™ logo should appear:

After a brief moment (~ 1 second), stripped bar target should display. This is the target chart for dynamic calibration. The bars are showed vertically. The width of the black bars should be equal and always 10 mm. The number of bars can be varying depends on size of the device screen. Please refer to section “Target Image Accuracy Check” in appendix “Intel® RealSense™ Dynamic Target Tool Phone App Technical Specification” for details.

There is hidden information page that can show the device display details. This is not normally needed, but a good information in case user need to verify. While in the Intel® RealSense™ Dynamic Target Tool app, swipe from left to right, or touch on the left edge will display this information page. Swipe again from right to left or touch again will dismiss the information page and return the full screen to target.

9.5.4 Perform Dynamic Calibration

Place the camera about 50 – 100 cm away and point to the phone target. It should not be too close or too far. Due to various phone sizes, camera device models, and lighting conditions, exact distance cannot be specified. User will need to move the camera in the specified range to find a distance that works best. In most cases, a distance of around 70 cm is sufficient.

On the host, start Intel RealSense Dynamic Calibrator app. Refer to section Calibration with Dynamic Calibrator App in this document for details.

On Linux:
/usr/bin/Intel.Realsense.DynamicCalibrator

On Windows:
Double-click the “Dynamic Calibrator” icon on Windows desktop to start the calibrator app.

10. Appendix D - Validated Devices for Intel® RealSense™ Dynamic Target Tool Phone App

The Intel® RealSense™ Dynamic Target Tool phone app is validated on a limited number of mobile devices.

10.1 Validated IOS Devices

On Apple iPhones, display resolution and pixel density is limited to a few combinations, so although validation is done only on limited number of devices, it covers all display combinations in existing Apple iPhones (except the latest iPhone 8 and iPhone X models).

Apple Device Displays
Resolution (pixel)
DPI (PPI)
Screen Physical Dimension (cm) WidthxHeight
Devices with this display
Devices tested formally in dev and QA
Devices tested ad hoc from others

4”

640 x 1136

326

4.99 x 8.85

iPhone 5/5s/5c/SE

iPhone 5s

iPhone 5

4.7”

750 x 1334

326

5.88 x 10.46

iPhone 6/6s/7

iPhone 6

iPhone 6s/7

5.5”

1080 x 1920

401

6.84 x 12.16

iPhone 6 Plus/6s Plus/7 Plus

Iphone 6 Plus/7 Plus

iPhone 6s Plus

10.2 Validated Android OS devices

Display resolution and pixel density varies from device and vendors on Android devices. The target image is dynamically generated to fit to the display on each device. The image generation algorithm is precise in case the display resolution and pixel density is accurate. However, there are known cases where the OS reports wrong pixel density that would impact target accuracy. It’s important to physically check the target sizes in order to identify such issue early on.

Device tested formally in dev and QA
Android OS

Samsung Galaxy S7

6.0.1 (Marshmallow)

Google Nexus 5

4.4.2 (KitKat)

LG G4

6.0 (Marshmallow)

Samsung Galaxy Note 5

6.0 and 7.0

Devices tested ad-hoc from others

Google Nexus 6

Samsung Galaxy S7 Edge

Samsung Galaxy S8 Plus

10.3 Devices with accuracy issue in phone target

The following table lists devices that are known to generate inaccurate targets. Please avoid using these devices:

Device with target accuracy issue
Andoir OS
Notes

Samsung Galaxy S5

5.1.1

Target on Galaxy S5 phone not accurate

Calibration Tools User Guide for Intel® RealSense™ D400 Series


Revision 2.6.4.0

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.