This notebook is intended to showcase effect of post processing filters
!wget -N https://librealsense.intel.com/rs-tests/TestData/stairs.bag
print("Data is Ready")
Data is Ready
'wget' is not recognized as an internal or external command, operable program or batch file.
In order to extract and visualise the data we will need to import python libraries:
import numpy as np # fundamental package for scientific computing
import matplotlib.pyplot as plt # 2D plotting library producing publication quality figures
import pyrealsense2 as rs # Intel RealSense cross-platform open-source API
print("Environment Ready")
Environment Ready
# Setup:
pipe = rs.pipeline()
cfg = rs.config()
cfg.enable_device_from_file("stairs.bag")
profile = pipe.start(cfg)
# Skip 5 first frames to give the Auto-Exposure time to adjust
for x in range(5):
pipe.wait_for_frames()
# Store next frameset for later processing:
frameset = pipe.wait_for_frames()
depth_frame = frameset.get_depth_frame()
# Cleanup:
pipe.stop()
print("Frames Captured")
--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-6-ac59184d4988> in <module> 3 cfg = rs.config() 4 cfg.enable_device_from_file("stairs.bag") ----> 5 profile = pipe.start(cfg) 6 7 # Skip 5 first frames to give the Auto-Exposure time to adjust RuntimeError: Failed to resolve request. Request to enable_device_from_file("stairs.bag") was invalid, Reason: Failed to create ros reader: Error opening file: stairs.bag
colorizer = rs.colorizer()
colorized_depth = np.asanyarray(colorizer.colorize(depth_frame).get_data())
plt.rcParams["axes.grid"] = False
plt.rcParams['figure.figsize'] = [8, 4]
plt.imshow(colorized_depth)
When using Depth-from-Stereo solution, z-accuracy is related to original spacial resolution.
If you are satisfied with lower spatial resolution, the Decimation Filter will reduce spatial resolution preserving z-accuracy and performing some rudamentary hole-filling.
decimation = rs.decimation_filter()
decimated_depth = decimation.process(depth_frame)
colorized_depth = np.asanyarray(colorizer.colorize(decimated_depth).get_data())
plt.imshow(colorized_depth)
You can control the amount of decimation (liniar scale factor) via filter_magnitude
option.
Note the changing image resolution
decimation.set_option(rs.option.filter_magnitude, 4)
decimated_depth = decimation.process(depth_frame)
colorized_depth = np.asanyarray(colorizer.colorize(decimated_depth).get_data())
plt.imshow(colorized_depth)
Spatial Filter is a fast implementation of Domain-Transform Edge Preserving Smoothing
spatial = rs.spatial_filter()
filtered_depth = spatial.process(depth_frame)
colorized_depth = np.asanyarray(colorizer.colorize(filtered_depth).get_data())
plt.imshow(colorized_depth)
We can emphesize the effect of the filter by cranking-up smooth_alpha
and smooth_delta
options:
spatial.set_option(rs.option.filter_magnitude, 5)
spatial.set_option(rs.option.filter_smooth_alpha, 1)
spatial.set_option(rs.option.filter_smooth_delta, 50)
filtered_depth = spatial.process(depth_frame)
colorized_depth = np.asanyarray(colorizer.colorize(filtered_depth).get_data())
plt.imshow(colorized_depth)
The filter also offers some basic spatial hole filling capabilities:
spatial.set_option(rs.option.holes_fill, 3)
filtered_depth = spatial.process(depth_frame)
colorized_depth = np.asanyarray(colorizer.colorize(filtered_depth).get_data())
plt.imshow(colorized_depth)
Our implementation of Temporal Filter does basic temporal smoothing and hole-filling. It is meaningless when applied to a single frame, so let's capture several consecutive frames:
profile = pipe.start(cfg)
frames = []
for x in range(10):
frameset = pipe.wait_for_frames()
frames.append(frameset.get_depth_frame())
pipe.stop()
print("Frames Captured")
Next, we need to "feed" the frames to the filter one by one:
temporal = rs.temporal_filter()
for x in range(10):
temp_filtered = temporal.process(frames[x])
colorized_depth = np.asanyarray(colorizer.colorize(temp_filtered).get_data())
plt.imshow(colorized_depth)
You can modify filter options to fine tune the result (with any temporal filtering there is a trade-off between smoothing and motion)
Hole Filling filter offers additional layer of depth exterpolation:
hole_filling = rs.hole_filling_filter()
filled_depth = hole_filling.process(depth_frame)
colorized_depth = np.asanyarray(colorizer.colorize(filled_depth).get_data())
plt.imshow(colorized_depth)
These filters work best when applied sequentially one after another. At longer range, it also helps using disparity_transform
to switch from depth representation to disparity form:
depth_to_disparity = rs.disparity_transform(True)
disparity_to_depth = rs.disparity_transform(False)
for x in range(10):
frame = frames[x]
frame = decimation.process(frame)
frame = depth_to_disparity.process(frame)
frame = spatial.process(frame)
frame = temporal.process(frame)
frame = disparity_to_depth.process(frame)
frame = hole_filling.process(frame)
colorized_depth = np.asanyarray(colorizer.colorize(frame).get_data())
plt.imshow(colorized_depth)
Want to learn more? Visit realsense.intel.com and github.com/IntelRealSense