VPipeline C++ library
v4.0.12
Table of contents
- Overview
- Versions
- Library files
- Video tracker centering mode
- Pan and tilt unit control notes
- VPipeline class description
- Enums
- VPipelineParams class description
- Build and connect to your project
- Portability
- Example
Overview
The VPipeline C++ library is the full functional video processing pipeline with interfaces to connect different video filters, video trackers, object detectors and video stabilization modules. The motivation of VPipeline library is to provide simple interface to complex video processing pipeline as well as camera and lens control, regardless of hardware. The library strikes two main goals: provide video processing pipeline and provide remote control interface for pipeline (encoding / decoding commands and telemetry). It can be used inside edge device for video processing as well as on client side to control video processing pipeline. The VPipeline library allows users to set their own implementation of particular algorithms through interfaces (VFilter, VTracker, VStabiliser, VSource and ObjectDetector). Thus the library offers flexibility in algorithms usage with keeping standard interfaces to control each of them. The library depends on (the repositories of the libraries must be placed next to the VPipeline repository, see Build and connect to your project): Frame library (describes video frame structure and pixel formats), ConfigReader library (provides JSON serialization of the parameters), Camera library (provides interface to control / obtain camera / image sensor parameters), Lens library (provides interface to control / obtaining lens parameters), ObjectDetector library (provides interface for object detectors), VSource library (provides interface for video capture), VTracker library (provides interface for video tracking), VStabiliser library (provides interface for video stabilization), VFilter library (provides interface for video filtering), Logger (provides functions for logging) and FormatConverter (provides functions to convert pixel formats). The VPipeline library doesn’t include implementation of particular video processing algorithms, but includes interfaces for them. The library uses C++17 standard and doesn’t have any dependency to be specially installed in OS. The library compatible with Linux and Windows. Video processing pipeline structure:
VPipeline is general-purpose software: it is not tied to any sector or application, and the developer of a program decides what the program does with it.
The processing inside VPipeline is distributed between multiple threads with synchronization between them. Each processing thread is responsible for particular algorithm (video filter, stabilization, video tracking and detection). User can initialize: camera controller module, lens controller module, video stabilizer module, video tracker module, up to seven video filters modules and up to 7 object detector modules. The library implements video processing pipeline (conveyor). The library includes following video processing steps: waiting next video frame from video source, converting source pixel format to YUV24, filtering frame by video filter 0, filtering frame by video filter 1, filtering frame by video filter 2, video stabilization, filtering frame by video filter 3, filtering frame by video filter 4, filtering frame by video filter 5, video tracking with frame centering in tracking mode if required, filtering frame by video filter 6, and after - object detection by up to 7 detectors. User can get video frame from pipeline in YUV24 format after video filter 6. Each steps waits results from previous and notifies next after processing. Each step performs processing in separate thread. The video processing FPS is determined by the FPS of the video source, as well as the slowest module, except for object detectors (if video processing in any module exceeds the time period between frames).
The VPipeline library provides simple interface to control pipeline including camera and lens control. Usage principles: user initializes pipeline by calling initVPipeline(…) method. The initVPipeline(…) method accepts big structure of parameters which includes parameters for all functional modules of the pipeline. Also user should put pointer to objects: video stabilizer, video tracker, object detectors (up to 7), video source, camera controller, lens controller and video filters (up to 7). If particular pointer not initialized the pipeline will initialize default “dummy” module which does nothing. After initialization the pipeline runs internal processing threads. User can control pipeline by changing parameters (setParam(…) method), executing command (executeCommand(…) method) and setting detection and filtering masks for object detectors and video filters (setMask(…) method). All this methods are non-blocking and thread-safe. To get all actual pipeline params including tracking and detection results user uses getParams(…) method. To get new video frame from video source after preprocessing (after video filter 6) the getFrame(…) method used. Thus the user have standard interface to camera, lens, video source, video tracker and others regardless of hardware type. VPipeline class provides the same set of commands and functions for any combination of lens and camera. If user doesn’t use lens and camera interface the pipeline can work without it and without any other functional module if needed. The VPipeline library simplifies embedded video processing software structure. Usage example:
Versions
Table 1 - Library versions.
| Version | Release date | What’s new |
|---|---|---|
| 1.0.0 | 18.05.2022 | First version. |
| 2.0.0 | 18.02.2024 | - Interface changed. - Added new video processing modules. - Video streaming functions excluded. - Documentation updated. |
| 3.0.0 | 14.03.2024 | - Full functional implementation. - Added VFilter interface. - Added test application. - Documentation updated. |
| 3.0.1 | 26.03.2024 | - Camera class updated. - FormatConverterOpenCv class updated. - Lens class updated. - Logger class updated. - ObjectDetector class updated. - VFilter class updated. - VSource class updated. - VStabiliser class updated. - VTracker class updated. - Documentation updated. |
| 3.0.2 | 21.05.2024 | - Submodules updated. - Documentation updated. |
| 3.0.3 | 08.08.2024 | - CMake structure updated. - Minor casting issues fixed. |
| 3.0.4 | 29.09.2024 | - Update VStabiliser interface. |
| 3.0.5 | 03.04.2025 | - Logger submodule update. |
| 3.0.6 | 18.04.2025 | - VStabiliser submodule update. |
| 4.0.0 | 04.01.2026 | - Excluded opencv dependencies. - Added additional slots for filters and object detectors. - Added video tracking centering mode. |
| 4.0.1 | 13.03.2026 | - Fixed time management in DummyVSource module. - Fixed cycle elapsed time calculation precision in main pipeline. |
| 4.0.2 | 16.03.2026 | - Added new option to get last frame in getFrame(…) method. |
| 4.0.3 | 28.03.2026 | - FormatConverter submodule updated. |
| 4.0.4 | 06.05.2026 | - Added video source reconnection in case capture errors. |
| 4.0.5 | 20.05.2026 | - Removed video source close duplication. |
| 4.0.6 | 06.06.2026 | - Improved video source auto re-initialization. |
| 4.0.7 | 12.08.2026 | - FormatConverter submodule updated. |
| 4.0.8 | 23.08.2026 | - FormatConverter submodule updated. Set default number of threads for format converter as max mum threads in the OS - 2. |
| 4.0.9 | 02.09.2026 | - FormatConverter submodule updated. |
| 4.0.10 | 02.09.2026 | - Submodules updated. |
| 4.0.11 | 19.10.2026 | - Git submodules excluded: the libraries VPipeline depends on are expected next to the repository, the 3rdparty folder removed. - “Dummy” modules moved to the dummy folder. - OpenCV excluded from the GUI test application: SimpleUI, SimpleFileDialog, VSourceFile and VideoRecorderMp4 libraries are used. - Decoding of params and commands: pointers, sizes, section sizes, module IDs and IDs of general module are checked, the params object is not changed if the data is not valid. - Values of general params are checked before conversion to int. - Video tracker centering protected from not valid frames and positions, exceptions of modules do not stop processing threads, stop of processing threads fixed. - Compiler and linker flags for warnings and exploit mitigation added, compiler extensions disabled, not applied high performance flags removed. - “Dummy” video source: thread-safe params, objects clipped to the frame, own pseudo random generator (the same test pattern on all platforms). - Protocol test application rewritten, documentation updated. |
| 4.0.12 | 25.10.2026 | - The library links the thread library (CMake target Threads::Threads): programs built on a C library older than glibc 2.34 no longer need -pthread. |
Library files
The VPipeline C++ library is provided as source code. Users are given a set of files in the form of a CMake project (repository). The repository structure is outlined below:
CMakeLists.txt --------------------------- Main CMake file of the library.
dummy ------------------------------------ Folder with "dummy" implementations of the functional modules.
DummyCamera -------------------------- Folder with "dummy" camera implementation.
CMakeLists.txt ------------------- CMake file for "dummy" camera.
DummyCamera.cpp ------------------ Source code file of the library.
DummyCamera.h -------------------- Header file of the library.
DummyCameraVersion.h ------------- Header file with library version.
DummyCameraVersion.h.in ---------- CMake service file to generate version header.
DummyLens ---------------------------- Folder with "dummy" lens implementation.
CMakeLists.txt ------------------- CMake file for "dummy" lens.
DummyLens.cpp -------------------- Source code file of the library.
DummyLens.h ---------------------- Header file of the library.
DummyLensVersion.h --------------- Header file with library version.
DummyLensVersion.h.in ------------ CMake service file to generate version header.
DummyObjectDetector ------------------ Folder with "dummy" object detector implementation.
CMakeLists.txt ------------------- CMake file for "dummy" object detector.
DummyObjectDetector.cpp ---------- Source code file of the library.
DummyObjectDetector.h ------------ Header file of the library.
DummyObjectDetectorVersion.h ----- Header file with library version.
DummyObjectDetectorVersion.h.in --- CMake service file to generate version header.
DummyVFilter ------------------------- Folder with "dummy" video filter implementation.
CMakeLists.txt ------------------- CMake file for "dummy" video filter.
DummyVFilter.cpp ----------------- Source code file of the library.
DummyVFilter.h ------------------- Header file of the library.
DummyVFilterVersion.h ------------ Header file with library version.
DummyVFilterVersion.h.in --------- CMake service file to generate version header.
DummyVSource ------------------------- Folder with "dummy" video source implementation.
CMakeLists.txt ------------------- CMake file for "dummy" video source.
DummyVSource.cpp ----------------- Source code file of the library.
DummyVSource.h ------------------- Header file of the library.
DummyVSourceVersion.h ------------ Header file with library version.
DummyVSourceVersion.h.in --------- CMake service file to generate version header.
DummyVStabiliser --------------------- Folder with "dummy" video stabilizer implementation.
CMakeLists.txt ------------------- CMake file for "dummy" video stabilizer.
DummyVStabiliser.cpp ------------- Source code file of the library.
DummyVStabiliser.h --------------- Header file of the library.
DummyVStabiliserVersion.h -------- Header file with library version.
DummyVStabiliserVersion.h.in ----- CMake service file to generate version header.
DummyVTracker ------------------------ Folder with "dummy" video tracker implementation.
CMakeLists.txt ------------------- CMake file for "dummy" video tracker library.
DummyVTracker.cpp ---------------- Source code file of the library.
DummyVTracker.h ------------------ Header file of the library.
DummyVTrackerVersion.h ----------- Header file with library version.
DummyVTrackerVersion.h.in -------- CMake service file to generate version header.
src -------------------------------------- Folder with source code of the library.
CMakeLists.txt ----------------------- CMake file of the library.
VPipeline.cpp ------------------------ C++ implementation file.
VPipeline.h -------------------------- Header file with VPipeline class declaration.
VPipelineVersion.h ------------------- Header file which includes version of the library.
VPipelineVersion.h.in ---------------- CMake service file to generate version header.
tests ------------------------------------ Folder with test applications.
GuiTest ------------------------------ Folder with test application with GUI.
CMakeLists.txt ------------------- CMake file of the test application.
main.cpp ------------------------- Source code file of test application.
ProtocolTest ------------------------- Folder with protocol test application.
CMakeLists.txt ------------------- CMake file of the test application.
main.cpp ------------------------- Source code file of test application.
Video tracker centering mode
Video tracking centering mode can be applied to image if video tracker mode in TRACKING mode (tracking object). It is designed to shift image to keep tracking object in the center of screen. For some cases it is more comfortable for user to see tracking object exactly in the center of the screen. Video tracking centering mode also provide very stable image despite of camera movement and vibrations which is necessary for some video filters (for example, motion magnification) and for some object detectors (for example, motion detector). Bellow is illustration of video tracker centering mode:
On the image above you can see original video frame with tracking rectangle and image after video tracker centering (image offset). After centering the tracking rectangle will be in the center of the screen. Blue part of video frame illustrates the image can have part of original image due to principle of image shifting. The VPipeline library has parameters enable/disable and max horizontal and vertical image offset (see GeneralParam enum). If max image offset >0 and tracking rectangle offset > limits the library will move image according to limits only. It is necessary to prevent unexpected big image shift. Maximum offset can be not more than half of image.
Pan and tilt unit control notes
In pan-tilt cameras and gimbals to track object (moving camera to keep object in the center of screen) it is necessary to calculate position of object relative to center of screen (we assume that center of screen coincides with optical center of the system). VPipeline library perform video stabilization and after video tracking on stabilized image. So, the user get from pipeline stabilized image and all coordinates related to stabilized image. To calculate control signal for pan-tilt unit the user need to have coordinates on not stabilized image (relatively to the optical center). Using coordinates as is can make system unstable especially when object located near the center of image. To recalculate position of tracking rectangle to the not stabilized image use should use transformation parameters from video stabilizer and image offsets from video tracker centering (if enabled). The code to recalculate tracking rectangle coordinates to not stabilized image:
// Pipeline object initialized by the initVPipeline(...) method.
rapidpixel::VPipeline pipeline;
// Get params from pipeline.
rapidpixel::VPipelineParams params;
pipeline.getParams(params);
// Get tracking rectangle position.
float xResult = static_cast<float>(params.videoTracker.rectX);
float yResult = static_cast<float>(params.videoTracker.rectY);
// Get video stabilizer transformation params.
float dx = static_cast<float>(params.videoStabiliser.instantXOffset);
float dy = static_cast<float>(params.videoStabiliser.instantYOffset);
float da = params.videoStabiliser.instantAOffset;
// Calculate source position.
int xSource = static_cast<int>((xResult - dx) * std::cos(-da) - (yResult - dy) * std::sin(-da));
int ySource = static_cast<int>((xResult - dx) * std::sin(-da) + (yResult - dy) * std::cos(-da));
// Add video tracker centering offsets.
xSource -= params.general.trackerCenteringOffsetX;
ySource -= params.general.trackerCenteringOffsetY;
std::cout << "Position on not stabilized image: " << xSource << ", " << ySource << std::endl;
VPipeline class description
VPipeline class declaration
VPipeline.h file contains VPipeline class declaration. Class declaration:
class VPipeline
{
public:
/// Class constructor.
VPipeline();
/// Class destructor.
~VPipeline();
/// Copy and move are deleted: the class owns the processing threads.
VPipeline(const VPipeline&) = delete;
VPipeline(VPipeline&&) = delete;
VPipeline& operator=(const VPipeline&) = delete;
VPipeline& operator=(VPipeline&&) = delete;
/// Get string of current class version.
static std::string getVersion();
/// Init video processing pipeline.
bool initVPipeline(
VPipelineParams& params,
rapidpixel::VSource* videoSource = nullptr,
rapidpixel::Lens* lens = nullptr,
rapidpixel::Camera* camera = nullptr,
rapidpixel::VStabiliser* videoStabiliser = nullptr,
rapidpixel::VTracker* videoTracker = nullptr,
rapidpixel::ObjectDetector* objectDetector0 = nullptr,
rapidpixel::ObjectDetector* objectDetector1 = nullptr,
rapidpixel::ObjectDetector* objectDetector2 = nullptr,
rapidpixel::ObjectDetector* objectDetector3 = nullptr,
rapidpixel::ObjectDetector* objectDetector4 = nullptr,
rapidpixel::ObjectDetector* objectDetector5 = nullptr,
rapidpixel::ObjectDetector* objectDetector6 = nullptr,
rapidpixel::VFilter* videoFilter0 = nullptr,
rapidpixel::VFilter* videoFilter1 = nullptr,
rapidpixel::VFilter* videoFilter2 = nullptr,
rapidpixel::VFilter* videoFilter3 = nullptr,
rapidpixel::VFilter* videoFilter4 = nullptr,
rapidpixel::VFilter* videoFilter5 = nullptr,
rapidpixel::VFilter* videoFilter6 = nullptr);
/// Get video processing pipeline params.
void getParams(VPipelineParams& params);
/// Set module parameter.
bool setParam(VPipelineModule module, int id, float value);
/// Execute command.
bool executeCommand(VPipelineModule module,
int id,
float arg1 = -1.0f,
float arg2 = -1.0f,
float arg3 = -1.0f);
/// Get source video frame after processing.
bool getFrame(rapidpixel::Frame& frame, int timeoutMsec = 0);
/// Set detection mask to module.
bool setMask(VPipelineModule module, rapidpixel::Frame mask);
/// Encode set param command.
static void encodeSetParamCommand(uint8_t* data,
int& size,
VPipelineModule module,
int id,
float value);
/// Encode action command.
static void encodeCommand(uint8_t* data,
int& size,
VPipelineModule module,
int id,
float arg1 = 0.0f,
float arg2 = 0.0f,
float arg3 = 0.0f);
/// Decode command.
static int decodeCommand(uint8_t* data,
int size,
VPipelineModule& module,
int& id,
float& value1,
float& value2,
float& value3);
/// Decode and execute command.
bool decodeAndExecuteCommand(uint8_t* data, int size);
};
getVersion method
The getVersion() static method returns string of current version of VPipeline class. Method declaration:
static std::string getVersion();
Method can be used without VPipeline class instance. Example:
std::cout << "VPipeline class version: " << rapidpixel::VPipeline::getVersion() << std::endl;
Console output:
VPipeline class version: 4.0.5
initVPipeline method
The initVPipeline(…) method initializes video processing pipeline. This method initializes all functional modules by parameters given by user and starts internal processing threads. This method must be called first before any other methods (excepts static methods to encode / decode commands and params). User should put pointers to objects for particular modules (camera controller, lens controller, video source, video stabilizer, video tracker, object detectors and video filters). If user doesn’t provide pointer to particular module the pipeline will initialize “dummy” module which does nothing. After initialization method will run internal video processing threads. Method declaration:
bool initVPipeline(
VPipelineParams& params,
rapidpixel::VSource* videoSource = nullptr,
rapidpixel::Lens* lens = nullptr,
rapidpixel::Camera* camera = nullptr,
rapidpixel::VStabiliser* videoStabiliser = nullptr,
rapidpixel::VTracker* videoTracker = nullptr,
rapidpixel::ObjectDetector* objectDetector0 = nullptr,
rapidpixel::ObjectDetector* objectDetector1 = nullptr,
rapidpixel::ObjectDetector* objectDetector2 = nullptr,
rapidpixel::ObjectDetector* objectDetector3 = nullptr,
rapidpixel::ObjectDetector* objectDetector4 = nullptr,
rapidpixel::ObjectDetector* objectDetector5 = nullptr,
rapidpixel::ObjectDetector* objectDetector6 = nullptr,
rapidpixel::VFilter* videoFilter0 = nullptr,
rapidpixel::VFilter* videoFilter1 = nullptr,
rapidpixel::VFilter* videoFilter2 = nullptr,
rapidpixel::VFilter* videoFilter3 = nullptr,
rapidpixel::VFilter* videoFilter4 = nullptr,
rapidpixel::VFilter* videoFilter5 = nullptr,
rapidpixel::VFilter* videoFilter6 = nullptr);
| Parameter | Description |
|---|---|
| params | VPipelineParams class object which contains all pipeline parameters including parameters for functional modules. |
| videoSource | Pointer to VSource object. The pipeline will use this object to capture video from video source. User should provide particular implementation of VSource interface. If user doesn’t provide pointer to video source the pipeline will create “dummy” video source module which generates artificial video frames itself with default parameters (resolution and FPS). If user provides pointer to video source implementation and if initialization failed the pipeline will initialize “dummy” video source. |
| lens | Pointer to Lens object. The pipeline will use this object to control lens. User should provide particular implementation of Lens interface. If user doesn’t provide pointer to lens controller the pipeline will create “dummy” lens controller. If particular controller has Lens and Camera interfaces (for block cameras which include image sensor and lens) user should put the same pointer separately for lens and camera interfaces. |
| camera | Pointer to Camera object. The pipeline will use this object to control camera. User should provide particular implementation of Camera interface. If user doesn’t provide pointer to camera controller the pipeline will create “dummy” camera controller. If particular controller has Lens and Camera interfaces (for block cameras which include image sensor and lens) user should put the same pointer separately for lens and camera interfaces. |
| videoStabiliser | Pointer to VStabiliser object. The pipeline will use this object to stabilize video frames. User should provide particular implementation of VStabiliser interface. If user doesn’t provide pointer to video stabilizer the pipeline will create “dummy” video stabilizer which passes video frames without changes. |
| videoTracker | Pointer to VTracker object. The pipeline will use this object for video tracking of single object. User should provide particular implementation of VTracker interface. If user doesn’t provide pointer to video tracker the pipeline will create “dummy” video tracker. |
| objectDetector0 | Pointer to ObjectDetector 0 object. The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes seven slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector. |
| objectDetector1 | Pointer to ObjectDetector 1 object. The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes seven slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector. |
| objectDetector2 | Pointer to ObjectDetector 2 object. The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes seven slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector. |
| objectDetector3 | Pointer to ObjectDetector 3 object. The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes seven slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector. |
| objectDetector4 | Pointer to ObjectDetector 4 object. The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes seven slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector. |
| objectDetector5 | Pointer to ObjectDetector 5 object. The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes seven slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector. |
| objectDetector6 | Pointer to ObjectDetector 6 object. The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes seven slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector. |
| videoFilter0 | Pointer to VFilter 0 object. User can assign any video filter compatible with VFilter interface. Usually first video filter is image flip function. VPipeline includes seven slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter. |
| videoFilter1 | Pointer to VFilter 1 object. User can assign any video filter compatible with VFilter interface. Usually second video filter is digital zoom function. VPipeline includes seven slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter. |
| videoFilter2 | Pointer to VFilter 2 object. User can assign any video filter compatible with VFilter interface. Usually third video filter is noise removal function (denoise). VPipeline includes seven slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter. |
| videoFilter3 | Pointer to VFilter 3 object. User can assign any video filter compatible with VFilter interface. Usually fourth video filter is defog / dehaze function. VPipeline includes seven slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter. |
| videoFilter4 | Pointer to VFilter 4 object. User can assign any video filter compatible with VFilter interface. Usually fifth video filter is motion magnification function. VPipeline includes seven slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter. |
| videoFilter5 | Pointer to VFilter 5 object. User can assign any video filter compatible with VFilter interface. Usually sixth video filter is sharpening function. VPipeline includes seven slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter. |
| videoFilter6 | Pointer to VFilter 6 object. User can assign any video filter compatible with VFilter interface. Usually seventh video filter is custom processing function. VPipeline includes seven slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter. |
Returns: TRUE if pipeline initialized (or already initialized) or FALSE if not. The method will return FALSE only if video source initialization failed or the processing threads can’t be started. The pipeline firstly will try initialize video source provided by user (if provided) and in case any errors will try initialize “dummy” video source.
getParams method
The getParams(…) method returns all video processing pipeline parameters including detection and video tracking results. The method will obtain current params from each module (video source, video stabilizer, video tracker, object detector and video filters). Method declaration:
void getParams(VPipelineParams& params);
| Parameter | Description |
|---|---|
| params | VPipelineParams class which contains all pipeline parameters including parameters for functional modules, detection and tracking results. |
setParam method
The setParam(…) changes parameter of particular video processing pipeline module. User must specify functional module of pipeline to set parameter value. The method will check module type and will call setParam(…) method of appropriate module (functional module object). Depends on particular implementation of functional module new parameters value can be accepted but not set. To control current parameters values use getParams(…) method. Method declaration:
bool setParam(VPipelineModule module, int id, float value);
| Parameter | Description |
|---|---|
| module | Functional module of pipeline according to VPipelineModule enum. Changing parameters available for all functional modules. |
| id | Parameter ID. Parameter ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Changing parameters available for all functional modules: VSourceParam - Video source parameters enum. Described in VSource interface class. LensParam - Lens parameters enum. Described in Lens interface class. CameraParam - Camera parameters enum. Described in Camera interface class. VTrackerParam - Video tracker parameters enum. Described in VTracker interface class. VStabiliserParam - Video stabilizer parameters enum. Described in VStabiliser interface class. VFilterParam - Video filter parameters enum. Described in VFilter interface class. GeneralParam - General pipeline parameters enum. Described in VPipeline.h file. ObjectDetectorParam - Object detector parameters enum. Described in ObjectDetector interface class. |
| value | Parameter value. Depends on parameter ID and functional module. To check valid parameters values see description of appropriate enum for particular implementation. |
Returns: TRUE if parameter was accepted or FALSE if not (the pipeline is not initialized, unknown module, the module did not accept the parameter or the value of a general parameter is not valid). The values of the general parameters LOG_MODE (0..3), TRACKER_CENTERING_MAX_OFFSET_X and TRACKER_CENTERING_MAX_OFFSET_Y must be finite and must fit in int, otherwise the method returns FALSE and the parameter is not changed. If the pipeline is not initialized the method prints an error message to the console.
Example how to set parameters for different video processing pipeline modules:
// Pipeline object initialized by the initVPipeline(...) method.
rapidpixel::VPipeline pipeline;
bool ok = true;
// Set video source parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_SOURCE, static_cast<int>(rapidpixel::VSourceParam::FOCUS_MODE), 0);
// Set lens parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::LENS, static_cast<int>(rapidpixel::LensParam::FOCUS_POS), 1000);
// Set camera parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::CAMERA, static_cast<int>(rapidpixel::CameraParam::BRIGHTNESS), 100);
// Set video tracker parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_TRACKER, static_cast<int>(rapidpixel::VTrackerParam::NUM_CHANNELS), 3);
// Set video stabilizer parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_STABILISER, static_cast<int>(rapidpixel::VStabiliserParam::TYPE), 1);
// Set video filter parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_FILTER_1, static_cast<int>(rapidpixel::VFilterParam::LEVEL), 20);
// Set video filter parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_FILTER_2, static_cast<int>(rapidpixel::VFilterParam::LEVEL), 10);
// Set video filter parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_FILTER_3, static_cast<int>(rapidpixel::VFilterParam::TYPE), 1);
// Set video filter parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_FILTER_4, static_cast<int>(rapidpixel::VFilterParam::LEVEL), 50);
// Set video filter parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::VIDEO_FILTER_5, static_cast<int>(rapidpixel::VFilterParam::CUSTOM_1), 1.5f);
// Set object detector parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::OBJECT_DETECTOR_1, static_cast<int>(rapidpixel::ObjectDetectorParam::SCALE_FACTOR), 2);
// Set object detector parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::OBJECT_DETECTOR_2, static_cast<int>(rapidpixel::ObjectDetectorParam::MIN_X_SPEED), 1.5f);
// Set object detector parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::OBJECT_DETECTOR_3, static_cast<int>(rapidpixel::ObjectDetectorParam::MIN_Y_SPEED), 2.5f);
// Set object detector parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::OBJECT_DETECTOR_4, static_cast<int>(rapidpixel::ObjectDetectorParam::MIN_Y_SPEED), 1.5f);
// Set general pipeline parameter.
ok &= pipeline.setParam(rapidpixel::VPipelineModule::GENERAL, static_cast<int>(rapidpixel::GeneralParam::LOG_MODE), 1);
if (!ok)
{
std::cerr << "ERROR: not all calls were accepted" << std::endl;
}
executeCommand method
The executeCommand(…) method performs action command for particular video processing pipeline module. User must specify functional module of pipeline to execute command. The method will check module type and will call executeCommand(…) method of appropriate module (functional module object). Method declaration:
bool executeCommand(VPipelineModule module,
int id,
float arg1 = -1.0f,
float arg2 = -1.0f,
float arg3 = -1.0f);
| Parameter | Description |
|---|---|
| module | Functional module of pipeline according to VPipelineModule enum. Commands available for all functional modules. |
| id | Action command ID. Command ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Commands available for all modules: VSourceCommand - Video source action commands enum. Described in VSource interface class. LensCommand - Lens action commands enum. Described in Lens interface class. CameraCommand - Camera action commands enum. Described in Camera interface class. VTrackerCommand - Video tracker action commands enum. Described in VTracker interface class. VStabiliserCommand - Video stabilizer action commands enum. Described in VStabiliser interface class. ObjectDetectorCommand - Object detector commands enum. Described in ObjectDetector interface class. VFilterCommand - Video filter action commands enum. Described in VFilter interface class. GeneralCommand - General pipeline action commands enum. Described in VPipeline.h file. |
| arg1 | First command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum for particular implementation. |
| arg2 | Second command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum for particular implementation. |
| arg3 | Third command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum for particular implementation. |
Returns: TRUE if the command executed (accepted by pipeline) or FALSE if not (the pipeline is not initialized, unknown module or ID, or the module did not accept the command). If the pipeline is not initialized the method prints an error message to the console.
Example how to execute command for different video processing pipeline modules:
// Pipeline object initialized by the initVPipeline(...) method.
rapidpixel::VPipeline pipeline;
bool ok = true;
// Execute video source command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_SOURCE, static_cast<int>(rapidpixel::VSourceCommand::RESTART));
// Execute lens command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::LENS, static_cast<int>(rapidpixel::LensCommand::ZOOM_TO_POS), 1000);
// Execute camera command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::CAMERA, static_cast<int>(rapidpixel::CameraCommand::FREEZE));
// Execute video tracker command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_TRACKER, static_cast<int>(rapidpixel::VTrackerCommand::CAPTURE), 100, 50, -1);
// Execute video stabilizer command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_STABILISER, static_cast<int>(rapidpixel::VStabiliserCommand::RESET));
// Execute video filter command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_FILTER_1, static_cast<int>(rapidpixel::VFilterCommand::ON));
// Execute video filter command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_FILTER_2, static_cast<int>(rapidpixel::VFilterCommand::OFF));
// Execute video filter command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_FILTER_3, static_cast<int>(rapidpixel::VFilterCommand::RESET));
// Execute video filter command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_FILTER_4, static_cast<int>(rapidpixel::VFilterCommand::ON));
// Execute video filter command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::VIDEO_FILTER_5, static_cast<int>(rapidpixel::VFilterCommand::ON));
// Execute object detector command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::OBJECT_DETECTOR_1, static_cast<int>(rapidpixel::ObjectDetectorCommand::ON));
// Execute object detector command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::OBJECT_DETECTOR_2, static_cast<int>(rapidpixel::ObjectDetectorCommand::OFF));
// Execute object detector command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::OBJECT_DETECTOR_3, static_cast<int>(rapidpixel::ObjectDetectorCommand::RESET));
// Execute object detector command.
ok &= pipeline.executeCommand(rapidpixel::VPipelineModule::OBJECT_DETECTOR_4, static_cast<int>(rapidpixel::ObjectDetectorCommand::RESET));
if (!ok)
{
std::cerr << "ERROR: not all calls were accepted" << std::endl;
}
getFrame method
The getFrame(…) method returns the current video frame captured from the video source after its processing by video filter 6. The pipeline can be enabled / disabled. If pipeline disabled the video source still works and the getFrame(…) method will return frame after video source module without processing by any functional module. Method declaration:
bool getFrame(rapidpixel::Frame& frame, int timeoutMsec = 0);
| Parameter | Description |
|---|---|
| frame | Reference to Frame object. The method will return the video frame if YUV24 format (YUV 24 bits). |
| timeoutMsec | Timeout to wait new frame, milliseconds. Values: -1 - Waiting endlessly for a new video frame to arrive. 0 - Just check if there is new video frame to return. If not, the method will return FALSE immediately. -2 (or any other negative value except -1) - just get last frame without checking new frame availability (can cause frame duplication). >0 - Wait for a new video frame a certain timeout. If there is no new frame for the specified timeout the method will return FALSE. |
Returns: TRUE if a new frame is returned or FALSE if not (the pipeline is not initialized, there is no new frame or no frame has been produced yet). The frame object is not changed if FALSE is returned. If the pipeline is not initialized the method prints an error message to the console.
setMask method
The setMask(…) method sets detection or filtering mask to object detectors or video filters in pipeline. The method will check module type and will call setMask(…) method of appropriate module (ObjectDetector or VFilter object). To set mask the particular object detector or video filter must be initialized in initVPipeline(…) method and must support this capability. If user didn’t set particular object detector or video filter in initVPipeline(…) method the pipeline will initialize “dummy” module which doesn’t support mask (method setMask(…) will return FALSE). The method accepts mask in form of video frame which allows user to set mask of any configuration (not only rectangular). Method declaration:
bool setMask(VPipelineModule module, rapidpixel::Frame mask);
| Parameter | Description |
|---|---|
| module | Functional module of pipeline according to VPipelineModule enum. This function available only for modules: VPipelineModule::OBJECT_DETECTOR_0 VPipelineModule::OBJECT_DETECTOR_1 VPipelineModule::OBJECT_DETECTOR_2 VPipelineModule::OBJECT_DETECTOR_3 VPipelineModule::OBJECT_DETECTOR_4 VPipelineModule::OBJECT_DETECTOR_5 VPipelineModule::OBJECT_DETECTOR_6 VPipelineModule::VIDEO_FILTER_0 VPipelineModule::VIDEO_FILTER_1 VPipelineModule::VIDEO_FILTER_2 VPipelineModule::VIDEO_FILTER_3 VPipelineModule::VIDEO_FILTER_4 VPipelineModule::VIDEO_FILTER_5 VPipelineModule::VIDEO_FILTER_6 |
| mask | Reference to Frame object which represents detection mask. Detection mask is an image with possible pixel formats: GRAY, NV12, NV21, YU12 and YV12 (Fourcc enum described in Frame library). Recommended GRAY pixel format (most common for all modules). The image can have any resolution. If detection mask resolution is different than input video frames for particular module (object detector or video filter) the particular module will rescale to original input image resolution for processing. If particular object detector or video filter doesn’t support mask the method will return FALSE. |
Returns: TRUE if the detection or filtering mask is set (accepted by pipeline) or FALSE if not (the pipeline is not initialized, the module does not support masks or the module did not accept the mask).
Example how to prepare rectangular mask and set it to different modules:
// Pipeline object initialized by the initVPipeline(...) method.
rapidpixel::VPipeline pipeline;
bool ok = true;
// Create mask with any resolution. It will be rescaled in pipeline. The
// pixels of the rectangle are 255 (processed), other pixels are 0 (omitted).
constexpr int MASK_SIZE = 256;
rapidpixel::Frame mask(MASK_SIZE, MASK_SIZE, rapidpixel::Fourcc::GRAY);
for (int y = 50; y < 100; ++y)
{
for (int x = 50; x < 100; ++x)
{
mask.data[y * MASK_SIZE + x] = 255;
}
}
// Set mask to all modules which support it.
ok &= pipeline.setMask(rapidpixel::VPipelineModule::OBJECT_DETECTOR_0, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::OBJECT_DETECTOR_1, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::OBJECT_DETECTOR_2, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::OBJECT_DETECTOR_3, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::OBJECT_DETECTOR_4, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::OBJECT_DETECTOR_5, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::OBJECT_DETECTOR_6, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::VIDEO_FILTER_0, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::VIDEO_FILTER_1, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::VIDEO_FILTER_2, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::VIDEO_FILTER_3, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::VIDEO_FILTER_4, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::VIDEO_FILTER_5, mask);
ok &= pipeline.setMask(rapidpixel::VPipelineModule::VIDEO_FILTER_6, mask);
if (!ok)
{
std::cerr << "ERROR: not all calls were accepted" << std::endl;
}
encodeSetParamCommand static method
The encodeSetParamCommand(…) static method encodes command to change any parameters of remote video processing pipeline. To control a pipeline remotely, the developer has to develop his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VPipeline class contains static methods for encoding the control commands. The VPipeline class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeSetParamCommand(…) designed to encode SET_PARAM command. To encode command user doesn’t need initialize pipeline. Method declaration:
static void encodeSetParamCommand(uint8_t* data,
int& size,
VPipelineModule module,
int id,
float value);
| Parameter | Description |
|---|---|
| data | Pointer to data buffer for encoded command. Must have size >= 32 bytes. If the pointer is nullptr the method does nothing and sets size to 0. |
| size | Size of encoded data (command): 15 bytes for the general module and 18 bytes for other modules. 0 if the command is not encoded (data is nullptr or unknown module). |
| module | Functional module of pipeline according to VPipelineModule enum. Commands available for all functional modules. |
| id | Parameter ID. Parameter ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Encoding SET_PARAMS command available for all functional modules: VSourceParam - Video source parameters enum. Described in VSource interface class. LensParam - Lens parameters enum. Described in Lens interface class. CameraParam - Camera parameters enum. Described in Camera interface class. VTrackerParam - Video tracker parameters enum. Described in VTracker interface class. VStabiliserParam - Video stabilizer parameters enum. Described in VStabiliser interface class. VFilterParam - Video filter parameters enum. Described in VFilter interface class. GeneralParam - General pipeline parameters enum. Described in VPipeline.h file. ObjectDetectorParam - Object detector parameters enum. Described in ObjectDetector interface class. |
| value | Parameter value. Depends on parameter ID and functional module. To check valid parameters values user should check description of appropriate enum. |
Example how to encode SET_PARAM command for different video processing pipeline modules:
uint8_t buffer[32];
int size = 0;
// Encode video source parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_SOURCE, static_cast<int>(rapidpixel::VSourceParam::FOCUS_MODE), 0);
// Encode lens parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::LENS, static_cast<int>(rapidpixel::LensParam::FOCUS_POS), 1000);
// Encode camera parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::CAMERA, static_cast<int>(rapidpixel::CameraParam::BRIGHTNESS), 100);
// Encode video tracker parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_TRACKER, static_cast<int>(rapidpixel::VTrackerParam::NUM_CHANNELS), 3);
// Encode video stabilizer parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_STABILISER, static_cast<int>(rapidpixel::VStabiliserParam::TYPE), 1);
// Encode video filter parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_1, static_cast<int>(rapidpixel::VFilterParam::LEVEL), 1.2f);
// Encode video filter parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_2, static_cast<int>(rapidpixel::VFilterParam::LEVEL), 10.5f);
// Encode video filter parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_3, static_cast<int>(rapidpixel::VFilterParam::TYPE), 2);
// Encode video filter parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_4, static_cast<int>(rapidpixel::VFilterParam::LEVEL), 21.6f);
// Encode video filter parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_5, static_cast<int>(rapidpixel::VFilterParam::TYPE), 0);
// Encode object detector parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_1, static_cast<int>(rapidpixel::ObjectDetectorParam::SCALE_FACTOR), 2);
// Encode object detector parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_2, static_cast<int>(rapidpixel::ObjectDetectorParam::MIN_X_SPEED), 1.5f);
// Encode object detector parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_3, static_cast<int>(rapidpixel::ObjectDetectorParam::MIN_Y_SPEED), 2.5f);
// Encode object detector parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_2, static_cast<int>(rapidpixel::ObjectDetectorParam::MIN_X_SPEED), 2.5f);
// Encode general pipeline parameter.
rapidpixel::VPipeline::encodeSetParamCommand(buffer, size, rapidpixel::VPipelineModule::GENERAL, static_cast<int>(rapidpixel::GeneralParam::LOG_MODE), 1);
encodeCommand static method
The encodeCommand(…) static method encodes action command for remote video processing pipeline. To control a pipeline remotely, the developer has to develop his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VPipeline class contains static methods for encoding the control commands. The VPipeline class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeCommand(…) designed to encode COMMAND command (action command). To encode command user doesn’t need initialize pipeline. Method declaration:
static void encodeCommand(uint8_t* data,
int& size,
VPipelineModule module,
int id,
float arg1 = 0.0f,
float arg2 = 0.0f,
float arg3 = 0.0f);
| Parameter | Description |
|---|---|
| data | Pointer to data buffer for encoded command. Must have size >= 32 bytes. If the pointer is nullptr the method does nothing and sets size to 0. |
| size | Size of encoded data (command): 11 bytes for the general module and from 14 to 26 bytes for other modules. 0 if the command is not encoded (data is nullptr or unknown module). |
| module | Functional module of pipeline according to VPipelineModule enum. Commands available for all functional modules. |
| id | Action command ID. Command ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Encoding COMMAND available for all modules: VSourceCommand - Video source action commands enum. Described in VSource interface class. LensCommand - Lens action commands enum. Described in Lens interface class. CameraCommand - Camera action commands enum. Described in Camera interface class. VTrackerCommand - Video tracker action commands enum. Described in VTracker interface class. VStabiliserCommand - Video stabilizer action commands enum. Described in VStabiliser interface class. ObjectDetectorCommand - Object detector commands enum. Described in ObjectDetector interface class. VFilterCommand - Video filter action commands enum. Described in VFilter interface class. GeneralCommand - General pipeline action commands enum. Described in VPipeline.h file. |
| arg1 | First command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum of particular implementation. |
| arg2 | Second command argument. Available only for video tracker module. Value depends on command ID. To check valid arguments values user should check description of appropriate enum of particular video tracker implementation. |
| arg3 | Third command argument. Available only for video tracker module. Value depends on command ID. To check valid arguments values user should check description of appropriate enum of particular video tracker implementation. |
Example how to encode COMMAND command for different video processing pipeline modules:
uint8_t buffer[32];
int size = 0;
// Encode video source action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_SOURCE, static_cast<int>(rapidpixel::VSourceCommand::RESTART));
// Encode lens action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::LENS, static_cast<int>(rapidpixel::LensCommand::ZOOM_TO_POS), 1000);
// Encode camera action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::CAMERA, static_cast<int>(rapidpixel::CameraCommand::FREEZE));
// Encode video tracker action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_TRACKER, static_cast<int>(rapidpixel::VTrackerCommand::CAPTURE), 100, 50, -1);
// Encode video stabilizer action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_STABILISER, static_cast<int>(rapidpixel::VStabiliserCommand::RESET));
// Encode video filter action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_1, static_cast<int>(rapidpixel::VFilterCommand::ON));
// Encode video filter action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_2, static_cast<int>(rapidpixel::VFilterCommand::OFF));
// Encode video filter action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_3, static_cast<int>(rapidpixel::VFilterCommand::RESET));
// Encode video filter action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_4, static_cast<int>(rapidpixel::VFilterCommand::RESET));
// Encode video filter action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::VIDEO_FILTER_5, static_cast<int>(rapidpixel::VFilterCommand::ON));
// Encode object detector action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_1, static_cast<int>(rapidpixel::ObjectDetectorCommand::ON));
// Encode object detector action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_2, static_cast<int>(rapidpixel::ObjectDetectorCommand::OFF));
// Encode object detector action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_3, static_cast<int>(rapidpixel::ObjectDetectorCommand::RESET));
// Encode object detector action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::OBJECT_DETECTOR_4, static_cast<int>(rapidpixel::ObjectDetectorCommand::RESET));
// Encode general pipeline action command.
rapidpixel::VPipeline::encodeCommand(buffer, size, rapidpixel::VPipelineModule::GENERAL, static_cast<int>(rapidpixel::GeneralCommand::ON));
decodeCommand static method
The decodeCommand(…) static method decodes commands on pipeline side encoded by encodeCommand(…) and encodeSetParamCommand(…) methods. To control a pipeline remotely, the developer has to develop his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VPipeline class contains static methods for encoding the control commands. The VPipeline class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). The decodeCommand(…) method designed to decode both input command types on pipeline side (edge device). This method is usually used if you want to intercept specific control commands. If it is not necessary to intercept specific commands, it is more convenient to use decodeAndExecuteCommand(…) method. Method declaration:
static int decodeCommand(uint8_t* data,
int size,
VPipelineModule& module,
int& id,
float& value1,
float& value2,
float& value3);
| Parameter | Description |
|---|---|
| data | Pointer to input command. Must not be nullptr. |
| size | Size of input command, bytes. |
| module | The pipeline module for which the command is intended. |
| id | Action commands ID or parameter ID. It can be different command and parameters structure. The method converts it to int. |
| value1 | Decoded parameters value in case decoding SET_PARAM command or first command argument in case decoding action command (COMMAND). |
| value2 | Second command argument in case decoding action command (COMMAND). Will be available only for VPipelineModule::VIDEO_TRACKER. |
| value3 | Third command argument in case decoding action command (COMMAND). Will be available only for VPipelineModule::VIDEO_TRACKER. |
Returns: 0 - in case decoding COMMAND, 1 - in case decoding SET_PARAM command or -1 in case errors (data is nullptr, the size, the header, the version, the module ID or the ID of the general module are not valid, or the decoder of the module rejected the command). The outputs module, id, value1, value2 and value3 are valid only if 0 or 1 is returned: they are set to zero if -1 is returned.
Example how to decode command:
bool decodeAndExecute(rapidpixel::VPipeline& pipeline, uint8_t* data,
int size)
{
// Decode command.
rapidpixel::VPipelineModule module;
int id;
float value1, value2, value3;
const int res = rapidpixel::VPipeline::decodeCommand(
data, size, module, id, value1, value2, value3);
if (res == 0) // COMMAND
{
return pipeline.executeCommand(module, id, value1, value2, value3);
}
if (res == 1) // SET_PARAM
{
return pipeline.setParam(module, id, value1);
}
return false; // ERROR
}
decodeAndExecuteCommand method
The decodeAndExecuteCommand(…) method decodes commands encoded by encodeSetParamCommand(…) and encodeCommand(…) methods using decodeCommand(…) method and calls setParam(…) or executeCommand(…) methods. This method used to simplify remote pipeline control. User just need to put input command data to pipeline. Method declaration:
bool decodeAndExecuteCommand(uint8_t* data, int size);
| Parameter | Description |
|---|---|
| data | Pointer to input command. Must not be nullptr. |
| size | Size of input command, bytes. |
Returns: TRUE if command decoded and executed or FALSE if not.
Implementation of the method inside pipeline which explaing principle of the method:
bool VPipeline::decodeAndExecuteCommand(uint8_t* data, int size)
{
// Decode command.
VPipelineModule module = VPipelineModule::VIDEO_SOURCE;
int id = 0; float arg1 = -1.0f; float arg2 = -1.0f; float arg3 = -1.0f;
int ret = VPipeline::decodeCommand(data, size, module, id, arg1, arg2, arg3);
// Set param or execute command.
if (ret == 1)
return VPipeline::setParam(module, id, arg1);
else if (ret == 0)
return VPipeline::executeCommand(module, id, arg1, arg2, arg3);
return false;
}
Enums
This section lists the C++ enums declared in the VPipeline.h file. Other C++ enums used in methods of VPipeline class are declared in libraries: Camera, Lens, ObjectDetector, VSource, VTracker, VFilter and VStabiliser.
VPipelineModule enum
The VPipelineModule enum declared in VPipeline.h file and lists pipeline functional modules. Enum declaration:
enum class VPipelineModule
{
/// Video source.
VIDEO_SOURCE = 1,
/// Lens.
LENS,
/// Camera.
CAMERA,
/// Video tracker.
VIDEO_TRACKER,
/// Video stabilizer.
VIDEO_STABILISER,
/// Video filter 0.
VIDEO_FILTER_0,
/// Video filter 1.
VIDEO_FILTER_1,
/// Video filter 2.
VIDEO_FILTER_2,
/// Video filter 3.
VIDEO_FILTER_3,
/// Video filter 4.
VIDEO_FILTER_4,
/// Video filter 5.
VIDEO_FILTER_5,
/// Video filter 6.
VIDEO_FILTER_6,
/// Object detector 0.
OBJECT_DETECTOR_0,
/// Object detector 1.
OBJECT_DETECTOR_1,
/// Object detector 2.
OBJECT_DETECTOR_2,
/// Object detector 3.
OBJECT_DETECTOR_3,
/// Object detector 4.
OBJECT_DETECTOR_4,
/// Object detector 5.
OBJECT_DETECTOR_5,
/// Object detector 6.
OBJECT_DETECTOR_6,
/// General.
GENERAL
};
Table 2 - VPipelineModule enum values.
| Value | Description |
|---|---|
| VIDEO_SOURCE | Video source module represented by VSource interface class. |
| LENS | Lens controller module represented by Lens interface class. |
| CAMERA | Camera controller module represented by Camera interface class. |
| VIDEO_TRACKER | Video tracker module represented by VTracker interface class. |
| VIDEO_STABILISER | Video stabilizer module represented by VStabiliser interface class. |
| VIDEO_FILTER_0 | Video filter 0 module represented by VFilter interface class. |
| VIDEO_FILTER_1 | Video filter 1 module represented by VFilter interface class. |
| VIDEO_FILTER_2 | Video filter 2 module represented by VFilter interface class. |
| VIDEO_FILTER_3 | Video filter 3 module represented by VFilter interface class. |
| VIDEO_FILTER_4 | Video filter 4 module represented by VFilter interface class. |
| VIDEO_FILTER_5 | Video filter 5 module represented by VFilter interface class. |
| VIDEO_FILTER_6 | Video filter 6 module represented by VFilter interface class. |
| OBJECT_DETECTOR_0 | Object detector 0 module represented by ObjectDetector interface class. |
| OBJECT_DETECTOR_1 | Object detector 1 module represented by ObjectDetector interface class. |
| OBJECT_DETECTOR_2 | Object detector 2 module represented by ObjectDetector interface class. |
| OBJECT_DETECTOR_3 | Object detector 3 module represented by ObjectDetector interface class. |
| OBJECT_DETECTOR_4 | Object detector 4 module represented by ObjectDetector interface class. |
| OBJECT_DETECTOR_5 | Object detector 5 module represented by ObjectDetector interface class. |
| OBJECT_DETECTOR_6 | Object detector 6 module represented by ObjectDetector interface class. |
| GENERAL | General module (general pipeline params). |
GeneralParam enum
The GeneralParam enum declared in VPipeline.h file and lists general pipeline parameters IDs. Enum declaration:
enum class GeneralParam
{
/// Logging mode. Values: 0 - Disable, 1 - Only file,
/// 2 - Only terminal (console), 3 - File and terminal (console).
LOG_MODE = 1,
/// Mode: 0 - disable, 1 - enable.
MODE,
/// Tracker centering mode: 0 - disable, 1 - enable.
TRACKER_CENTERING_ENABLE_MODE,
/// Tracker centering max offset X. Values: 0 - half of the frame width,
/// >0 - offset in pixels (limited by half of the frame width).
TRACKER_CENTERING_MAX_OFFSET_X,
/// Tracker centering max offset Y. Values: 0 - half of the frame height,
/// >0 - offset in pixels (limited by half of the frame height).
TRACKER_CENTERING_MAX_OFFSET_Y
};
Table 3 - GeneralParam enum values.
| Value | Description |
|---|---|
| LOG_MODE | Logging mode: 0 - Disable (default value). 1 - Only file. 2 - Only terminal (console). 3 - File and terminal (console). The VPipeline library uses Logger library for printing logs into console and(or) file. Note: to enable printing logs in files for all modules user have to initialize it by using static method setSaveLogParams(…) of Logger library. This parameter used only for pipeline and doesn’t set logging mode for particular functional modules. To set logging mode for particular functional modules use appropriate parameters enums. |
| MODE | Video processing pipeline mode: 0 - disabled, 1 - enabled. If pipeline is disabled, it should not perform any processing. If the pipeline is switched on again, processing must be resumed. |
| TRACKER_CENTERING_ENABLE_MODE | Video tracking centering mode: 0 - disabled, 1 - enabled. If video tracking centering mode enabled and if video tracker in TRACKING mode (mode == 1) the pipeline will move image to keep tracking rectangle in the center of image (see Video tracker centering mode). |
| TRACKER_CENTERING_MAX_OFFSET_X | Maximum image horizontal offset (pixels) for video tracking centering mode. If video tracking centering mode enabled and if video tracker in TRACKING mode (mode == 1) the pipeline will move image to keep tracking rectangle in the center of image (see Video tracker centering mode) but offset can’t be more than maximum offset (the image will be moved according to maximum allowed offset). |
| TRACKER_CENTERING_MAX_OFFSET_Y | Maximum image vertical offset (pixels) for video tracking centering mode. If video tracking centering mode enabled and if video tracker in TRACKING mode (mode == 1) the pipeline will move image to keep tracking rectangle in the center of image (see Video tracker centering mode) but offset can’t be more than maximum offset (the image will be moved according to maximum allowed offset). |
GeneralCommand enum
The GeneralCommand enum declared in VPipeline.h file and lists general pipeline action commands IDs. Enum declaration:
enum class GeneralCommand
{
/// Enable video processing pipeline.
ON = 1,
/// Disable video processing pipeline.
OFF
};
Table 4 - GeneralCommand enum values.
| Value | Description |
|---|---|
| ON | Enable pipeline. If the pipeline is already enabled this command will do nothing. If the pipeline is disabled this command will start video processing. Even if the pipeline is disabled use can get video from video source without processing by any functional modules of the pipeline. |
| OFF | Disable pipeline. If the pipeline is already disabled this command will do nothing. If the pipeline is enabled this command will exclude all functional modules from video processing excepts video source. Even if the pipeline is disabled use can get video from video source without processing by any functional modules of the pipeline. |
VPipelineParams class description
VPipelineParams class used for pipeline initialization (initVPipeline(…) method) or to get all actual params (getParams(…) method) including detection and video tracking results. Also VPipelineParams provide structure to write/read params from JSON files (JSON_READABLE macro) and provide methods to encode (serialize) and decode (deserialize) params.
VPipelineParams class declaration
The VPipeline.h file contains VPipelineParams class declaration. Class declaration:
class VPipelineParams
{
public:
/// General params.
class GeneralParams
{
public:
/// Logging mode. Values: 0 - Disable, 1 - Only file,
/// 2 - Only terminal (console), 3 - File and terminal (console).
int logMode{0};
/// Main cycle processing time.
int mainProcessingTimeUs{0};
/// Enable.
bool enable{true};
/// Tracker centering enable.
bool trackerCenteringEnable{ false };
/// Tracker centering current offset X.
int trackerCenteringOffsetX{ 0 };
/// Tracker centering current offset Y.
int trackerCenteringOffsetY{ 0 };
/// Tracker centering max offset X.
int trackerCenteringMaxOffsetX{ 0 };
/// Tracker centering max offset Y.
int trackerCenteringMaxOffsetY{ 0 };
JSON_READABLE(GeneralParams, logMode, enable, trackerCenteringEnable,
trackerCenteringMaxOffsetX, trackerCenteringMaxOffsetY)
};
/// Video source params.
rapidpixel::VSourceParams videoSource;
/// Video stabilizer params.
rapidpixel::VStabiliserParams videoStabiliser;
/// Video tracker params.
rapidpixel::VTrackerParams videoTracker;
/// Object detector 0 params.
rapidpixel::ObjectDetectorParams objectDetector0;
/// Object detector 1 params.
rapidpixel::ObjectDetectorParams objectDetector1;
/// Object detector 2 params.
rapidpixel::ObjectDetectorParams objectDetector2;
/// Object detector 3 params.
rapidpixel::ObjectDetectorParams objectDetector3;
/// Object detector 4 params.
rapidpixel::ObjectDetectorParams objectDetector4;
/// Object detector 5 params.
rapidpixel::ObjectDetectorParams objectDetector5;
/// Object detector 6 params.
rapidpixel::ObjectDetectorParams objectDetector6;
/// Lens params.
rapidpixel::LensParams lens;
/// Camera params.
rapidpixel::CameraParams camera;
/// Video filter 0 params.
rapidpixel::VFilterParams videoFilter0;
/// Video filter 1 params.
rapidpixel::VFilterParams videoFilter1;
/// Video filter 2 params.
rapidpixel::VFilterParams videoFilter2;
/// Video filter 3 params.
rapidpixel::VFilterParams videoFilter3;
/// Video filter 4 params.
rapidpixel::VFilterParams videoFilter4;
/// Video filter 5 params.
rapidpixel::VFilterParams videoFilter5;
/// Video filter 6 params.
rapidpixel::VFilterParams videoFilter6;
/// General params.
GeneralParams general;
JSON_READABLE(VPipelineParams, videoSource, videoStabiliser, videoTracker,
objectDetector0, objectDetector1, objectDetector2,
objectDetector3, objectDetector4, objectDetector5,
objectDetector6, lens, camera, videoFilter0, videoFilter1,
videoFilter2, videoFilter3, videoFilter4, videoFilter5,
videoFilter6, general)
/**
* @brief Encode params (serialize).
* @param data Pointer to data buffer.
* @param bufferSize Data buffer size.
* @param size Size of data.
* @param mask Pointer to parameters mask.
* @return TRUE if params encoded (serialized) or FALSE if not.
*/
bool encode(uint8_t* data, int bufferSize, int& size,
VPipelineParamsMask* mask = nullptr);
/**
* @brief Decode params (deserialize).
* @param data Pointer to data.
* @param dataSize Data size.
* @return TRUE is params decoded or FALSE if not.
*/
bool decode(uint8_t* data, int dataSize);
};
Table 5 - GeneralParams class fields description.
| Field | type | Description |
|---|---|---|
| logMode | int | Logging mode: 0 - Disable (default value). 1 - Only file. 2 - Only terminal (console). 3 - File and terminal (console). The VPipeline library uses Logger library for printing logs into console and(or) file. Note: to enable printing logs in files user have to initialize it by using static method setSaveLogParams(…) of Logger library. This parameter used only for pipeline and doesn’t set logging mode for particular functional modules. To set logging mode for particular functional modules use appropriate parameters enums. |
| enable | bool | Video processing pipeline disable / enable. If pipeline is disabled, it will not perform any processing. Even if the pipeline is disabled use can get video from video source without processing by any functional modules of the pipeline. |
| mainProcessingTimeUs | int | Main video processing cycle time. Required to control pipeline performance. |
| trackerCenteringEnable | bool | Video tracker centering disable / enable. If video tracking centering mode enabled and if video tracker in TRACKING mode (mode == 1) the pipeline will move image to keep tracking rectangle in the center of image (see Video tracker centering mode). |
| trackerCenteringOffsetX | int | Current horizontal image offset for video tracker centering mode. If video tracker centering mode is disabled the value will be 0. |
| trackerCenteringOffsetY | int | Current vertical image offset for video tracker centering mode. If video tracker centering mode is disabled the value will be 0. |
| trackerCenteringMaxOffsetX | int | Maximum image horizontal offset (pixels) for video tracking centering mode. If video tracking centering mode enabled and if video tracker in TRACKING mode (mode == 1) the pipeline will move image to keep tracking rectangle in the center of image (see Video tracker centering mode) but offset can’t be more than maximum offset (the image will be moved according to maximum allowed offset). |
| trackerCenteringMaxOffsetY | int | Maximum image vertical offset (pixels) for video tracking centering mode. If video tracking centering mode enabled and if video tracker in TRACKING mode (mode == 1) the pipeline will move image to keep tracking rectangle in the center of image (see Video tracker centering mode) but offset can’t be more than maximum offset (the image will be moved according to maximum allowed offset). |
Other parameters: VSourceParams - Video source params class. Described in VSource interface class.
LensParams - Lens params class. Described in Lens interface class.
CameraParams - Camera params class. Described in Camera interface class.
VTrackerParams - Video tracker params class. Described in VTracker interface class.
VStabiliserParams - Video stabilizer params class. Described in VStabiliser interface class.
ObjectDetectorParams - Object detector params class. Described in ObjectDetector interface class.
VFilterParams - Video filter params class. Described in VFilter interface class.
Serialize params
The VPipelineParams class provides method encode(…) to serialize pipeline params (fields of VPipelineParams class). Serialization of params necessary in case when you need to send params via communication channels. Method provide options to exclude particular parameters from serialization. To do this method inserts binary mask into encoded data where each bit represents particular parameter and decode(…) method recognizes it. Method declaration:
bool encode(uint8_t* data, int bufferSize, int& size,
VPipelineParamsMask* mask = nullptr);
| Parameter | Value |
|---|---|
| data | Pointer to data buffer. Must not be nullptr. |
| bufferSize | Size of the data buffer, bytes. |
| size | Size of encoded data, bytes. 0 if the params are not encoded. |
| mask | Parameters mask - pointer to VPipelineParamsMask structure. VPipelineParamsMask (declared in VPipeline.h file) determines flags for each field (parameter) declared in VPipelineParams class. If the user wants to exclude a particular parameter from serialization, he should set the corresponding FALSE flag in the VPipelineParamsMask structure and put pointer to method call. VPipelineParamsMask includes additional mask structures: VSourceParamsMask (declared in VSource interface class), VStabiliserParamsMask (declared in VStabiliser interface class), VTrackerParamsMask (declared in VTracker interface class), ObjectDetectorParamsMask (declared in ObjectDetector interface class), LensParamsMask (declared in Lens interface class), VFilterParamsMask (declared in VFilter interface class), CameraParamsMask (declared in Camera interface class) and GeneralParamsMask (declared in VPipeline.h file). |
Returns: TRUE if params serialized or FALSE if not (data is nullptr or bufferSize not enough for serialized data). The content of the buffer is not valid if FALSE is returned.
VPipelineParamsMask and additional structures declaration:
struct GeneralParamsMask
{
/// Serialize logMode field.
bool logMode{ true };
/// Serialize mainProcessingTimeUs field.
bool mainProcessingTimeUs{ true };
/// Serialize enable field.
bool enable{ true };
/// Serialize trackerCenteringEnable field.
bool trackerCenteringEnable{ true };
/// Serialize trackerCenteringOffsetX field.
bool trackerCenteringOffsetX{ true };
/// Serialize trackerCenteringOffsetY field.
bool trackerCenteringOffsetY{ true };
/// Serialize trackerCenteringMaxOffsetX field.
bool trackerCenteringMaxOffsetX{ true };
/// Serialize trackerCenteringMaxOffsetY field.
bool trackerCenteringMaxOffsetY{ true };
};
struct VPipelineParamsMask
{
rapidpixel::VSourceParamsMask videoSource;
rapidpixel::VStabiliserParamsMask videoStabiliser;
rapidpixel::VTrackerParamsMask videoTracker;
rapidpixel::ObjectDetectorParamsMask objectDetector0;
rapidpixel::ObjectDetectorParamsMask objectDetector1;
rapidpixel::ObjectDetectorParamsMask objectDetector2;
rapidpixel::ObjectDetectorParamsMask objectDetector3;
rapidpixel::ObjectDetectorParamsMask objectDetector4;
rapidpixel::ObjectDetectorParamsMask objectDetector5;
rapidpixel::ObjectDetectorParamsMask objectDetector6;
rapidpixel::LensParamsMask lens;
rapidpixel::CameraParamsMask camera;
rapidpixel::VFilterParamsMask videoFilter0;
rapidpixel::VFilterParamsMask videoFilter1;
rapidpixel::VFilterParamsMask videoFilter2;
rapidpixel::VFilterParamsMask videoFilter3;
rapidpixel::VFilterParamsMask videoFilter4;
rapidpixel::VFilterParamsMask videoFilter5;
rapidpixel::VFilterParamsMask videoFilter6;
GeneralParamsMask general;
};
Parameters serialization example without parameters mask:
// Create params object.
rapidpixel::VPipelineParams in;
// Encode params.
uint8_t data[8192];
int size = 0;
if (!in.encode(data, 8192, size))
{
std::cout << "ERROR: Can't encode params" << std::endl;
}
std::cout << "Encoded data size: " << size << " bytes" << std::endl;
Parameters serialization example with parameters mask:
// Fill params mask.
rapidpixel::VPipelineParamsMask mask;
// Exclude some lens params from serialization.
mask.lens.zoomPos = false;
// Exclude some camera params from serialization.
mask.camera.width = false;
mask.camera.height = false;
// Exclude some video source params from serialization.
mask.videoSource.logLevel = false;
mask.videoSource.cycleTimeMks = false;
// Exclude some video stabilizer params from serialization.
mask.videoStabiliser.scaleFactor = false;
mask.videoStabiliser.xOffsetLimit = false;
// Exclude some video tracker params from serialization.
mask.videoTracker.mode = false;
mask.videoTracker.rectX = false;
mask.videoTracker.rectY = false;
// Exclude some first object detector params from serialization.
mask.objectDetector1.logMode = false;
mask.objectDetector1.frameBufferSize = false;
// Exclude some second object detector params from serialization.
mask.objectDetector2.logMode = false;
mask.objectDetector2.frameBufferSize = false;
// Exclude some third object detector params from serialization.
mask.objectDetector3.logMode = false;
mask.objectDetector3.frameBufferSize = false;
// Exclude some others params from serialization.
mask.videoFilter2.mode = false;
mask.videoFilter3.level = false;
// Create params object.
rapidpixel::VPipelineParams in;
// Encode params.
uint8_t data[8192];
int size = 0;
if (!in.encode(data, 8192, size, &mask))
return false;
std::cout << "Encoded data size: " << size << " bytes" << std::endl;
Deserialize params
The VPipelineParams class provides method decode(…) to deserialize params (fields of VPipelineParams class). Deserialization of params necessary in case when you need to receive params via communication channels. Method automatically recognizes which parameters were serialized by encode(…) method. Method declaration:
bool decode(uint8_t* data, int dataSize);
| Parameter | Value |
|---|---|
| data | Pointer to data buffer. Must not be nullptr. |
| dataSize | Size of serialized data, bytes. |
Returns: TRUE if params deserialized or FALSE if not (data is nullptr, the data type or the version is not valid, the data is truncated or the size of a section is not valid). The params object is not changed if FALSE is returned. The bytes after the serialized data are ignored. The values of the params are not checked against any range.
Example:
// Encode params.
rapidpixel::VPipelineParams in;
uint8_t data[8192];
int size = 0;
if (!in.encode(data, 8192, size))
{
std::cout << "ERROR: Can't encode params" << std::endl;
}
std::cout << "Encoded data size: " << size << " bytes" << std::endl;
// Decode params.
rapidpixel::VPipelineParams out;
if (!out.decode(data, size))
{
std::cout << "ERROR: Can't decode params" << std::endl;
}
Read params from JSON file and write to JSON file
The VPipeline library depends on ConfigReader library which provides method to read params from JSON file and to write params to JSON file. Example of writing and reading params to JSON file:
// Prepare params.
rapidpixel::VPipelineParams in;
// Write params to file.
rapidpixel::ConfigReader inConfig;
if (!inConfig.set(in, "VPipelineParams") ||
!inConfig.writeToFile("TestVPipelineParams.json"))
{
std::cout << "Can't write config file" << std::endl;
}
// Read params from file.
rapidpixel::ConfigReader outConfig;
if (!outConfig.readFromFile("TestVPipelineParams.json"))
{
std::cout << "Can't open config file" << std::endl;
}
// Parse JSON file to params structure.
rapidpixel::VPipelineParams out;
if (!outConfig.get(out, "VPipelineParams"))
{
std::cout << "Can't read params from file" << std::endl;
}
TestVPipelineParams.json will look like (part of file):
{
"VPipelineParams": {
"camera": {
"agcMode": 243,
"alcGate": 119,
...
},
"general": {
"enable": true,
"logMode": 82,
"trackerCenteringEnable": true,
"trackerCenteringMaxOffsetX": 45,
"trackerCenteringMaxOffsetY": 197
},
"lens": {
"afHwSpeed": 221,
"afRange": 12,
"afRoiMode": 6,
...
},
"objectDetector0": {
"classNames": [
"0",
"1",
"2"
],
"custom1": 205.0,
"custom2": 225.0,
"custom3": 15.0,
"enable": false,
"frameBufferSize": 211,
...
},
"objectDetector1": {
...
},
"objectDetector2": {
...
},
"objectDetector3": {
...
},
"objectDetector4": {
...
},
"objectDetector5": {
...
},
"objectDetector6": {
...
},
"videoFilter0": {
"custom1": 142.0,
"custom2": 32.0,
"custom3": 127.0,
"level": 232.0,
"mode": 217,
"type": 145
},
"videoFilter1": {
...
},
"videoFilter2": {
...
},
"videoFilter3": {
...
},
"videoFilter4": {
...
},
"videoFilter5": {
...
},
"videoFilter6": {
...
},
"videoSource": {
"custom1": 211.0,
"custom2": 179.0,
"custom3": 25.0,
"exposureMode": 218,
"focusMode": 192,
...
},
"videoStabiliser": {
"aFilterCoeff": 251.0,
"aOffsetLimit": 144.0,
"backend": 99,
...
},
"videoTracker": {
"custom1": 100.0,
"custom2": 150.0,
...
}
}
}
Build and connect to your project
Build stand alone
The repository does not contain sub-repositories. The libraries VPipeline depends on (Frame, ConfigReader, Camera, Lens, ObjectDetector, VSource, VStabiliser, VTracker, VFilter, Logger, FormatConverter) must be cloned into the same folder as VPipeline. The test application with GUI (GuiTest) also uses the libraries SimpleUI, SimpleFileDialog, VCodec, VDecoderH264, VDecoderH265, VDecoderJPEG, VSourceFile, VEncoderJPEG, VideoRecorderMp4: they are needed only if the test applications are built. The folder structure must be:
<work folder>
VPipeline
Frame
ConfigReader
Camera
Lens
ObjectDetector
VSource
VStabiliser
VTracker
VFilter
Logger
FormatConverter
SimpleUI
SimpleFileDialog
VCodec
VDecoderH264
VDecoderH265
VDecoderJPEG
VSourceFile
VEncoderJPEG
VideoRecorderMp4
Typical commands to build the library and the test applications (use the URLs of the repositories provided to you):
git clone <VPipeline repository URL>
git clone <Frame repository URL>
git clone <ConfigReader repository URL>
git clone <Camera repository URL>
git clone <Lens repository URL>
git clone <ObjectDetector repository URL>
git clone <VSource repository URL>
git clone <VStabiliser repository URL>
git clone <VTracker repository URL>
git clone <VFilter repository URL>
git clone <Logger repository URL>
git clone <FormatConverter repository URL>
git clone <SimpleUI repository URL>
git clone <SimpleFileDialog repository URL>
git clone <VCodec repository URL>
git clone <VDecoderH264 repository URL>
git clone <VDecoderH265 repository URL>
git clone <VDecoderJPEG repository URL>
git clone <VSourceFile repository URL>
git clone <VEncoderJPEG repository URL>
git clone <VideoRecorderMp4 repository URL>
cd VPipeline
mkdir build
cd build
cmake ..
make
On Windows use cmake –build . –config Release instead of make. If VPipeline is the top-level CMake project, the main CMakeLists.txt file adds the folders of the libraries (../Frame, ../ConfigReader, ../Camera, ../Lens, ../ObjectDetector, ../VSource, ../VStabiliser, ../VTracker, ../VFilter, ../Logger and ../FormatConverter, in this order) before the library itself, and the folders of the libraries of the test applications before the test applications. If a folder does not exist CMake stops with an error. If VPipeline is included in another project the libraries are not added: they must be added by the project (see Connect to your project).
CMake options
Table 6 - CMake options of the repository. In the options below ${PARENT} is the prefix defined by the top-level project (empty if VPipeline is the top-level project).
| Option | Default | Description |
|---|---|---|
| ${PARENT}_VPIPELINE | ON | Build the library. |
| ${PARENT}_VPIPELINE_TESTS | ON (stand alone), OFF (included in other project) | Build the test applications (ProtocolTest and GuiTest). |
| ${PARENT}_SUBMODULE_CACHE_OVERWRITE | ON | Forced rewriting of the options above in the CMake cache. The top-level project sets it to OFF when it includes VPipeline as a sub-repository. |
| CMAKE_BUILD_TYPE | Release (if not specified) | Build type. |
Connect to your project
If you want connect VPipeline library to your CMake project as source code you can make follow. VPipeline does not add the libraries it depends on when it is included in another project: your project must add them in the order of their mutual dependencies (VPipeline can be added before or after them). For example, if your repository has structure:
CMakeLists.txt
src
CMakeList.txt
yourLib.h
yourLib.cpp
Create 3rdparty folder in your repository and copy the VPipeline repository folder and the repository folders of the libraries it depends on there. The structure of your repository:
CMakeLists.txt
src
CMakeList.txt
yourLib.h
yourLib.cpp
3rdparty
CMakeLists.txt
Frame
ConfigReader
Camera
Lens
ObjectDetector
VSource
VStabiliser
VTracker
VFilter
Logger
FormatConverter
VPipeline
Create CMakeLists.txt file in 3rdparty folder. CMakeLists.txt should contain:
cmake_minimum_required(VERSION 3.13)
################################################################################
## 3RD-PARTY
## dependencies for the project
################################################################################
project(3rdparty LANGUAGES CXX)
################################################################################
## SETTINGS
## basic 3rd-party settings before use
################################################################################
# To inherit the top-level architecture when the project is used as a submodule.
SET(PARENT ${PARENT}_YOUR_PROJECT_3RDPARTY)
# Disable self-overwriting of parameters inside included subdirectories.
SET(${PARENT}_SUBMODULE_CACHE_OVERWRITE OFF CACHE BOOL "" FORCE)
################################################################################
## CONFIGURATION
## 3rd-party submodules configuration
################################################################################
SET(${PARENT}_SUBMODULE_FRAME ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_FRAME)
SET(${PARENT}_FRAME ON CACHE BOOL "" FORCE)
SET(${PARENT}_FRAME_TEST OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_CONFIG_READER ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_CONFIG_READER)
SET(${PARENT}_CONFIG_READER ON CACHE BOOL "" FORCE)
SET(${PARENT}_CONFIG_READER_EXAMPLES OFF CACHE BOOL "" FORCE)
SET(${PARENT}_CONFIG_READER_TESTS OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_CAMERA ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_CAMERA)
SET(${PARENT}_CAMERA ON CACHE BOOL "" FORCE)
SET(${PARENT}_CAMERA_TEST OFF CACHE BOOL "" FORCE)
SET(${PARENT}_CAMERA_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_LENS ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_LENS)
SET(${PARENT}_LENS ON CACHE BOOL "" FORCE)
SET(${PARENT}_LENS_TEST OFF CACHE BOOL "" FORCE)
SET(${PARENT}_LENS_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_OBJECT_DETECTOR ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_OBJECT_DETECTOR)
SET(${PARENT}_OBJECT_DETECTOR ON CACHE BOOL "" FORCE)
SET(${PARENT}_OBJECT_DETECTOR_TEST OFF CACHE BOOL "" FORCE)
SET(${PARENT}_OBJECT_DETECTOR_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_VSOURCE ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VSOURCE)
SET(${PARENT}_VSOURCE ON CACHE BOOL "" FORCE)
SET(${PARENT}_VSOURCE_TEST OFF CACHE BOOL "" FORCE)
SET(${PARENT}_VSOURCE_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_VSTABILISER ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VSTABILISER)
SET(${PARENT}_VSTABILISER ON CACHE BOOL "" FORCE)
SET(${PARENT}_VSTABILISER_TEST OFF CACHE BOOL "" FORCE)
SET(${PARENT}_VSTABILISER_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_VTRACKER ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VTRACKER)
SET(${PARENT}_VTRACKER ON CACHE BOOL "" FORCE)
SET(${PARENT}_VTRACKER_TEST OFF CACHE BOOL "" FORCE)
SET(${PARENT}_VTRACKER_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_VFILTER ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VFILTER)
SET(${PARENT}_VFILTER ON CACHE BOOL "" FORCE)
SET(${PARENT}_VFILTER_TEST OFF CACHE BOOL "" FORCE)
SET(${PARENT}_VFILTER_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_LOGGER ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_LOGGER)
SET(${PARENT}_LOGGER ON CACHE BOOL "" FORCE)
SET(${PARENT}_LOGGER_EXAMPLE OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_FORMAT_CONVERTER ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_FORMAT_CONVERTER)
SET(${PARENT}_FORMAT_CONVERTER ON CACHE BOOL "" FORCE)
SET(${PARENT}_FORMAT_CONVERTER_BENCHMARK OFF CACHE BOOL "" FORCE)
endif()
SET(${PARENT}_SUBMODULE_VPIPELINE ON CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VPIPELINE)
SET(${PARENT}_VPIPELINE ON CACHE BOOL "" FORCE)
SET(${PARENT}_VPIPELINE_TESTS OFF CACHE BOOL "" FORCE)
endif()
################################################################################
## INCLUDING SUBDIRECTORIES
## Adding subdirectories according to the 3rd-party configuration
################################################################################
if (${PARENT}_SUBMODULE_FRAME)
add_subdirectory(Frame)
endif()
if (${PARENT}_SUBMODULE_CONFIG_READER)
add_subdirectory(ConfigReader)
endif()
if (${PARENT}_SUBMODULE_CAMERA)
add_subdirectory(Camera)
endif()
if (${PARENT}_SUBMODULE_LENS)
add_subdirectory(Lens)
endif()
if (${PARENT}_SUBMODULE_OBJECT_DETECTOR)
add_subdirectory(ObjectDetector)
endif()
if (${PARENT}_SUBMODULE_VSOURCE)
add_subdirectory(VSource)
endif()
if (${PARENT}_SUBMODULE_VSTABILISER)
add_subdirectory(VStabiliser)
endif()
if (${PARENT}_SUBMODULE_VTRACKER)
add_subdirectory(VTracker)
endif()
if (${PARENT}_SUBMODULE_VFILTER)
add_subdirectory(VFilter)
endif()
if (${PARENT}_SUBMODULE_LOGGER)
add_subdirectory(Logger)
endif()
if (${PARENT}_SUBMODULE_FORMAT_CONVERTER)
add_subdirectory(FormatConverter)
endif()
if (${PARENT}_SUBMODULE_VPIPELINE)
add_subdirectory(VPipeline)
endif()
File 3rdparty/CMakeLists.txt adds the libraries in the order of their mutual dependencies and the folder VPipeline to your project and excludes test applications from compiling (by default test applications are excluded from compiling if VPipeline is included as sub-repository). Next you need include folder 3rdparty in main CMakeLists.txt file of your repository. Add string at the end of your main CMakeLists.txt:
add_subdirectory(3rdparty)
Next you have to include VPipeline library in your src/CMakeLists.txt file:
target_link_libraries(${PROJECT_NAME} VPipeline)
If your project does not add the libraries VPipeline depends on the compilation stops with an error about a header of a library, for example Camera.h: No such file or directory. Done!
Portability
The library is written in standard C++17 (compiler extensions are not used) and does not use code specific for a CPU architecture. The library was built and tested with MSVC 2022 (x86-64), GCC 12 and 13 (x86-64), Clang 15 and 18 (x86-64) and GCC 13 cross compiler for aarch64 (test run under qemu). The encoded params and commands store int and float values as raw 4 bytes values in the byte order of the host: the library assumes that int and float have size 4 bytes (checked during compilation) and that the byte order of both sides of a communication channel is the same (all supported platforms are little-endian).
Example
This example shows how to create VPipeline object with all “dummy” functional modules, obtain video frames and control pipeline. “Dummy” video source module will generate artificial video. Example uses SimpleUI library to display video (the library must be added to the project in the same way as the libraries VPipeline depends on). The test application GuiTest (tests/GuiTest folder) shows the same pipeline and can also use a video file as video source (VSourceFile library) and record video (VideoRecorderMp4 library).
#include <exception>
#include <iostream>
#include <string>
#include "SimpleUi.h"
#include "VPipeline.h"
int main()
{
try
{
std::cout << "VPipeline v" <<
rapidpixel::VPipeline::getVersion() << std::endl;
// Prepare some pipeline params.
rapidpixel::VPipelineParams params;
params.videoSource.width = 1280;
params.videoSource.height = 720;
params.videoSource.fps = 30;
params.general.logMode = 2;
params.videoSource.logLevel = 2;
params.objectDetector1.logMode = 2;
params.lens.logMode = 2;
params.camera.logMode = 2;
// Create and init pipeline.
rapidpixel::VPipeline pipeline;
if (!pipeline.initVPipeline(params))
{
std::cerr << "ERROR: Can't init pipeline" << std::endl;
return -1;
}
// Create window to display video.
rapidpixel::SimpleUiWindow window("VPipeline");
window.show();
// Output video frame and text color (red).
rapidpixel::Frame frame;
const rapidpixel::Color red{255, 0, 0, 255};
// Main loop.
while (window.isOpen())
{
// Wait YUV24 video frame from pipeline 1 sec.
if (!pipeline.getFrame(frame, 1000))
{
continue;
}
// Get pipeline params.
pipeline.getParams(params);
// Print processing time.
rapidpixel::SimpleUiDraw::drawText(
frame, 10, 30, "MAIN PROCESSING TIME " +
std::to_string(static_cast<float>(
params.general.mainProcessingTimeUs) / 1000.0f) +
" msec", red, 20);
// Print centering mode.
rapidpixel::SimpleUiDraw::drawText(
frame, 10, 60, "TRACKER CENTERING: " +
std::string(params.general.trackerCenteringEnable ?
"ON" : "OFF"), red, 20);
// Display YUV24 frame.
window.render(frame);
// Process keyboard events.
switch (window.getKey(1))
{
case 27: // ESC - exit.
return 0;
case 32: // SPACE - Enable/disable pipeline.
if (!pipeline.executeCommand(
rapidpixel::VPipelineModule::GENERAL,
params.general.enable ?
static_cast<int>(rapidpixel::GeneralCommand::OFF) :
static_cast<int>(rapidpixel::GeneralCommand::ON)))
{
std::cerr << "ERROR: Can't change pipeline mode" <<
std::endl;
}
break;
case 13: // ENTER - Enable/disable tracker centering.
if (!pipeline.setParam(
rapidpixel::VPipelineModule::GENERAL,
static_cast<int>(rapidpixel::GeneralParam::
TRACKER_CENTERING_ENABLE_MODE),
params.general.trackerCenteringEnable ? 0.0f : 1.0f))
{
std::cerr << "ERROR: Can't change centering mode" <<
std::endl;
}
break;
default:
break;
}
}
}
catch (const std::exception& exception)
{
std::cerr << "ERROR: " << exception.what() << std::endl;
return -1;
}
return 0;
}