vpipeline_web_logo

VPipeline C++ library

v3.0.4

Overview
Versions
Library files
VPipeline class description
Enums
VPipelineParams class description
Build and connect to your project
Example

Overview

The VPipeline C++ library is the full functional video processing pipeline with interfaces to connect different video filters, video trackers, object detectors and video stabilization modules. The motivation of VPipeline library is to provide simple interface to complex video processing pipeline as well as camera and lens control, regardless of hardware. The library strikes two main goals: provide video processing pipeline and provide remote control interface for pipeline (encoding / decoding commands and telemetry). It can be used inside edge device for video processing as well as on client side to control video processing pipeline. The VPipeline library allows users to set their own implementation of particular algorithms through interfaces (VFilter, VTracker, VStabiliser, VSource and ObjectDetector). Thus the library offers flexibility in algorithms usage with keeping standard interfaces to control each of them. The library depends on: Camera library (provides interface to control / obtain camera / image sensor parameters, source code included, Apache 2.0 license), Lens library (provides interface to control / obtaining lens parameters, source code included, Apache 2.0 license), ObjectDetector library (provides interface for object detectors, source code included, Apache 2.0 license), VSource library (provides interface for video capture, source code included, Apache 2.0 license), VTracker library (provides interface for video tracking, source code included, Apache 2.0 license), VStabiliser library (provides interface for video stabilization, source code included, Apache 2.0 license), VFilter library (provides interface for video filtering, source code included, Apache 2.0 license), Logger (provides functions for logging, source code included, Apache 2.0 license) and FormatConverterOpenCv (provides functions to convert pixel formats, source code included). The VPipeline library doesn’t include implementation of particular video processing algorithms, but includes interfaces for them. The library uses C++17 standard and doesn’t have any dependency to be specially installed in OS. The library compatible with Linux and Windows. Video processing pipeline structure:

vpipeline_structure

The library includes following video processing steps: waiting next video frame from video source, converting source pixel format to YUV24, filtering frame by first video filter (usually image flip function), filtering frame by second video filter (usually digital zoom), stabilizing video by video stabilizer library, filtering frame by third video filter (usually denoise), filtering frame by fourth video filter (usually defog / dehaze), filtering frame by fifth video filter (usually motion magnification), sending video frame to video tracker thread, first, second, third and fourth object detectors threads. Also the pipeline sends result video frame after stabilization to lens controller for autofocus functions. The library splits video preprocessing (video capture -> fifth video filter) by separate threads: first thread (video capture -> format converter -> first video filter -> second video filter), second thread (video stabilizer), third thread (third video filter), fourth thread (fourth video filter), fifth thread (fifth video filter). Each filter waits results from previous and notifies next after processing. Video tracker, first, second, third and fourth object detector are working in separate threads.

The VPipeline library provides simple interface to control pipeline including camera and lens control. Usage principles: user initializes pipeline by calling initVPipeline(…) method. The initVPipeline(…) method accepts big structure of parameters which includes parameters for all functional modules of the pipeline. Also user should put pointer to objects: video stabilizer, video tracker, object detectors, video source, camera controller, lens controller and video filters. If particular pointer not initialized the pipeline will initialize default “dummy” module which does nothing. After initialization the pipeline runs internal processing threads. User can control pipeline by changing parameters (setParam(…) method), executing command (executeCommand(…) method) and setting detection and filtering masks for object detectors and video filters (setMask(…) method). All this methods are non-blocking and thread-safe. To get all actual pipeline params including tracking and detection results user uses getParams(…) method. To get new video frame from video source after preprocessing (after fifth video filter) the getFrame(…) method used. Thus the user have standard interface to camera, lens, video source, video tracker and others regardless of hardware type. VPipeline class provides the same set of commands and functions for any combination of lens and camera. If user doesn’t use lens and camera interface the pipeline can work without it and without any other functional module if needed. The VPipeline library simplifies embedded video processing software structure. Usage example:

vpipeline_principles

Versions

Table 1 - Library versions.

Version	Release date	What’s new
1.0.0	18.05.2022	First version.
2.0.0	18.02.2024	- Interface changed. - Added new video processing modules. - Video streaming functions excluded. - Documentation updated.
3.0.0	14.03.2024	- Full functional implementation. - Added VFilter interface. - Added test application. - Documentation updated.
3.0.1	26.03.2024	- Camera class updated. - FormatConverterOpenCv class updated. - Lens class updated. - Logger class updated. - ObjectDetector class updated. - VFilter class updated. - VSource class updated. - VStabiliser class updated. - VTracker class updated. - Documentation updated.
3.0.2	21.05.2024	- Submodules updated. - Documentation updated.
3.0.3	08.08.2024	- CMake structure updated. - Minor casting issues fixed.
3.0.4	29.09.2024	- Update VStabiliser interface.

Library files

The VPipeline C++ library is provided as source code. Users are given a set of files in the form of a CMake project (repository). The repository structure is outlined below:

CMakeLists.txt --------------------------- Main CMake file of the library.
3rdparty --------------------------------- Folder with third-party libraries.
    CMakeLists.txt ----------------------- CMake file to include third-party libraries.
    Camera ------------------------------- Folder with the Camera library source code.
    Lens --------------------------------- Folder with the Lens library source code.
    ObjectDetector ----------------------- Folder with the ObjectDetector library source code.
    VSource ------------------------------ Folder with the VSource library source code.
    VStabiliser -------------------------- Folder with the VStabiliser library source code.
    VTracker ----------------------------- Folder with the VTracker library source code.
    VFilter ------------------------------ Folder with VFilter library source code.
    FormatConverterOpenCv ---------------- Folder with FormatConverterOpenCv library source code.
    DummyCamera -------------------------- Folder with "dummy" camera implementation.
        CMakeLists.txt ------------------- CMake file for "dummy" camera.
        DummyCamera.cpp ------------------ Source code file of the library.
        DummyCamera.h -------------------- Header file of the library.
        DummyCameraVersion.h ------------- Header file with library version.
        DummyCameraVersion.h.in ---------- CMake service file to generate version header.
    DummyLens ---------------------------- Folder with "dummy" lens implementation.
        CMakeLists.txt ------------------- CMake file for "dummy" lens.
        DummyLens.cpp -------------------- Source code file of the library.
        DummyLens.h ---------------------- Header file of the library.
        DummyLensVersion.h --------------- Header file with library version.
        DummyLensVersion.h.in ------------ CMake service file to generate version header.
    DummyObjectDetector ------------------ Folder with "dummy" object detector implementation.
        CMakeLists.txt ------------------- CMake file for "dummy" object detector.
        DummyObjectDetector.cpp ---------- Source code file of the library.
        DummyObjectDetector.h ------------ Header file of the library.
        DummyObjectDetectorVersion.h ----- Header file with library version.
        DummyObjectDetectorVersion.h.in -- CMake service file to generate version header.
    DummyVFilter ------------------------- Folder with "dummy" video filter implementation.
        CMakeLists.txt ------------------- CMake file for "dummy" video filter.
        DummyVFilter.cpp ----------------- Source code file of the library.
        DummyVFilter.h ------------------- Header file of the library.
        DummyVFilterVersion.h ------------ Header file with library version.
        DummyVFilterVersion.h.in --------- CMake service file to generate version header.
    DummyVSource ------------------------- Folder with "dummy" video source implementation.
        CMakeLists.txt ------------------- CMake file for "dummy" video source.
        DummyVSource.cpp ----------------- Source code file of the library.
        DummyVSource.h ------------------- Header file of the library.
        DummyVSourceVersion.h ------------ Header file with library version.
        DummyVSourceVersion.h.in --------- CMake service file to generate version header.
    DummyVStabiliser --------------------- Folder with "dummy" video stabilizer implementation.
        CMakeLists.txt ------------------- CMake file for "dummy" video stabilizer.
        DummyVStabiliser.cpp ------------- Source code file of the library.
        DummyVStabiliser.h --------------- Header file of the library.
        DummyVStabiliserVersion.h -------- Header file with library version.
        DummyVStabiliserVersion.h.in ----- CMake service file to generate version header.
    DummyVTracker ------------------------ Folder with "dummy" video tracker implementation.
        CMakeLists.txt ------------------- CMake file for "dummy" video tracker library.
        DummyVTracker.cpp ---------------- Source code file of the library.
        DummyVTracker.h ------------------ Header file of the library.
        DummyVTrackerVersion.h ----------- Header file with library version.
        DummyVTrackerVersion.h.in -------- CMake service file to generate version header.
src -------------------------------------- Folder with source code of the library.
    CMakeLists.txt ----------------------- CMake file of the library.
    VPipeline.cpp ------------------------ C++ implementation file.
    VPipeline.h -------------------------- Header file with VPipeline class declaration.
    VPipelineVersion.h ------------------- Header file which includes version of the library.
    VPipelineVersion.h.in ---------------- CMake service file to generate version header.
tests ------------------------------------ Folder with test applications.
    GuiTest ------------------------------ Folder with test application with GUI.
        CMakeLists.txt ------------------- CMake file of the test application.
        main.cpp ------------------------- Source code file of test application.
    ProtocolTest ------------------------- Folder with protocol test application.
        CMakeLists.txt ------------------- CMake file of the test application.
        main.cpp ------------------------- Source code file of test application.

VPipeline class description

VPipeline class declaration

VPipeline.h file contains VPipeline class declaration. Class declaration:

class VPipeline
{
public:

    /// Class constructor.
    VPipeline();

    /// Class destructor.
    ~VPipeline();

    /// Get string of current class version.
    static std::string getVersion();

    /// Init video processing pipeline.
    bool initVPipeline(
        VPipelineParams& params,
        cr::video::VSource* videoSource = nullptr,
        cr::lens::Lens* lens = nullptr,
        cr::camera::Camera* camera = nullptr,
        cr::vstab::VStabiliser* videoStabiliser = nullptr,
        cr::vtracker::VTracker* videoTracker = nullptr,
        cr::detector::ObjectDetector* objectDetector1 = nullptr,
        cr::detector::ObjectDetector* objectDetector2 = nullptr,
        cr::detector::ObjectDetector* objectDetector3 = nullptr,
        cr::detector::ObjectDetector* objectDetector4 = nullptr,
        cr::video::VFilter* videoFilter1 = nullptr,
        cr::video::VFilter* videoFilter2 = nullptr,
        cr::video::VFilter* videoFilter3 = nullptr,
        cr::video::VFilter* videoFilter4 = nullptr,
        cr::video::VFilter* videoFilter5 = nullptr);

    /// Get video processing pipeline params.
    void getParams(VPipelineParams& params);

    /// Set module parameter.
    bool setParam(VPipelineModule module, int id, float value);

    /// Execute command.
    bool executeCommand(VPipelineModule module,
                        int id,
                        float arg1 = -1.0f,
                        float arg2 = -1.0f,
                        float arg3 = -1.0f);

    /// Get source video frame after preprocessing (fifth video filter).
    bool getFrame(cr::video::Frame& frame, int timeoutMsec = 0);

    /// Set detection mask to module.
    bool setMask(VPipelineModule module, cr::video::Frame mask);

    /// Encode set param command.
    static void encodeSetParamCommand(uint8_t* data,
                                      int& size,
                                      VPipelineModule module,
                                      int id,
                                      float value);

    /// Encode action command.
    static void encodeCommand(uint8_t* data,
                              int& size,
                              VPipelineModule module,
                              int id,
                              float arg1 = 0.0f,
                              float arg2 = 0.0f,
                              float arg3 = 0.0f);

    /// Decode command.
    static int decodeCommand(uint8_t* data,
                             int size,
                             VPipelineModule& module,
                             int& id,
                             float& value1,
                             float& value2,
                             float& value3);

    /// Decode and execute command.
    bool decodeAndExecuteCommand(uint8_t* data, int size);
};

getVersion method

The getVersion() static method returns string of current version of VPipeline class. Method declaration:

static std::string getVersion();

Method can be used without VPipeline class instance. Example:

cout << "VPipeline class version: " << VPipeline::getVersion() << endl;

Console output:

VPipeline class version: 3.0.2

initVPipeline method

The initVPipeline(…) method initializes video processing pipeline. This method initializes all functional modules by parameters given by user and starts internal processing threads. This method must be called first before any other methods (excepts static methods to encode / decode commands and params). User should put pointers to objects for particular modules (camera controller, lens controller, video source, video stabilizer, video tracker, object detectors and video filters). If user doesn’t provide pointer to particular module the pipeline will initialize “dummy” module which does nothing. After initialization method will run internal video processing threads. Method declaration:

bool initVPipeline(
        VPipelineParams& params,
        cr::video::VSource* videoSource = nullptr,
        cr::lens::Lens* lens = nullptr,
        cr::camera::Camera* camera = nullptr,
        cr::vstab::VStabiliser* videoStabiliser = nullptr,
        cr::vtracker::VTracker* videoTracker = nullptr,
        cr::detector::ObjectDetector* objectDetector1 = nullptr,
        cr::detector::ObjectDetector* objectDetector2 = nullptr,
        cr::detector::ObjectDetector* objectDetector3 = nullptr,
        cr::detector::ObjectDetector* objectDetector4 = nullptr,
        cr::video::VFilter* videoFilter1 = nullptr,
        cr::video::VFilter* videoFilter2 = nullptr,
        cr::video::VFilter* videoFilter3 = nullptr,
        cr::video::VFilter* videoFilter4 = nullptr,
        cr::video::VFilter* videoFilter5 = nullptr);

Parameter	Description
params	VPipelineParams class object which contains all pipeline parameters including parameters for functional modules.
videoSource	Pointer to VSource object. The pipeline will use this object to capture video from video source. User should provide particular implementation of VSource interface. If user doesn’t provide pointer to video source the pipeline will create “dummy” video source module which generates artificial video frames itself with default parameters (resolution and FPS). If user provides pointer to video source implementation and if initialization failed the pipeline will initialize “dummy” video source.
lens	Pointer to Lens object. The pipeline will use this object to control lens. User should provide particular implementation of Lens interface. If user doesn’t provide pointer to lens controller the pipeline will create “dummy” lens controller. If particular controller has Lens and Camera interfaces (for block cameras which include image sensor and lens) user should put the same pointer separately for lens and camera interfaces.
camera	Pointer to Camera object. The pipeline will use this object to control camera. User should provide particular implementation of Camera interface. If user doesn’t provide pointer to camera controller the pipeline will create “dummy” camera controller. If particular controller has Lens and Camera interfaces (for block cameras which include image sensor and lens) user should put the same pointer separately for lens and camera interfaces.
videoStabiliser	Pointer to VStabiliser object. The pipeline will use this object to stabilize video frames. User should provide particular implementation of VStabiliser interface. If user doesn’t provide pointer to video stabilizer this function will be not available for user.
videoTracker	Pointer to VTracker object. The pipeline will use this object for video tracking of single object. User should provide particular implementation of VTracker interface. If user doesn’t provide pointer to video tracker the pipeline will create “dummy” video tracker.
objectDetector1	Pointer to first ObjectDetector object (first object detector). The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes four slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector.
objectDetector2	Pointer to second ObjectDetector object (second object detector). The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes four slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector.
objectDetector3	Pointer to third ObjectDetector object (third object detector). The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes four slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector.
objectDetector4	Pointer to fourth ObjectDetector object (fourth object detector). The pipeline will use this object to detect objects on video. Object detector can be: motion detector, changes detector, neural networks etc. But all detectors must be compatible with ObjectDetector interface to be understandable by pipeline. VPipeline includes four slots for object detectors. User can obtain detection results with getParams(…) method. If user doesn’t provide pointer to object detector the pipeline will create “dummy” object detector.
videoFilter1	Pointer to first VFilter object (first video filter). User can assign any video filter compatible with VFilter interface. Usually first video filter is image flip function. VPipeline includes five slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter.
videoFilter2	Pointer to second VFilter object (second video filter). User can assign any video filter compatible with VFilter interface. Usually second video filter is digital zoom function. VPipeline includes five slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter.
videoFilter3	Pointer to third VFilter object (third video filter). User can assign any video filter compatible with VFilter interface. Usually third video filter is noise removal function (denoise). VPipeline includes five slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter.
videoFilter4	Pointer to fourth VFilter object (fourth video filter). User can assign any video filter compatible with VFilter interface. Usually fourth video filter is defog / dehaze function. VPipeline includes five slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter.
videoFilter5	Pointer to fifth VFilter object (fifth video filter). User can assign any video filter compatible with VFilter interface. Usually fifth video filter is motion magnification function. VPipeline includes five slots for video filters. User can obtain video filter params with getParams(…) method. If user doesn’t provide pointer to video filter the pipeline will create “dummy” video filter.

Returns: TRUE if pipeline initialized (or already initialized) or FALSE if not. The method will return FALSE only if video source initialization failed. The pipeline firstly will try initialize video source provided by user (if provided) and in case any errors will try initialize “dummy” video source.

getParams method

The getParams(…) method returns all video processing pipeline parameters including detection and video tracking results. The method will obtain current params from each module (video source, video stabilizer, video tracker, object detector and video filters). Method declaration:

void getParams(VPipelineParams& params);

Parameter	Description
params	VPipelineParams class which contains all pipeline parameters including parameters for functional modules, detection and tracking results.

setParam method

The setParam(…) changes parameter of particular video processing pipeline module. User must specify functional module of pipeline to set parameter value. The method will check module type and will call setParam(…) method of appropriate module (functional module object). Depends on particular implementation of functional module new parameters value can be accepted but not set. To control current parameters values use getParams(…) method. Method declaration:

bool setParam(VPipelineModule module, int id, float value);

Parameter	Description
module	Functional module of pipeline according to VPipelineModule enum. Changing parameters available for all functional modules: VPipelineModule::VIDEO_SOURCE VPipelineModule::LENS VPipelineModule::CAMERA VPipelineModule::VIDEO_TRACKER VPipelineModule::VIDEO_STABILISER VPipelineModule::OBJECT_DETECTOR_1 VPipelineModule::OBJECT_DETECTOR_2 VPipelineModule::OBJECT_DETECTOR_3 VPipelineModule::OBJECT_DETECTOR_4 VPipelineModule::VIDEO_FILTER_1 VPipelineModule::VIDEO_FILTER_2 VPipelineModule::VIDEO_FILTER_3 VPipelineModule::VIDEO_FILTER_4 VPipelineModule::VIDEO_FILTER_5 VPipelineModule::GENERAL
id	Parameter ID. Parameter ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Changing parameters available for all functional modules: VSourceParam - Video source parameters enum. Described in VSource interface class. LensParam - Lens parameters enum. Described in Lens interface class. CameraParam - Camera parameters enum. Described in Camera interface class. VTrackerParam - Video tracker parameters enum. Described in VTracker interface class. VStabiliserParam - Video stabilizer parameters enum. Described in VStabiliser interface class. VFilterParam - Video filter parameters enum. Described in VFilter interface class. GeneralParam - General pipeline parameters enum. Described in VPipeline.h file. ObjectDetectorParam - Object detector parameters enum. Described in ObjectDetector interface class.
value	Parameter value. Depends on parameter ID and functional module. To check valid parameters values see description of appropriate enum for particular implementation.

Returns: TRUE if parameter was accepted or FALSE if not.

Example how to set parameters for different video processing pipeline modules:

// Set video source parameter.
pipeline->setParam(VPipelineModule::VIDEO_SOURCE,
                   (int)VSourceParam::FOCUS_MODE, 0);

// Set lens parameter.
pipeline->setParam(VPipelineModule::LENS,
                   (int)LensParam::FOCUS_POS, 1000);

// Set camera parameter.
pipeline->setParam(VPipelineModule::CAMERA,
                   (int)CameraParam::BRIGHTNESS, 100);

// Set video tracker parameter.
pipeline->setParam(VPipelineModule::VIDEO_TRACKER,
                   (int)VTrackerParam::NUM_CHANNELS, 3);

// Set video stabilizer parameter.
pipeline->setParam(VPipelineModule::VIDEO_STABILISER,
                   (int)VStabiliserParam::TYPE, 1);

// Set first video filter parameter.
pipeline->setParam(VPipelineModule::VIDEO_FILTER_1,
                   (int)VFilterParam::LEVEL, 20);

// Set second video filter parameter.
pipeline->setParam(VPipelineModule::VIDEO_FILTER_2,
                   (int)VFilterParam::LEVEL, 10);

// Set third video filter parameter.
pipeline->setParam(VPipelineModule::VIDEO_FILTER_3,
                   (int)VFilterParam::TYPE, 1);

// Set fourth video filter parameter.
pipeline->setParam(VPipelineModule::VIDEO_FILTER_4,
                   (int)VFilterParam::LEVEL, 50);

// Set fifth video filter parameter.
pipeline->setParam(VPipelineModule::VIDEO_FILTER_5,
                   (int)VFilterParam::CUSTOM_1, 1.5);

// Set first object detector parameter.
pipeline->setParam(VPipelineModule::OBJECT_DETECTOR_1,
                   (int)ObjectDetectorParam::SCALE_FACTOR, 2);

// Set second object detector parameter.
pipeline->setParam(VPipelineModule::OBJECT_DETECTOR_2,
                   (int)ObjectDetectorParam::MIN_X_SPEED, 1.5);

// Set third object detector parameter.
pipeline->setParam(VPipelineModule::OBJECT_DETECTOR_3,
                   (int)ObjectDetectorParam::MIN_Y_SPEED, 2.5);

// Set fourth object detector parameter.
pipeline->setParam(VPipelineModule::OBJECT_DETECTOR_4,
                   (int)ObjectDetectorParam::MIN_Y_SPEED, 1.5);

// Set general pipeline parameter.
pipeline->setParam(VPipelineModule::GENERAL,
                   (int)GeneralParam::LOG_MODE, 1);

executeCommand method

The executeCommand(…) method performs action command for particular video processing pipeline module. User must specify functional module of pipeline to execute command. The method will check module type and will call executeCommand(…) method of appropriate module (functional module object). Method declaration:

bool executeCommand(VPipelineModule module,
                    int id,
                    float arg1 = -1.0f,
                    float arg2 = -1.0f,
                    float arg3 = -1.0f);

Parameter	Description
module	Functional module of pipeline according to VPipelineModule enum. Commands available for all modules: VPipelineModule::VIDEO_SOURCE VPipelineModule::LENS VPipelineModule::CAMERA VPipelineModule::VIDEO_TRACKER VPipelineModule::VIDEO_STABILISER VPipelineModule::OBJECT_DETECTOR_1 VPipelineModule::OBJECT_DETECTOR_2 VPipelineModule::OBJECT_DETECTOR_3 VPipelineModule::OBJECT_DETECTOR_4 VPipelineModule::VIDEO_FILTER_1 VPipelineModule::VIDEO_FILTER_2 VPipelineModule::VIDEO_FILTER_3 VPipelineModule::VIDEO_FILTER_4 VPipelineModule::VIDEO_FILTER_5 VPipelineModule::GENERAL
id	Action command ID. Command ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Commands available for all modules: VSourceCommand - Video source action commands enum. Described in VSource interface class. LensCommand - Lens action commands enum. Described in Lens interface class. CameraCommand - Camera action commands enum. Described in Camera interface class. VTrackerCommand - Video tracker action commands enum. Described in VTracker interface class. VStabiliserCommand - Video stabilizer action commands enum. Described in VStabiliser interface class. ObjectDetectorCommand - Object detector commands enum. Described in ObjectDetector interface class. VFilterCommand - Video filter action commands enum. Described in VFilter interface class. GeneralCommand - General pipeline action commands enum. Described in VPipeline.h file.
arg1	First command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum for particular implementation.
arg2	Second command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum for particular implementation.
arg3	Third command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum for particular implementation.

Returns: TRUE if the command executed (accepted by pipeline) or FALSE if not.

Example how to execute command for different video processing pipeline modules:

// Execute video source command.
pipeline->executeCommand(VPipelineModule::VIDEO_SOURCE,
                         (int)VSourceCommand::RESTART);
    
// Execute lens command.
pipeline->executeCommand(VPipelineModule::LENS,
                         (int)LensCommand::ZOOM_TO_POS, 1000);
    
// Execute camera command.
pipeline->executeCommand(VPipelineModule::CAMERA,
                         (int)CameraCommand::FREEZE);
    
// Execute video tracker command.
pipeline->executeCommand(VPipelineModule::VIDEO_TRACKER,
                         (int)VTrackerCommand::CAPTURE, 100, 50, -1);
    
// Execute video stabilizer command.
pipeline->executeCommand(VPipelineModule::VIDEO_STABILISER,
                         (int)VStabiliserCommand::RESET);

// Execute first video filter command.
pipeline->executeCommand(VPipelineModule::VIDEO_FILTER_1,
                         (int)VFilterCommand::ON);

// Execute second video filter command.
pipeline->executeCommand(VPipelineModule::VIDEO_FILTER_2,
                         (int)VFilterCommand::OFF);

// Execute third video filter command.
pipeline->executeCommand(VPipelineModule::VIDEO_FILTER_3,
                         (int)VFilterCommand::RESET);

// Execute fourth video filter command.
pipeline->executeCommand(VPipelineModule::VIDEO_FILTER_4,
                         (int)VFilterCommand::ON);

// Execute fifth video filter command.
pipeline->executeCommand(VPipelineModule::VIDEO_FILTER_5,
                         (int)VFilterCommand::ON);
    
// Execute first object detector command.
pipeline->executeCommand(VPipelineModule::OBJECT_DETECTOR_1,
                         (int)ObjectDetectorCommand::ON);
    
// Execute second object detector command.
pipeline->executeCommand(VPipelineModule::OBJECT_DETECTOR_2,
                         (int)ObjectDetectorCommand::OFF);
    
// Execute third object detector command.
pipeline->executeCommand(VPipelineModule::OBJECT_DETECTOR_3,
                         (int)ObjectDetectorCommand::RESET);

// Execute fourth object detector command.
pipeline->executeCommand(VPipelineModule::OBJECT_DETECTOR_4,
                         (int)ObjectDetectorCommand::RESET);

getFrame method

The getFrame(…) method returns the current video frame captured from the video source after its processing by fifth video filter. The pipeline can be enabled / disabled. If pipeline disabled the video source still works and the getFrame(…) method will return frame after video source module without processing by any functional module. Method declaration:

bool getFrame(cr::video::Frame& frame, int timeoutMsec = 0);

Parameter	Description
frame	Reference to Frame object. The method will return the video frame if YUV24 format (YUV 24 bits).
timeoutMsec	Timeout to wait new frame, milliseconds. Values: 0 - Waiting endlessly for a new video frame to arrive. 0 - Just check if there is new video frame to return. If not, the method will return FALSE immediately. >0 - Wait for a new video frame a certain timeout. If there is no new frame for the specified timeout the method will return FALSE.

Returns: TRUE if a new frame is returned or FALSE if not.

setMask method

The setMask(…) method sets detection or filtering mask to object detectors or video filters in pipeline. The method will check module type and will call setMask(…) method of appropriate module (ObjectDetector or VFilter object). To set mask the particular object detector or video filter must be initialized in initVPipeline(…) method and must support this capability. If user didn’t set particular object detector or video filter in initVPipeline(…) method the pipeline will initialize “dummy” module which doesn’t support mask (method setMask(…) will return FALSE). The method accepts mask in form of video frame which allows user to set mask of any configuration (not only rectangular). Method declaration:

bool setMask(VPipelineModule module, cr::video::Frame mask);

Parameter	Description
module	Functional module of pipeline according to VPipelineModule enum. This function available only for modules: VPipelineModule::OBJECT_DETECTOR_1 VPipelineModule::OBJECT_DETECTOR_2 VPipelineModule::OBJECT_DETECTOR_3 VPipelineModule::OBJECT_DETECTOR_4 VPipelineModule::VIDEO_FILTER_1 VPipelineModule::VIDEO_FILTER_2 VPipelineModule::VIDEO_FILTER_3 VPipelineModule::VIDEO_FILTER_4 VPipelineModule::VIDEO_FILTER_5
mask	Reference to Frame object which represents detection mask. Detection mask is an image with possible pixel formats: GRAY, NV12, NV21, YU12 and YV12 (Fourcc enum described in Frame library). Recommended GRAY pixel format (most common for all modules). The image can have any resolution. If detection mask resolution is different than input video frames for particular module (object detector or video filter) the particular module will rescale to original input image resolution for processing. If particular object detector or video filter doesn’t support mask the method will return FALSE.

Parameter

Description

module

Functional module of pipeline according to VPipelineModule enum. This function available only for modules:
VPipelineModule::OBJECT_DETECTOR_1
VPipelineModule::OBJECT_DETECTOR_2
VPipelineModule::OBJECT_DETECTOR_3
VPipelineModule::OBJECT_DETECTOR_4
VPipelineModule::VIDEO_FILTER_1
VPipelineModule::VIDEO_FILTER_2
VPipelineModule::VIDEO_FILTER_3
VPipelineModule::VIDEO_FILTER_4
VPipelineModule::VIDEO_FILTER_5

mask

Reference to Frame object which represents detection mask. Detection mask is an image with possible pixel formats: GRAY, NV12, NV21, YU12 and YV12 (Fourcc enum described in Frame library). Recommended GRAY pixel format (most common for all modules). The image can have any resolution. If detection mask resolution is different than input video frames for particular module (object detector or video filter) the particular module will rescale to original input image resolution for processing. If particular object detector or video filter doesn’t support mask the method will return FALSE.

Returns: TRUE if a the detection or filtering mask is set (accepted by pipeline) or FALSE if not.

Example how to prepare rectangular mask with OpenCV and set it to different modules:

// Create mask with any resolution. It will be rescaled in pipeline.
Frame mask(256, 256, Fourcc::GRAY);
cv::Mat maskImg(256, 256, CV_8UC1, mask.data);
cv::rectangle(maskImg, cv::Point(50, 50), cv::Point(100, 100),
              cv::Scalar(255, 255, 255), cv::FILLED);

// Set mask to all modules which support it.
pipeline->setMask(VPipelineModule::OBJECT_DETECTOR_1, mask);
pipeline->setMask(VPipelineModule::OBJECT_DETECTOR_2, mask);
pipeline->setMask(VPipelineModule::OBJECT_DETECTOR_3, mask);
pipeline->setMask(VPipelineModule::OBJECT_DETECTOR_4, mask);
pipeline->setMask(VPipelineModule::VIDEO_FILTER_1, mask);
pipeline->setMask(VPipelineModule::VIDEO_FILTER_2, mask);
pipeline->setMask(VPipelineModule::VIDEO_FILTER_3, mask);
pipeline->setMask(VPipelineModule::VIDEO_FILTER_4, mask);
pipeline->setMask(VPipelineModule::VIDEO_FILTER_5, mask);

encodeSetParamCommand static method

The encodeSetParamCommand(…) static method encodes command to change any parameters of remote video processing pipeline. To control a pipeline remotely, the developer has to develop his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VPipeline class contains static methods for encoding the control commands. The VPipeline class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeSetParamCommand(…) designed to encode SET_PARAM command. To encode command user doesn’t need initialize pipeline. Method declaration:

static void encodeSetParamCommand(uint8_t* data,
                                  int& size,
                                  VPipelineModule module,
                                  int id,
                                  float value);

Parameter	Description
data	Pointer to data buffer for encoded command. Must have size >= 32 bytes.
size	Size of encoded data (command). Will be minimum 15 bytes.
module	Functional module of pipeline according to VPipelineModule enum. Commands available for all modules: VPipelineModule::VIDEO_SOURCE VPipelineModule::LENS VPipelineModule::CAMERA VPipelineModule::VIDEO_TRACKER VPipelineModule::VIDEO_STABILISER VPipelineModule::OBJECT_DETECTOR_1 VPipelineModule::OBJECT_DETECTOR_2 VPipelineModule::OBJECT_DETECTOR_3 VPipelineModule::OBJECT_DETECTOR_4 VPipelineModule::VIDEO_FILTER_1 VPipelineModule::VIDEO_FILTER_2 VPipelineModule::VIDEO_FILTER_3 VPipelineModule::VIDEO_FILTER_4 VPipelineModule::VIDEO_FILTER_5 VPipelineModule::GENERAL
id	Parameter ID. Parameter ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Encoding SET_PARAMS command available for all functional modules: VSourceParam - Video source parameters enum. Described in VSource interface class. LensParam - Lens parameters enum. Described in Lens interface class. CameraParam - Camera parameters enum. Described in Camera interface class. VTrackerParam - Video tracker parameters enum. Described in VTracker interface class. VStabiliserParam - Video stabilizer parameters enum. Described in VStabiliser interface class. VFilterParam - Video filter parameters enum. Described in VFilter interface class. GeneralParam - General pipeline parameters enum. Described in VPipeline.h file. ObjectDetectorParam - Object detector parameters enum. Described in ObjectDetector interface class.
value	Parameter value. Depends on parameter ID and functional module. To check valid parameters values user should check description of appropriate enum.

Example how to encode SET_PARAM command for different video processing pipeline modules:

uint8_t buffer[32];
int size = 0;

// Encode video source parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_SOURCE,
                                 (int)VSourceParam::FOCUS_MODE, 0);

// Encode lens parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::LENS,
                                 (int)LensParam::FOCUS_POS, 1000);

// Encode camera parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::CAMERA,
                                 (int)CameraParam::BRIGHTNESS, 100);

// Encode video tracker parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_TRACKER,
                                 (int)VTrackerParam::NUM_CHANNELS, 3);

// Encode video stabilizer parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_STABILISER,
                                 (int)VStabiliserParam::TYPE, 1);

// Encode first video filter parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_FILTER_1,
                                 (int)VFilterParam::LEVEL, 1.2);

// Encode second video filter parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_FILTER_2,
                                 (int)VFilterParam::LEVEL, 10.5);

// Encode third video filter parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_FILTER_3,
                                 (int)VFilterParam::TYPE, 2);

// Encode fourth video filter parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_FILTER_4,
                                 (int)VFilterParam::LEVEL, 21.6);

// Encode fifth video filter parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::VIDEO_FILTER_5,
                                 (int)VFilterParam::TYPE, 0);

// Encode first object detector parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::OBJECT_DETECTOR_1,
                                 (int)ObjectDetectorParam::SCALE_FACTOR, 2);

// Encode second object detector parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::OBJECT_DETECTOR_2,
                                 (int)ObjectDetectorParam::MIN_X_SPEED, 1.5);

// Encode third object detector parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::OBJECT_DETECTOR_3,
                                 (int)ObjectDetectorParam::MIN_Y_SPEED, 2.5);

// Encode fourth object detector parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::OBJECT_DETECTOR_2,
                                 (int)ObjectDetectorParam::MIN_X_SPEED, 2.5);

// Encode general pipeline parameter.
VPipeline::encodeSetParamCommand(buffer, size,
                                 VPipelineModule::GENERAL,
                                 (int)GeneralParam::LOG_MODE, 1);

encodeCommand static method

The encodeCommand(…) static method encodes action command for remote video processing pipeline. To control a pipeline remotely, the developer has to develop his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VPipeline class contains static methods for encoding the control commands. The VPipeline class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeCommand(…) designed to encode COMMAND command (action command). To encode command user doesn’t need initialize pipeline. Method declaration:

static void encodeCommand(uint8_t* data,
                          int& size,
                          VPipelineModule module,
                          int id,
                          float arg1 = 0.0f,
                          float arg2 = 0.0f,
                          float arg3 = 0.0f);

Parameter	Description
data	Pointer to data buffer for encoded command. Must have size >= 32 bytes.
size	Size of encoded data (command). Will be minimum 10 bytes.
module	Functional module of pipeline according to VPipelineModule enum. Commands available for all modules: VPipelineModule::VIDEO_SOURCE VPipelineModule::LENS VPipelineModule::CAMERA VPipelineModule::VIDEO_TRACKER VPipelineModule::VIDEO_STABILISER VPipelineModule::OBJECT_DETECTOR_1 VPipelineModule::OBJECT_DETECTOR_2 VPipelineModule::OBJECT_DETECTOR_3 VPipelineModule::OBJECT_DETECTOR_4 VPipelineModule::VIDEO_FILTER_1 VPipelineModule::VIDEO_FILTER_2 VPipelineModule::VIDEO_FILTER_3 VPipelineModule::VIDEO_FILTER_4 VPipelineModule::VIDEO_FILTER_5 VPipelineModule::GENERAL
id	Action command ID. Command ID depends on functional module. See example bellow. Appropriate enum must be converted to int value. Encoding COMMAND available for all modules: VSourceCommand - Video source action commands enum. Described in VSource interface class. LensCommand - Lens action commands enum. Described in Lens interface class. CameraCommand - Camera action commands enum. Described in Camera interface class. VTrackerCommand - Video tracker action commands enum. Described in VTracker interface class. VStabiliserCommand - Video stabilizer action commands enum. Described in VStabiliser interface class. ObjectDetectorCommand - Object detector commands enum. Described in ObjectDetector interface class. VFilterCommand - Video filter action commands enum. Described in VFilter interface class. GeneralCommand - General pipeline action commands enum. Described in VPipeline.h file.
arg1	First command argument. Depends on command ID and functional module. To check valid arguments values user should check description of appropriate enum of particular implementation.
arg2	Second command argument. Available only for video tracker module. Value depends on command ID. To check valid arguments values user should check description of appropriate enum of particular video tracker implementation.
arg3	Third command argument. Available only for video tracker module. Value depends on command ID. To check valid arguments values user should check description of appropriate enum of particular video tracker implementation.

Example how to encode COMMAND command for different video processing pipeline modules:

uint8_t buffer[32];
int size = 0;

// Encode video source action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_SOURCE,
                         (int)VSourceCommand::RESTART);
    
// Encode lens action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::LENS,
                         (int)LensCommand::ZOOM_TO_POS, 1000);
    
// Encode camera action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::CAMERA,
                         (int)CameraCommand::FREEZE);
    
// Encode video tracker action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_TRACKER,
                         (int)VTrackerCommand::CAPTURE, 100, 50, -1);
    
// Encode video stabilizer action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_STABILISER,
                         (int)VStabiliserCommand::RESET);

// Encode first video filter action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_FILTER_1,
                         (int)VFilterCommand::ON);

// Encode second video filter action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_FILTER_2,
                         (int)VFilterCommand::OFF);

// Encode third video filter action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_FILTER_3,
                         (int)VFilterCommand::RESET);

// Encode fourth video filter action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_FILTER_4,
                         (int)VFilterCommand::RESET);

// Encode fifth video filter action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::VIDEO_FILTER_5,
                         (int)VFilterCommand::ON);
    
// Encode first object detector action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::OBJECT_DETECTOR_1,
                         (int)ObjectDetectorCommand::ON);
    
// Encode second object detector action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::OBJECT_DETECTOR_2,
                         (int)ObjectDetectorCommand::OFF);
    
// Encode third object detector action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::OBJECT_DETECTOR_3,
                         (int)ObjectDetectorCommand::RESET);

// Encode fourth object detector action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::OBJECT_DETECTOR_4,
                         (int)ObjectDetectorCommand::RESET);

// Encode general pipeline action command.
VPipeline::encodeCommand(buffer, size, VPipelineModule::GENERAL,
                         (int)GeneralCommand::ON);

decodeCommand static method

The decodeCommand(…) static method decodes commands on pipeline side encoded by encodeCommand(…) and encodeSetParamCommand(…) methods. To control a pipeline remotely, the developer has to develop his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VPipeline class contains static methods for encoding the control commands. The VPipeline class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). The decodeCommand(…) method designed to decode both input command types on pipeline side (edge device). This method is usually used if you want to intercept specific control commands. If it is not necessary to intercept specific commands, it is more convenient to use decodeAndExecuteCommand(…) method. Method declaration:

static int decodeCommand(uint8_t* data,
                         int size,
                         VPipelineModule& module,
                         int& id,
                         float& value1,
                         float& value2,
                         float& value3);

Parameter	Description
data	Pointer to input command.
size	Size of input command.
module	The pipeline module for which the command is intended.
id	Action commands ID or parameter ID. It can be different command and parameters structure. The method converts it to int.
value1	Decoded parameters value in case decoding SET_PARAM command or first command argument in case decoding action command (COMMAND).
value2	Second command argument in case decoding action command (COMMAND). Will be available only for VPipelineModule::VIDEO_TRACKER.
value3	Second command argument in case decoding action command (COMMAND). Will be available only for VPipelineModule::VIDEO_TRACKER.

Returns: 0 - in case decoding COMMAND, 1 - in case decoding SET_PARAM command or -1 in case errors.

Example how to decode command:

// Decode command.
VPipelineModule module;
int id;
float value1, value2, value3;
int res = VPipeline::decodeCommand(data, size, module, id, value1, value2, value3);
if (res == 0) // COMMAND
    return pipeline->executeCommand(module, id, value1, value2, value3);
else if (res == 1) // SET_PARAM
    return pipeline->setParam(module, id, value1);
else // ERROR
    return false;

decodeAndExecuteCommand method

The decodeAndExecuteCommand(…) method decodes commands encoded by encodeSetParamCommand(…) and encodeCommand(…) methods using decodeCommand(…) method and calls setParam(…) or executeCommand(…) methods. This method used to simplify remote pipeline control. User just need to put input command data to pipeline. Method declaration:

bool decodeAndExecuteCommand(uint8_t* data, int size);

Parameter	Description
data	Pointer to input command.
size	Size of input command.

Returns: TRUE if command decoded and executed or FALSE if not.

Implementation of the method inside pipeline which explaing principle of the method:

bool VPipeline::decodeAndExecuteCommand(uint8_t* data, int size)
{
    // Decode command.
    VPipelineModule module = VPipelineModule::VIDEO_SOURCE;
    int id = 0; float arg1 = -1.0f; float arg2 = -1.0f; float arg3 = -1.0f;
    int ret = VPipeline::decodeCommand(data, size, module, id, arg1, arg2, arg3);
    
    // Set param or execute command.
    if (ret == 1)
        return VPipeline::setParam(module, id, arg1);
    else if (ret == 0)
        return VPipeline::executeCommand(module, id, arg1, arg2, arg3);

    return false;
}

Enums

This section lists the C++ enums declared in the VPipeline.h file. Other C++ enums used in methods of VPipeline class are declared in libraries: Camera, Lens, ObjectDetector, VSource, VTracker, VFilter and VStabiliser.

VPipelineModule enum

The VPipelineModule enum declared in VPipeline.h file and lists pipeline functional modules. Enum declaration:

enum class VPipelineModule
{
    /// Video source.
    VIDEO_SOURCE = 1,
    /// Lens.
    LENS,
    /// Camera.
    CAMERA,
    /// Video tracker.
    VIDEO_TRACKER,
    /// Video stabilizer.
    VIDEO_STABILISER,
    /// Video filter 1.
    VIDEO_FILTER_1,
    /// Video filter 2.
    VIDEO_FILTER_2,
    /// Video filter 3.
    VIDEO_FILTER_3,
    /// Video filter 4.
    VIDEO_FILTER_4,
    /// Video filter 5.
    VIDEO_FILTER_5,
    /// Object detector 1.
    OBJECT_DETECTOR_1,
    /// Object detector 2.
    OBJECT_DETECTOR_2,
    /// Object detector 3.
    OBJECT_DETECTOR_3,
    /// Object detector 4.
    OBJECT_DETECTOR_4,
    /// General.
    GENERAL
};

Table 2 - VPipelineModule enum values.

Value	Description
VIDEO_SOURCE	Video source module represented by VSource interface class.
LENS	Lens controller module represented by Lens interface class.
CAMERA	Camera controller module represented by Camera interface class.
VIDEO_TRACKER	Video tracker module represented by VTracker interface class.
VIDEO_STABILIZER	Video stabilizer module represented by VStabiliser interface class.
VIDEO_FILTER_1	First video filter module represented by VFilter interface class.
VIDEO_FILTER_2	Second video filter module represented by VFilter interface class.
VIDEO_FILTER_3	Third video filter module represented by VFilter interface class.
VIDEO_FILTER_4	Fourth video filter module represented by VFilter interface class.
VIDEO_FILTER_5	Fifth video filter module represented by VFilter interface class.
OBJECT_DETECTOR_1	First object detector module represented by ObjectDetector interface class.
OBJECT_DETECTOR_2	Second object detector module represented by ObjectDetector interface class.
OBJECT_DETECTOR_3	Third object detector module represented by ObjectDetector interface class.
OBJECT_DETECTOR_4	Fourth object detector module represented by ObjectDetector interface class.
GENERAL	General module (general pipeline params).

GeneralParam enum

The GeneralParam enum declared in VPipeline.h file and lists general pipeline parameters IDs. Enum declaration:

enum class GeneralParam
{
    /// Logging mode. Values: 0 - Disable, 1 - Only file,
    /// 2 - Only terminal (console), 3 - File and terminal (console).
    LOG_MODE = 1,
    /// Mode: 0 - disable, 1 - enable.
    MODE
};

Table 3 - GeneralParam enum values.

Value	Description
LOG_MODE	Logging mode: 0 - Disable (default value). 1 - Only file. 2 - Only terminal (console). 3 - File and terminal (console). The VPipeline library uses Logger library for printing logs into console and(or) file. Note: to enable printing logs in files for all modules user have to initialize it by using static method setSaveLogParams(…) of Logger library. This parameter used only for pipeline and doesn’t set logging mode for particular functional modules. To set logging mode for particular functional modules use appropriate parameters enums.
MODE	Video processing pipeline mode: 0 - disabled, 1 - enabled. If pipeline is disabled, it should not perform any processing. If the pipeline is switched on again, processing must be resumed.

Value

Description

LOG_MODE

Logging mode:
0 - Disable (default value).
1 - Only file.
2 - Only terminal (console).
3 - File and terminal (console).
The VPipeline library uses Logger library for printing logs into console and(or) file. Note: to enable printing logs in files for all modules user have to initialize it by using static method setSaveLogParams(…) of Logger library. This parameter used only for pipeline and doesn’t set logging mode for particular functional modules. To set logging mode for particular functional modules use appropriate parameters enums.

MODE

Video processing pipeline mode: 0 - disabled, 1 - enabled. If pipeline is disabled, it should not perform any processing. If the pipeline is switched on again, processing must be resumed.

GeneralCommand enum

The GeneralParam enum declared in VPipeline.h file and lists general pipeline action commands IDs. Enum declaration:

enum class GeneralCommand
{
    /// Enable video processing pipeline.
    ON = 1,
    /// Disable video processing pipeline.
    OFF
};

Table 4 - GeneralCommand enum values.

Value	Description
ON	Enable pipeline. If the pipeline is already enabled this command will do nothing. If the pipeline is disabled this command will start video processing. Even if the pipeline is disabled use can get video from video source without processing by any functional modules of the pipeline.
OFF	Disable pipeline. If the pipeline is already disabled this command will do nothing. If the pipeline is enabled this command will exclude all functional modules from video processing excepts video source. Even if the pipeline is disabled use can get video from video source without processing by any functional modules of the pipeline.

VPipelineParams class description

VPipelineParams class used for pipeline initialization (initVPipeline(…) method) or to get all actual params (getParams(…) method) including detection and video tracking results. Also VPipelineParams provide structure to write/read params from JSON files (JSON_READABLE macro) and provide methods to encode (serialize) and decode (deserialize) params.

VPipelineParams class declaration

The VPipeline.h file contains VPipelineParams class declaration. Class declaration:

class VPipelineParams
{
public:

    /// General params.
    class GeneralParams
    {
    public:
        /// Logging mode. Values: 0 - Disable, 1 - Only file,
        /// 2 - Only terminal (console), 3 - File and terminal (console).
        int logMode{0};
        /// Main cycle processing time.
        int mainProcessingTimeUs{0};
        /// Enable.
        bool enable{true};

        JSON_READABLE(GeneralParams, logMode, enable)
    };

    /// Video source params.
    cr::video::VSourceParams videoSource;
    /// Video stabilizer params.
    cr::vstab::VStabiliserParams videoStabiliser;
    /// Video tracker params.
    cr::vtracker::VTrackerParams videoTracker;
    /// Object detector 1 params.
    cr::detector::ObjectDetectorParams objectDetector1;
    /// Object detector 2 params.
    cr::detector::ObjectDetectorParams objectDetector2;
    /// Object detector 3 params.
    cr::detector::ObjectDetectorParams objectDetector3;
    /// Object detector 4 params.
    cr::detector::ObjectDetectorParams objectDetector4;
    /// Lens params.
    cr::lens::LensParams lens;
    /// Camera params.
    cr::camera::CameraParams camera;
    /// Video filter 1 params.
    cr::video::VFilterParams videoFilter1;
    /// Video filter 2 params.
    cr::video::VFilterParams videoFilter2;
    /// Video filter 3 params.
    cr::video::VFilterParams videoFilter3;
    /// Video filter 4 params.
    cr::video::VFilterParams videoFilter4;
    /// Video filter 5 params.
    cr::video::VFilterParams videoFilter5;
    /// General params.
    GeneralParams general;

    JSON_READABLE(VPipelineParams, videoSource, videoStabiliser, videoTracker,
                  objectDetector1, objectDetector2, objectDetector3,
                  objectDetector4, lens, camera, videoFilter1, videoFilter2,
                  videoFilter3, videoFilter4, videoFilter5, general)

    /// operator =
    VPipelineParams& operator= (const VPipelineParams& src);

    /// Encode params (serialize).
    bool encode(uint8_t* data, int bufferSize, int& size,
                VPipelineParamsMask* mask = nullptr);

    /// Decode params (deserialize).
    bool decode(uint8_t* data, int dataSize);
};

Table 5 - GeneralParams class fields description.

Field	type	Description
logMode	float	Logging mode: 0 - Disable (default value). 1 - Only file. 2 - Only terminal (console). 3 - File and terminal (console). The VPipeline library uses Logger library for printing logs into console and(or) file. Note: to enable printing logs in files user have to initialize it by using static method setSaveLogParams(…) of Logger library. This parameter used only for pipeline and doesn’t set logging mode for particular functional modules. To set logging mode for particular functional modules use appropriate parameters enums.
enable	bool	Video processing pipeline disable / enable. If pipeline is disabled, it will not perform any processing. Even if the pipeline is disabled use can get video from video source without processing by any functional modules of the pipeline.
mainProcessingTimeUs	int	Main video processing cycle time. Required to control pipeline performance.

Other parameters: VSourceParams - Video source action commands enum. Described in VSource interface class.
LensParams - Lens action commands enum. Described in Lens interface class.
CameraParams - Camera action commands enum. Described in Camera interface class.
VTrackerParams - Video tracker action commands enum. Described in VTracker interface class.
VStabiliserParams - Video stabilizer action commands enum. Described in VStabiliser interface class.
ObjectDetectorParams - Object detector commands enum. Described in ObjectDetector interface class.
VFilterParams - Video filter action commands enum. Described in VFilter interface class.

Serialize params

The VPipelineParams class provides method encode(…) to serialize pipeline params (fields of VPipelineParams class). Serialization of params necessary in case when you need to send params via communication channels. Method provide options to exclude particular parameters from serialization. To do this method inserts binary mask into encoded data where each bit represents particular parameter and decode(…) method recognizes it. Method declaration:

bool encode(uint8_t* data, int bufferSize, int& size,
            VPipelineParamsMask* mask = nullptr);

Parameter	Value
data	Pointer to data buffer.
size	Size of encoded data.
mask	Parameters mask - pointer to VPipelineParamsMask structure. VPipelineParamsMask (declared in VPipeline.h file) determines flags for each field (parameter) declared in VPipelineParams class. If the user wants to exclude a particular parameter from serialization, he should set the corresponding FALSE flag in the VPipelineParamsMask structure and put pointer to method call. VPipelineParamsMask includes additional mask structures: VSourceParamsMask (declared in VSource interface class), VStabiliserParamsMask (declared in VStabiliser interface class), VTrackerParamsMask (declared in VTracker interface class), ObjectDetectorParamsMask (declared in ObjectDetector interface class), LensParamsMask (declared in Lens interface class), VFilterParamsMask (declared in VFilter interface class), CameraParamsMask (declared in Camera interface class) and GeneralParamsMask (declared in VPipeline.h file).

Returns: TRUE if params serialized or FALSE if not (bufferSize not enough for serialized data).

VPipelineParamsMask and additional structures declaration:

typedef struct GeneralParamsMask
{
    bool logMode{ true };
    bool mainProcessingTimeUs{ true };
    bool enable{ true };
} GeneralParamsMask;

typedef struct VPipelineParamsMask
{
    cr::video::VSourceParamsMask videoSource;
    cr::vstab::VStabiliserParamsMask videoStabiliser;
    cr::vtracker::VTrackerParamsMask videoTracker;
    cr::detector::ObjectDetectorParamsMask objectDetector1;
    cr::detector::ObjectDetectorParamsMask objectDetector2;
    cr::detector::ObjectDetectorParamsMask objectDetector3;
    cr::detector::ObjectDetectorParamsMask objectDetector4;
    cr::lens::LensParamsMask lens;
    cr::camera::CameraParamsMask camera;
    cr::video::VFilterParamsMask videoFilter1;
    cr::video::VFilterParamsMask videoFilter2;
    cr::video::VFilterParamsMask videoFilter3;
    cr::video::VFilterParamsMask videoFilter4;
    cr::video::VFilterParamsMask videoFilter5;
    GeneralParamsMask general;
} VPipelineParamsMask;

Parameters serialization example without parameters mask:

// Create params object.
VPipelineParams in;

// Encode params.
uint8_t data[8192];
int size = 0;
if (!in.encode(data, 8192, size))
    cout << "ERROR: Can't encode params" << endl;

cout << "Encoded data size: " << size << " bytes" << endl;

Parameters serialization example with parameters mask:

// Fill params mask.
VPipelineParamsMask mask;
// Exclude some lens params from serialization.
mask.lens.zoomPos = false;
// Exclude some camera params from serialization.
mask.camera.width = false;
mask.camera.height = false;
// Exclude some video source params from serialization.
mask.videoSource.logLevel = false;
mask.videoSource.cycleTimeMks = false;
// Exclude some video stabilizer params from serialization.
mask.videoStabiliser.scaleFactor = false;
mask.videoStabiliser.xOffsetLimit = false;
// Exclude some video tracker params from serialization.
mask.videoTracker.mode = false;
mask.videoTracker.rectX = false;
mask.videoTracker.rectY = false;
// Exclude some first object detector params from serialization.
mask.objectDetector1.logMode = false;
mask.objectDetector1.frameBufferSize = false;
// Exclude some second object detector params from serialization.
mask.objectDetector2.logMode = false;
mask.objectDetector2.frameBufferSize = false;
// Exclude some third object detector params from serialization.
mask.objectDetector3.logMode = false;
mask.objectDetector3.frameBufferSize = false;
// Exclude some others params from serialization.
mask.videoFilter2.enable = false;
mask.videoFilter3.level = false;

// Create params object.
VPipelineParams in;

// Encode params.
uint8_t data[8192];
int size = 0;
if (!in.encode(data, 8192, size, &mask))
    return false;

cout << "Encoded data size: " << size << " bytes" << endl;

Deserialize params

The VPipelineParams class provides method decode(…) to deserialize params (fields of VPipelineParams class). Deserialization of params necessary in case when you need to receive params via communication channels. Method automatically recognizes which parameters were serialized by encode(…) method. Method declaration:

bool decode(uint8_t* data, int dataSize);

Parameter	Value
data	Pointer to data buffer.
dataSize	Size of serialized data.

Returns: TRUE if params deserialized or FALSE if not.

Example:

// Encode params.
VPipelineParams in;
uint8_t data[8192];
int size = 0;
if (!in.encode(data, 8192, size))
    cout << "ERROR: Can't encode params" << endl;

cout << "Encoded data size: " << size << " bytes" << endl;

// Decode params.
VPipelineParams out;
if (!out.decode(data, size))
    cout << "ERROR: Can't decode params" << endl;

Read params from JSON file and write to JSON file

The VPipeline library depends on ConfigReader library which provides method to read params from JSON file and to write params to JSON file. Example of writing and reading params to JSON file:

// Prepare params.
VPipelineParams in;

// Write params to file.
cr::utils::ConfigReader inConfig;
inConfig.set(in, "VPipelineParams");
inConfig.writeToFile("TestVPipelineParams.json");

// Read params from file.
cr::utils::ConfigReader outConfig;
if (!outConfig.readFromFile("TestVPipelineParams.json"))
    cout << "Can't open config file" << endl;

// Parse JSON file to params structure.
VPipelineParams out;
if (!outConfig.get(out, "VPipelineParams"))
    cout << "Can't read params from file" << endl;

TestVPipelineParams.json will look like (part of file):

"VPipelineParams": {
        "camera": {
            "agcMode": 140,
            "alcGate": 51,
            "autoNucIntervalMsec": 234,
            "blackAndWhiteFilterMode": 81,
            "brightness": 87,
            "brightnessMode": 46,
            "changingLevel": 56.0,
            "changingMode": 243,
            "chromaLevel": 173,
            "contrast": 115,
            "custom1": 154.0,
            "custom2": 55.0,
            "custom3": 206.0,
            "ddeLevel": 19.0,
            "ddeMode": 253,
            "defogMode": 166,
            "dehazeMode": 141,
            "detail": 112,
            "digitalZoom": 121.0,
            "digitalZoomMode": 73,
            "displayMode": 128,
            "exposureCompensationMode": 58,
            "exposureCompensationPosition": 118,
            "exposureMode": 149,
            "exposureTime": 5,
            "filterMode": 122,
            "fps": 216.0,
            "gain": 135,
            "gainMode": 96,
            "height": 20,
            "imageFlip": 229,
            "initString": "dfhglsjirhuhjfb",
            "isoSensitivity": 36,
            "logMode": 207,
            "noiseReductionMode": 61,
            "nucMode": 165,
            "palette": 164,
            "profile": 68,
            "roiX0": 134,
            "roiX1": 163,
            "roiY0": 239,
            "roiY1": 197,
            "sceneMode": 72,
            "sensitivity": 114.0,
            "sharpening": 197,
            "sharpeningMode": 242,
            "shutterMode": 215,
            "shutterPos": 131,
            "shutterSpeed": 223,
            "stabilizationMode": 234,
            "type": 165,
            "videoOutput": 161,
            "whiteBalanceArea": 125,
            "whiteBalanceMode": 28,
            "wideDynamicRangeMode": 32,
            "width": 224
        },
        "general": {
            "enable": true,
            "logMode": 180
        },
        "lens": {
            "afHwSpeed": 16,
            "afRange": 8,
            "afRoiMode": 156,
            "afRoiX0": 77,
            "afRoiX1": 71,
            "afRoiY0": 126,
            "afRoiY1": 140,
            "autoAfRoiBorder": 210,
            "autoAfRoiHeight": 213,

...

Build and connect to your project

Typical commands to build VPipeline library:

cd VPipeline
mkdir build
cd build
cmake ..
make

If you want connect VPipeline library to your CMake project as source code you can make follow. For example, if your repository has structure:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp

Create 3rdparty folder in your repository and copy VPipeline repository folder there. New structure of your repository:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    VPipeline

Create CMakeLists.txt file in 3rdparty folder. CMakeLists.txt should contain:

cmake_minimum_required(VERSION 3.13)

################################################################################
## 3RD-PARTY
## dependencies for the project
################################################################################
project(3rdparty LANGUAGES CXX)

################################################################################
## SETTINGS
## basic 3rd-party settings before use
################################################################################
# To inherit the top-level architecture when the project is used as a submodule.
SET(PARENT ${PARENT}_YOUR_PROJECT_3RDPARTY)
# Disable self-overwriting of parameters inside included subdirectories.
SET(${PARENT}_SUBMODULE_CACHE_OVERWRITE OFF CACHE BOOL "" FORCE)

################################################################################
## CONFIGURATION
## 3rd-party submodules configuration
################################################################################
SET(${PARENT}_SUBMODULE_VPIPELINE                       ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VPIPELINE)
    SET(${PARENT}_VPIPELINE                             ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_VPIPELINE_TESTS                       OFF CACHE BOOL "" FORCE)
endif()

################################################################################
## INCLUDING SUBDIRECTORIES
## Adding subdirectories according to the 3rd-party configuration
################################################################################
if (${PARENT}_SUBMODULE_VPIPELINE)
    add_subdirectory(VPipeline)
endif()

File 3rdparty/CMakeLists.txt adds folder VPipeline to your project and excludes test applications from compiling (by default test applications are excluded from compiling if VPipeline is included as sub-repository). The new structure of your repository:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    CMakeLists.txt
    VPipeline

Next you need include folder 3rdparty in main CMakeLists.txt file of your repository. Add string at the end of your main CMakeLists.txt:

add_subdirectory(3rdparty)

Next you have to include VPipeline library in your src/CMakeLists.txt file:

target_link_libraries(${PROJECT_NAME} VPipeline)

Done!

Example

This example shows how to create VPipeline object with all “dummy” functional modules, obtain video frames and control pipeline. “Dummy” video source module will generate artificial video. Example uses OpenCV to display video.

#include <iostream>
#include <opencv2/opencv.hpp>
#include "VPipeline.h"

int main(void)
{
    std::cout << "VPipeline v" <<
    cr::pipeline::VPipeline::getVersion() << std::endl;

    // Prepare some pipeline params.
    cr::pipeline::VPipelineParams params;
    params.videoSource.width = 1280;
    params.videoSource.height = 720;
    params.videoSource.fps = 30;
    params.general.logMode = 2;
    params.videoSource.logLevel = 2;
    params.objectDetector1.logMode = 2;
    params.lens.logMode = 2;
    params.camera.logMode = 2;

    // Create and init pipeline.
    cr::pipeline::VPipeline pipeline;
    if (!pipeline.initVPipeline(params))
        return -1;

    // Input video frame.
    cr::video::Frame srcFrame;
    cv::Mat bgrFrame;

    // Main loop.
    while (true)
    {
        // Wait YUV24 video frame from pipeline 1 sec.
        if (!pipeline.getFrame(srcFrame, 1000))
            continue;

        // Convert YUV24 to BGR24 to display.
        cv::Mat yuvFrame(srcFrame.height, srcFrame.width, CV_8UC3, srcFrame.data);
        cv::cvtColor(yuvFrame, bgrFrame, cv::COLOR_YUV2BGR);

        // Get pipeline params.
        pipeline.getParams(params);

        // Print processing time.
        cv::putText(bgrFrame, "MAIN PROCESSING TIME " +
        std::to_string(params.general.mainProcessingTimeUs) + " msec",
        cv::Point(10, 30), cv::FONT_HERSHEY_SIMPLEX, 0.7,
        cv::Scalar(0, 0, 255), 1, cv::LINE_AA);

        // Display BGR frame.
        cv::imshow("VPipeline", bgrFrame);

        // Process keyboard events.
        switch (cv::waitKey(24)) {
        case 27: // ESC - exit.
            exit(0);
        break;
        case 32: // SPACE - Enable/disable pipeline.
            pipeline.executeCommand(
            cr::pipeline::VPipelineModule::GENERAL,
            params.general.enable ? (int)cr::pipeline::GeneralCommand::OFF :
            (int)cr::pipeline::GeneralCommand::ON);
        break;
        }
    }
    return 1;
}