vpipeline_control_web_logo

VPipelineControl application

An example of controlling VPipeline C++ library v1.0.2

Table of contents

Overview

The VPipelineControl is an example application, that shows how to control all parameters from VPipeline, which instance is run onboard. The example works only for Intel-based platforms. The repository includes the source code of the application itself and source code of all the libraries required for it to work. The main purpose of the application is to demonstrate how different modules of VPipeline can be controlled. In this particular application XBOX Controller was used to change parameters of modules, select rectangle for tracking purposes or control connected Pan-Tilt unit. The application (repository) is a CMake project and can be used as template for your project. It provides UDPDataChannel initialization, basic user interface based on OpenCV, and control of all video processing pipeline modules (camera, lens, filters, video tracker, detectors) alongside with Pan-Tilt unit. If correct data is sent through the UDP Port, the user can control parameters of: ViscaCamera (as Camera and Lens modules), ImageFlip (as first video filter), DigitalZoom (as second video filter), VideoStabiliserOpenCv (as video stabilization module), Dehazer (as fourth video filter), MotionMagnificator (as fifth video filter), CvTracker (as video tracker module), Gmd (motion detector library as first object detector), Ged (video changes detector library as second object detector) and DnnOpenVinoDetector (neural network object detector library as third objects detector). There is also used VCodecOneVpl library used for deserializing frame that comes from UDP Port. The application utilizes C++17 standard.

NOTE: Since VCodecOneVpl library is compatible only with Intel processors, VPipelineOnControl example can be used only on Intel hardware. Check [How to create your own example depending on platform](#how-to-create-your-own-example-depending-on-platform) chapter to see how to create VPipeline controller on different platforms.

After initialization application will run internal processing loop which reads data from UDP Port, updates the telemetry data, deserialize frame, checks what buttons were clicked on the controller and sends commands related to them back through UDP Port.

Application versions

Table 1 - Application versions.

Version Release date What’s new
1.0.0 06.06.2024 First version.
1.0.1 06.08.2024 - Submodules updated.
1.0.2 05.10.2024 - Update VPipeline submodule.

Application files

The VPipelineControl is provided as source code. The user receives a set of files in the form of a CMake project (repository). The repository structure is outlined below:

CMakeLists.txt ----------- Main CMake file of the application.
3rdparty ----------------- Folder with third-party libraries.
    CMakeLists.txt ------- CMake file which includes third-party libraries.
    Joystick ------------- Folder with Joystick library source code.
    PanTilt -------------- Folder with PanTilt library source code.
    UdpDataChannel ------- Folder with UdpDataChannel library source code.
    VCodecOneVpl --------- Folder with VCodecOneVpl library source code.
    VPipeline ------------ Folder with VPipeline library source code.
src ---------------------- Folder with source code of the library.
    CMakeLists.txt ------- CMake file of the application.
    main.cpp ------------- Source code file of the application.

Build application

The VPipelineControl is a complete repository in form of the CMake project. The application depends only on OpenCV to provide visualization of incoming frames and OneVpl to desarialize them. Before compiling you have to install those libraries into your OS. Also to compile library you have to install CMake.

On Linux

Below are the steps to configure on Linux operating system:

At first there has to be installed OneVpl codec for Intel platforms for obtaining VCodecOneVpl usage. To install OneVpl follow instruction: OneVpl for Linux.

sudo apt-get install cmake libopencv-dev

Typical commands to build VPipelineControl application (Release mode):

cd VPipelineControl
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make

On Windows

Below are the steps to configure on Windows operating system:

  1. At first there has to be installed OneVpl codec for Intel platforms for obtaining VCodecOneVpl usage. To install OneVpl follow instruction: OneVpl for Windows.
  2. Install CMake: Open the link, download installer and follow instructions: CMake install. You can use different version if needed, all above 3.15 are supported.
  3. Install OpenCV: Open the link, download installer and follow instructions: OpenCV 4.9. Install to “C:/OpenCV”. You can use different version if needed, all above 4.5 are supported. Locate OpenCVConfig.cmake file add to cmake command OpenCV_DIR that points to the location of this file, shown below.
  4. Commands to build VPipelineControl application (Release mode):
cd VPipelineOnboard
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -D OpenCV_DIR="C:/OpenCV/opencv/build"
cmake --build . --config Release

How to create your own example depending on platform

The VPipelineControl is an example how to control VPipeline instance on your platform, but it utilizes VCodecOneVpl library, which is suited only for Intel-based platforms. If you want to create your own VPipelineControl example you can copy whole structure, but you have to replace codec used for transcoding frame. Table 2 demonstrates what codec is recommended for different platforms:

Table 2 - What codec to use depending on platform.

Device Codec to use
Raspberry Pi 4
Raspberry Pi 2 zero		 VCodecV4L2
Raspberry Pi 5 VCodecLibav
NVidia Jetson devices VCodecJetPack

Launch

After compiling you will get VPipelineControl.exe executable file on Windows or VPipelineControl executable file on Linux. To run application on Linux run command:

./VPipelineControl

Note: on Windows you may need copy OpenCV .dll files to application’s executable file folder.

Control

To control the application, it is necessary that the main video display window was active (in focus). The program is controlled by the XBOX Controller.

Below you can see the scheme of XBOX Controller and all buttons described in the Table 2.

xbox_scheme

Table 2 - Buttons to control VPipeline instance.

Button name Usage
X Move to the previous module menu.
B Move to the next module menu.
Y Move up in selected module menu.
A Move down in selected module menu.
RB Increase selected parameter value. If selected parameter utilizes only ON/OFF values, this button sets the value to ON.
LB Decrease selected parameter value. If selected parameter utilizes only ON/OFF values, this button sets the value to OFF.
RSB move Moving RSB joystick moves tracking rectangle on the screen. If tracking mode is ON (tracking is performed) the movement will be more sensitive. If tracking mode is OFF (tracker module waiting for object to track) the movement will be much faster.
RSB click Clicking the RSB joystick down sends command “Capture” to tracking module and starts tracking object in rectangle selected by moving the joystick.
LSB move Moving RSB joystick moves sends the command to the Pan-Tilt unit to move in given direction. If tracking mode is ON (tracking is performed) the Pan-Tilt unit will not move.
LSB click Clicking the RSB joystick down sends command “Reset” to tracking module and stop tracking.
D-PAD The usage of D-PAD (HAT) button is related to changing the size of tracking rectangle.
TOP side of the button - Increase height of tracking rectangle.
BOTTOM side of the button - Decrease height of tracking rectangle
RIGHT side of the button - Increase width of tracking rectangle.
LEFT side of the button - Decrease width of tracking rectangle.

Prepare compiled library files

If you want to compile and collect all libraries from VPipelineControl repository with their header files on Linux you can create bach script in repository root folder which will collect all necessary files in one place. To do this compile VPipelineControl and after make follow steps:

  1. Create bash script (your have to have installed nano editor):
cd VPipelineControl
nano prepareCompiled
  1. Copy next text there:
#!/bin/bash

# Define the directory where you want to copy all .h files.
# Make sure to replace /path/to/destination with your actual destination directory path.
HEADERS_DESTINATION_DIR="./include"
LIB_DESTINATION_DIR="./lib"

# Check if the destination directory exists. If not, create it.
if [ ! -d "$HEADERS_DESTINATION_DIR" ]; then
    mkdir -p "$HEADERS_DESTINATION_DIR"
fi
# Find and copy all .h files from the current directory to the destination directory.
# The "." specifies the current directory. Adjust it if you want to run the script from a different location.
find . -type f -name '*.h' -exec cp {} "$HEADERS_DESTINATION_DIR" \;


found_dir=$(find . -type d -name "nlohmann" -print -quit)
if [ -n "$found_dir" ]; then
    cp -r "$found_dir" "$HEADERS_DESTINATION_DIR"
    echo "Directory nlohmann has been copied to $HEADERS_DESTINATION_DIR."
fi

# Check if the destination directory exists. If not, create it.
if [ ! -d "$LIB_DESTINATION_DIR" ]; then
    mkdir -p "$LIB_DESTINATION_DIR"
fi

# Find and copy all .a files from the current directory to the destination directory.
# The "." specifies the current directory. Adjust it if you want to run the script from a different location.
find . -type f -name '*.a' -exec cp {} "$LIB_DESTINATION_DIR" \;
  1. Save “Crtr + S” and close “Ctrl + X”.
  2. Copy make file executable and run:
sudo chmod +x prepareCompiled
./prepareCompiled