How to integrate RapidPixel SDK into your project

The RapidPixel SDK is a set of cross-platform C++ libraries that allows creating real-time video processing applications with proven performance. In order to build all libraries correctly, including those dependent on external libraries, start by installing the External dependencies. If the required external dependencies are not installed, only the libraries that do not rely on these dependencies will be built.

• How to build the SDK

Then all you need to do is connecting RapidPixel SDK to your project, depending on your configuration:

• Connect using CMake-based project

• Connect using Visual Studio project

• Connect using Qt Creator project

External dependencies

Some of the libraries in RapidPixel SDK are dependent on few external libraries. If the external library (that is necessary for proper usage of chosen repository) is not installed, relating repository of RapidPixel SDK will not be built. The detailed list of libraries and their dependencies can be found here RapidPixel. The Table 1 below briefly presents external libraries and RapidPixel libraries that are dependent on them:

Table 1 - External dependencies.

External dependency name	Dependent libraries
OpenCv	VPipeline VStabiliserOpenCv DnnOpenVinoDetector DnnOpenCvDetector RtspServer RtspServerLive555 VSourceOpenCv Dehazer DigitalZoom ImageFlip FormatConverterOpenCv PlotOpenCv VideoRecorderOpenCv Denoiser
OpenVINO	DnnOpenVinoDetector
FFmpeg and Libav	VCodecLibav
OpenH264	VCodecOpenH264
Libcamera	VSourceLibCamera
OneVpl	VCodecOneVpl
Live555 and OpenSSL	RtspServerLive555
JetPack	VCodecJetPack

OpenCv

• On Windows

  Only versions >= 4.5.0 are supported. Download OpenCV installer from official website, for example LINK. Run opencv-version-windows.exe file and extract library to your desired directory, for example C:/Libs/OpenCV.

• On Linux On linux it can be installed either from source code or from apt package manager. If you are installing by using package manager, ensure that you have proper version of OpenCV. Command to install:

  sudo apt-get install -y libopencv-dev
  

OpenVINO

Current tested and working perfectly with DnnOpenVinoDetector library OpenVino runtime library version is 2023.3.0.

• On Windows Download OpenVINO™ runtime library for Windows. Install runtime library according to instruction: Instruction for Windows.

• On Linux

  Download OpenVINO™ runtime library for Linux. Install runtime library according to instruction: Instruction for Linux.

FFmpeg and Libav

• Only on Linux

  sudo apt-get install -y build-essential cmake ffmpeg libavcodec-dev libavutil-dev libavformat-dev libavdevice-dev libavfilter-dev libcurl4
  

OpenH264

• Only on Linux

  sudo apt-get -y install nasm
  git clone https://github.com/cisco/openh264.git
  cd openh264
  make OS=linux ARCH=x86_64
  sudo make install
  sudo ln -s /usr/local/lib/libopenh264.so.7 /usr/lib/libopenh264.so.7
  

Libcamera

• Only on Linux

  sudo apt-get install meson python3-pip
  pip3 install ply jinja2
  git clone https://git.libcamera.org/libcamera/libcamera.git
  cd libcamera
  meson setup build
  sudo ninja -C build install
  

OneVpl

• On Windows Use command-line interface and type following commands:

  git clone https://github.com/oneapi-src/oneVPL.git
  cd oneVPL
  script/bootstrap.bat
  script/build.bat
  script/install.bat
  

• On Linux Use command-line interface and type following commands:

• Install LibVA:

  sudo apt-get install git cmake pkg-config meson libdrm-dev automake libtool
  cd Downloads
  git clone https://github.com/intel/libva.git
  cd libva
  ./autogen.sh --prefix=/usr --libdir=/usr/lib/x86_64-linux-gnu
  make
  sudo make install
  

• Install gmmlib:

  cd Downloads
  git clone https://github.com/intel/gmmlib.git
  cd gmmlib
  mkdir build
  cd build
  cmake -DCMAKE_BUILD_TYPE=Release ..
  make -j"$(nproc)"
  sudo make install
  

• Install intel media driver for VAAPI:

  cd Downloads
  git clone https://github.com/intel/media-driver.git
  mkdir build_media && cd build_media
  cmake ../media-driver
  make -j"$(nproc)"
  sudo make install
  

• Install oneVPL-intel-gpu:

  cd Downloads
  git clone https://github.com/oneapi-src/oneVPL-intel-gpu onevpl-gpu
  cd onevpl-gpu
  mkdir build && cd build
  cmake ..
  make -j"$(nproc)"
  sudo make install
  

• Install additional packets to oneVPL-intel-gpu:

  sudo apt update
  sudo apt install -y gpg-agent wget
  wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | sudo gpg --dearmor --output /usr/share/keyrings/intel-graphics.gpg
  echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy/production/2328 unified" | sudo tee /etc/apt/sources.list.d/intel-gpu-jammy.list
  sudo apt update
  sudo apt install -y linux-headers-$(uname -r) flex bison intel-fw-gpu intel-i915-dkms xpu-smi
  sudo reboot 
  sudo apt install -y intel-opencl-icd intel-level-zero-gpu level-zero intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri libglapi-mesa libgles2-mesa-dev libglx-mesa0 libigdgmm12 libxatracker2 mesa-va-drivers mesa-vdpau-drivers mesa-vulkan-drivers va-driver-all vainfo hwinfo clinfo
  sudo apt install -y libigc-dev intel-igc-cm libigdfcl-dev libigfxcmrt-dev level-zero-dev
  

• Install oneVPL:

  cd Downloads
  git clone https://github.com/oneapi-src/oneVPL.git
  cd oneVPL
  sudo script/bootstrap
  script/build
  sudo script/install
  

Live555 and OpenSSL

Necessary in order to build RtspServerLive555.

• On Linux:

  How to install OpenSSL

  Run command :

    sudo apt-get install libssl-dev
  

  How to install Live555 using package manager

  • Run command :

  sudo apt-get install liblivemedia-dev
  

  If you get error from package manager, go to THIS step and install live555 from source code.

  Important note : There will be a compilation error if your system has live555 from source code installation and package manager installation at the same time. In order to solve problem you should remove one of them. For example:

  sudo apt-get remove liblivemedia-dev
  

JetPack

In general, when installing JetPack on your Jetson platform via Nvidia SDK Manager, there should be a jetson_multimedia_api folder in the /usr/src directory. If it is missing, you need to copy the jetson_multimedia_api folder from the /_static/jetson_multimedia_api.zip to the /usr/src directory on your Jetson.

How to build the SDK

1. Download the SDK package.

2. Unpacked folder should look like this:

   RapidPixelSdk
   	> example
   	> modules
   	> RapidPixel
   		CMakeLists.txt
   	GettingStarted.md
   	RapidPixelConfig.cmake
   	CMakeLists.txt
   

3. Open terminal inside RapidPixelSdk folder and run following commands:

   For Windows:

   # Configure CMake project with specified build type (Debug/Release).
   cmake -B build
   # Build all libraries with proper target and specified build type (Debug/Release).
   cmake --build build --config Release --target ALL_BUILD
   

   For Linux:

   # Configure CMake project with specified build type (Debug/Release).
   cmake -B build -D CMAKE_BUILD_TYPE=Release
   # Build all libraries with proper target and specified build type (Debug/Release).
   cmake --build build --config Release  --target all
   

4. Review your new RapidPixelSdk structure:

   > RapidPixelSdk
   	> build
   	> docs
   	> example
   	> include
   	> lib
       > modules
       > RapidPixel
       	CMakeLists.txt
   		RapidPixel.h
   	GettingStarted.md
   	RapidPixelConfig.cmake
   	CMakeLists.txt
   

5. Now everything is set up to start using RapidPixel SDK in your project. Check Example application here to see example of usage.

Connect using CMake-based project

1. Go to your CMake-based project directory and modify CMakeLists.txt file to include RapidPixel library.

   # Link the RapidPixel libraries
   find_package(RapidPixel REQUIRED)
   target_link_libraries(${PROJECT_NAME} RapidPixel)
   

2. Configure your project with defining RapidPixel_DIR macro:

   mkdir build
   cd build
   cmake .. -D RapidPixel_DIR=C:\Libs\Windows_RP_INCLUDE_PACK\RapidPixel
   

Connect using Visual Studio project

1. Open your project, for instance YourProject.sln.

2. Rightclick on YourProject in Solution Explorer and hit Properties.

3. In Properties go to VC++ Directories -> General -> Include Directories -> Edit and add RapidPixel include directory:

   C:\Libs\Windows_RP_INCLUDE_PACK\RapidPixel\include
   

4. In Properties go to VC++ Directories -> General -> Library Directories -> Edit and add RapidPixel lib directory:

   Debug configuration:
   C:\Libs\Windows_RP_INCLUDE_PACK\RapidPixel\lib\Debug
   Release configuration:
   C:\Libs\Windows_RP_INCLUDE_PACK\RapidPixel\lib\Release
   

5. In Properties go to Linker -> Input -> Additional Dependencies -> Edit and add RapidPixel.lib or RapidPixeld.lib directory, depending on your current project configuration:

   Debug configuration:
   RapidPixeld.lib
   Release configuration:
   RapidPixel.lib
   

6. Remember to apply C++ 17 standard: General -> C++ Language Standard -> ISO C++17 Standard (/std:c++17).

7. Hit Apply button and save close Properties page.

OpenCV

In order to use RapidPixel libraries, that are dependent on OpenCV library (for example CvTracker), it is needed to also add OpenCV to project configuration. Remember to adjust your paths and lib names according to currently in use OpenCV version.

1. In Properties go to VC++ Directories -> General -> Include Directories -> Edit and add OpenCV include directory:

   C:\Libs\OpenCV\opencv\build\include
   

2. In Properties go to VC++ Directories -> General -> Library Directories -> Edit and add OpenCV lib directory:

   C:\Libs\OpenCV\opencv\build\x64\vc16\lib
   

3. In Properties go to Linker -> Input -> Additional Dependencies -> Edit and add lib files, depending on your current project configuration:

   Debug configuration:
   opencv_world480d.lib
   Release configuration:
   opencv_world480.lib
   

OpenVino

In order to use RapidPixel libraries, that are dependent on OpenVino library (for example DnnOpeVinoDetector), it is needed to also add OpenVino to project configuration. Remember to adjust your paths and lib names according to currently in use OpenVino version.

1. In Properties go to VC++ Directories -> General -> Include Directories -> Edit and add OpenVino include directory:

   C:\Program Files (x86)\Intel\openvino_2022\runtime\include
   

2. In Properties go to VC++ Directories -> General -> Library Directories -> Edit and add OpenVino lib directory:

   Debug configuration:
   C:\Program Files (x86)\Intel\openvino_2022\runtime\lib\intel64\Debug
   Release configuration:
   C:\Program Files (x86)\Intel\openvino_2022\runtime\lib\intel64\Release
      
   

3. In Properties go to Linker -> Input -> Additional Dependencies -> Edit and add lib files, depending on your current project configuration:

   Debug configuration:
   openvinod.lib
   Release configuration:
   openvino.lib
   

Connect using Qt Creator project

1. Open your qmake project, for instance YourProject.

2. To include headers, edit YourProject.pro file with:

   INCLUDEPATH += C:\Libs\Windows_RP_INCLUDE_PACK\RapidPixel\include
   

3. Specify lib files:

   Debug
   {
       LIBS += C:\Libs\Windows_RP_INCLUDE_PACK\RapidPixel\lib\Debug\RapidPixeld.lib 
   }
      
   Release
   {
       LIBS += C:\Libs\Windows_RP_INCLUDE_PACK\RapidPixel\lib\Release\RapidPixel.lib 
   }
   

4. Remember to keep C++17 language standard:

   CONFIG += c++17
   

5. Save YourProject.pro file, it is now ready to use RapidPixel SDK.

OpenCV

In order to use RapidPixel libraries, that are dependent on OpenCV library (for example CvTracker), it is needed to also add OpenCV to project configuration. Remember to adjust your paths and lib names according to currently in use OpenCV version.

1. To include headers, edit YourProject.pro file with:

   INCLUDEPATH += C:\Libs\OpenCV\opencv\build\include
   

2. Specify lib files:

   Debug
   {
       LIBS += C:\Libs\OpenCV\opencv\build\x64\vc16\lib\opencv_world480d.lib
   }
      
   Release
   {
       LIBS += C:\Libs\OpenCV\opencv\build\x64\vc16\lib\opencv_world480.lib
   }
   

OpenVino

In order to use RapidPixel libraries, that are dependent on OpenVino library (for example DnnOpeVinoDetector), it is needed to also add OpenVino to project configuration. Remember to adjust your paths and lib names according to currently in use OpenVino version.

1. To include headers, edit YourProject.pro file with:

   INCLUDEPATH += C:\Program Files (x86)\Intel\openvino_2022\runtime\include
   

2. Specify lib files:

   Debug
   {
       LIBS += C:\Program Files (x86)\Intel\openvino_2022\runtime\lib\intel64\Debug\openvinod.lib
   }
      
   Release
   {
       LIBS += C:\Program Files (x86)\Intel\openvino_2022\runtime\lib\intel64\Release\openvino.lib
   }
   

Example application

The example application provides brief showcase of how to include RapidPixel SDK headers into your project and use all utilities. The example is a CMake-based project with following CMakeLists.txt file:

cmake_minimum_required(VERSION 3.15)

project(RapidPixelExample VERSION 1.0.0 LANGUAGES CXX)

set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# create glob files for *.h, *.cpp
file (GLOB H_FILES   ${CMAKE_CURRENT_SOURCE_DIR}/*.h)
file (GLOB CPP_FILES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp)
# concatenate the results (glob files) to variable
set  (SOURCES ${CPP_FILES} ${H_FILES})
# create executable from src
if (NOT TARGET ${PROJECT_NAME})
    add_executable(${PROJECT_NAME} ${SOURCES})
endif()

# Link the RapidPixel libraries
find_package(RapidPixel REQUIRED)
target_link_libraries(${PROJECT_NAME} RapidPixel)

While including RapidPixel SDK headers remember to add RapidPixel/ before library name, use the following structure:

#include <RapidPixel/LibraryNameHeader.h>

Alternatively, if it’s more convenient, you can include all headers at once using:

#include <RapidPixel/RapidPixel.h>

NOTE: Remember! To use particular library all the dependencies has to be included in your project. For instance, in order to use library that is openCv dependent you have add the following code to your project:

find_package(OpenCV)
target_link_libraries(${PROJECT_NAME} ${OpenCV_LIBS})

Source code of the example:

#include <iostream>

#include <RapidPixel/RapidPixel.h>


int main()
{
	std::cout << "======================================" << std::endl;
	std::cout << "========= RapidPixel example =========" << std::endl;
	std::cout << "======================================" << std::endl;

	std::cout << "VSourceOpenCv version: " << cr::video::VSourceOpenCv::getVersion() << std::endl;
	std::cout << "VTracker version: " << cr::vtracker::VTracker::getVersion() << std::endl;
	std::cout << "DnnOpenCvDetector version: " << cr::detector::DnnOpenCvDetector::getVersion() << std::endl;

	return 0;
}

The example application is configured with the appropriate command, which sets up the necessary paths for RapidPixel SDK and external dependencies such as OpenCV and OpenVino. The configuration will depend on the specific libraries of the SDK that the user wants to use. The command to configure the project is:

cmake -B build -D RapidPixel_DIR=path_to_RapidPixelSdk_folder -D OpenCV_DIR=path_to_opencv_folder

Adjust the paths according to your installations.