VCodecLibav C++ library

v1.3.2

Table of contents

• Overview
• Versions
• Library files
• VCodecLibav class description
  • VCodecLibav class declaration
  • getVersion method
  • transcode method
  • setParam method
  • getParam method
  • executeCommand method
• Build and connect to your project
  • Installation on Linux
  • Installation on Windows
  • Build and connect to project
• Simple example
• Test application

Overview

VCodecLibav C++ library provides video encoding and decoding functions for H264, HEVC(H265) and JPEG codecs for Linux and Windows OS based on FFmpeg. The library supports software encoders / decoders and Intel hardware encoders / decoders. The library compatible with any CPU (software encoder) which supports FFmpeg. VCodecLibav video codec class inherits interface and data structures form VCodec interface library. The library depends on open source VCodec library (provides codec interface, source code included, Apache 2.0 license), open source Libav library (part of FFmpeg, linked) and open source Logger (provides functions to print logs, source code included, Apache 2.0). the library uses C++17 standard. Used FFmpeg codecs:

Codec	ffmpeg encoder	ffmpeg decoder
H264	h264_vaapi (Intel VAAPI based codec) for hardware encoding on Linux and h264_qsv for hardware encoding on Windows. For software encoding used libx264 for both Linux and Windows.	h264_qsv (Intel QuickSync based) for hardware decoding or libx264 for software decoding for both Linux and Windows.
HEVC (H265)	hevc_vaapi (Intel VAAPI based codec) for hardware encoding on Linux and hevc_qsv for hardware encoding on Windows. For software encoding used libx265 for both Linux and Windows.	hevc_qsv (Intel QuickSync based) for hardware decoding or libx265 for software decoding for both Linux and Windows.
JPEG	mjpeg_vaapi (Intel VAAPI based codec) for hardware encoding on Linux and mjpeg_qsv for hardware encoding on Windows. For software encoding used mjpeg for both Linux and Windows. Warning: mjpeg software codec produces not fully compatible JPEG (doesn’t have DHT like JPEG does).	mjpeg_qsv (Intel QuickSync based) for hardware decoding and mjpeg for software decoding.

The primary difference between software (SW) and hardware (HW) encoders and decoders is the latency they introduce. SW encoders and decoders have zero frame latency, but they take more time than HW encoders and decoders. On the other hand, HW encoders have 1 frame latency, and HW decoders have 3 frames latency for H264 codecs. Encoding/decoding time on 11th Gen Intel(R) Core(TM) i5-1145G7E @ 2.60GHz, Ubuntu 22.10, msec:

Hardware codec / resolution	1920x1080	1280x720	640x512
H264	5 msec / 3 msec	2 msec / 1.6 msec	0.6 msec / 0.5 msec
H265	5 msec / 3 msec	3 msec / 1.2 msec	2 msec / 0.6 msec
JPEG	2 msec / 4 msec	1 msec / 1.5 msec	0.3 msec / 0.6 msec
Software codec (1 thread) / resolution	1920x1080	1280x720	640x512
H264	8 msec / 3 msec	3.5 msec / 1.5 msec	1.8 msec / 0.9 msec
H265	35 msec / 10 msec	25 msec / 8 msec	15 msec / 7 msec
JPEG	4 msec / 3 msec	1.8 msec / 1.4 msec	0.9 msec / 0.7 msec

Versions

Table 1 - Library versions.

Version	Release date	What’s new
1.0.0	29.06.2023	First version.
1.0.1	02.07.2023	- Source code cleaned up. - Documentation updated.
1.0.2	16.11.2023	- VCodec class updated.
1.0.3	20.12.2023	- Add specific encoder settings for noisy video. - Example added.
1.1.0	17.01.2024	- Add software encoder\decoder support for JPEG, H264 and HEVC. - Examples updated.
1.1.1	11.03.2024	- Added Windows support. - Documentation updated.
1.1.2	26.04.2024	- Submodules updated. - Documentation updated.
1.1.3	17.05.2024	- Submodules updated. - Documentation updated.
1.2.0	16.09.2024	- Add data action command RESET and MAKE_KEY_FRAME.
1.2.1	10.10.2024	- Fix resolution align issue for SW codec.
1.2.2	21.10.2024	- Fix constant bitrate encoding settings for software encoder.
1.3.0	01.11.2024	- Fix settings for CBR and VBR mode for hardware and software encoders. - Update VCodec interface with variable bitrate params. - Add variable bitrate params.
1.3.1	06.11.2024	- Fix device name initialization in test application.
1.3.2	19.11.2024	- Add x264 speed preset control (ultrafast or superfast).

Library files

The library supplied by source code only. The user would be given a set of files in the form of a CMake project (repository). The repository structure is shown below:

CMakeLists.txt --------------- Main CMake file of the library.
3rdparty --------------------- Folder with third-party libraries.
    CMakeLists.txt ----------- CMake file to include third-party libraries.
    VCodec ------------------- Folder with VCodec interface library files.
    Logger ------------------- Folder with Logger library files.
src -------------------------- Folder with library source code.
    CMakeLists.txt ----------- CMake file of the library.
    VCodecLibav.h ------------ Main library header file.
    VCodecLibavVersion.h ----- Header file with library version.
    VCodecLibavVersion.h.in -- File for CMake to generate version header.
    VCodecLibav.cpp ---------- C++ implementation file.
test ------------------------- Folder for test application.
    CMakeLists.txt ----------- CMake file for test application.
    main.cpp ----------------- Source C++ file of test application.
example ---------------------- Folder for simple example.
    CMakeLists.txt ----------- CMake file of example.
    main.cpp ----------------- Source C++ file of example.

VCodecLibav class description

VCodecLibav class declaration

VCodecLibav class declared in VCodecLibav.h file. Class declaration:

namespace cr
{
namespace video
{
class VCodecLibav : public VCodec
{
public:

    /// Get library version.
    static std::string getVersion();

    /// Class constructor.
    VCodecLibav();

    /// Class destructor.
    ~VCodecLibav();

    /// Encode video frame.
    bool transcode(Frame& src, Frame& dst);

    /// Set parameter value.
    bool setParam(VCodecParam id, float value);

    /// Get parameter value.
    float getParam(VCodecParam id);

    /// Execute command.
    bool executeCommand(VCodecCommand id);
};
}
}

getVersion method

The getVersion() method returns string of current version of VCodecLibav class. Method declaration:

static std::string getVersion();

Method can be used without VCodecLibav class instance:

std::cout << "VCodecLibav class version: " << VCodecLibav::getVersion() << std::endl;

Console output:

VCodecLibav class version: 1.3.0

transcode method

The transcode(…) method intended to encode and decode video frame (Frame class). Video codec encodes / decodes video frames frame-by-frame. Method declaration:

bool transcode(Frame& src, Frame& dst);

Parameter	Value
src	Source video frame (see Frame class description). To encode video data src frame must have NV12 pixel format. To decode video data src frame must have compressed pixel format (field fourcc of Frame class): JPEG, H264 or HEVC.
dst	Result video frame (see Frame class description). To decode video data src frame must have compressed pixel format (field fourcc of Frame class): JPEG, H264 or HEVC. User must specify output pixel format in advance. In case decoding video codec will set NV12 pixel format automatically.

Returns: TRUE if frame was encoded/decoded or FALSE if not.

setParam method

The setParam(…) method designed to set new video codec parameters value. Method declaration:

bool setParam(VCodecParam id, float value);

Parameter	Description
id	Video codec parameter ID according to VCodecParam enum.
value	Video codec parameter value. Valid values depends on parameter ID.

Returns: TRUE is the parameter is set or FALSE if not.

VCodec.h file of VCodec library defines IDs for parameters (VCodecParam enum) and IDs for commands (VCodecCommand enum). VCodecParam declaration:

namespace cr
{
namespace video
{
enum class VCodecParam
{
    /// [read/write] Log level: 0-Disable, 1-Console, 2-File, 3-Console and file.
    LOG_LEVEL = 1,
    /// [read/write] Bitrate, kbps. For H264 and H265 codecs.
    BITRATE_KBPS,
    /// [read/write] Minimum bitrate, kbps. For variable bitrate mode.
    MIN_BITRATE_KBPS,
    /// [read/write] Maximum bitrate, kbps. For variable bitrate mode.
    MAX_BITRATE_KBPS,
    /// [read/write] Bitrate mode: 0 - constant bitrate, 1 - variable bitrate.
    BITRATE_MODE,
    /// [read/write] Quality 0-100%. For JPEG codecs.
    QUALITY,
    /// [read/write] FPS. For H264 and H265 codecs.
    FPS,
    /// [read/write] GOP size. For H264 and H265 codecs.
    GOP,
    /// [read/write] H264 profile: 0 - Baseline, 1 - Main, 2 - High.
    H264_PROFILE,
    /// [read/write] Codec type. Depends on implementation.
    TYPE,
    /// Custom 1. Depends on implementation.
    CUSTOM_1,
    /// Custom 2. Depends on implementation.
    CUSTOM_2,
    /// Custom 3. Depends on implementation.
    CUSTOM_3
};
}
}

Table 2 - Video codec params description. Some params not supported by particular VCodecLibav library.

Parameter	Access	Description
LOG_LEVEL	read / write	Logging mode. Values: 0 - Disable. 1 - Only file. 2 - Only terminal (console). 3 - File and terminal.
BITRATE_KBPS	read / write	Bitrate, kbps. Default 5000 kbps. For H264 and H265(HEVC) encoding. According to this value, FPS and GOP size video codec calculate parameter for H264 or H265(HEVC) encoding. - Sets special settings for noisy video if bitrate is 0 for hardware encoding. - For software JPEG encoding bitrate incising gives higher quality.
MIN_BITRATE_KBPS	read / write	Minimum bitrate, kbps. Only for H264 and H265(HEVC) codecs. Default 2000 kbps. For variable bitrate mode. For H264 and H265(HEVC) encoding.
MAX_BITRATE_KBPS	read / write	Maximum bitrate, kbps. Only for H264 and H265(HEVC) codecs. Default 8000 kbps. For variable bitrate mode. For H264 and H265(HEVC) encoding.
BITRATE_MODE	read / write	Only for H264 and H265(HEVC) codecs. Bitrate mode: 0 - constant bitrate (default), 1 - variable bitrate. In variable bitrate mode for software encoder
QUALITY	read / write	Quality 0(low quality)-100%(maximum quality). Only for hardware JPEG encoding. Not supported by JPEG SW encoding.
FPS	read / write	FPS. For H264 and H265(HEVC) encoding only. According to this value, FPS and GOP size video codec calculate parameter for H264 and H265(HEVC) encoding.
GOP	read / write	GOP size (Period of key frames) for H264 and H265(HEVC) encoding. Value: 1 - each output frame is key frame, 20 - each 20th frame is key frame etc.
H264_PROFILE	read / write	H264 profile for H264 encoding: 0 - Baseline. 1 - Main. 2 - High. For hardware encoding only.
TYPE	read / write	Type of encoder/decoder: 0 - hardware (default). 1 - software.
CUSTOM_1	read / write	Number of thread for software encoder/decoder. Thread’s count for software encoder\decoder H264 / HEVC(H265) (not for JPEG): 1 - 32, default value 1. Note: in case one thread software encoder produces only one slice per frame. In case multiple threads frame can have multiple slices which can effect on RTSP servers.
CUSTOM_2	read / write	x264 codec speed preset: 0 - ultrafast (default), 1 - superfast.
CUSTOM_3	read / write	Not supported by VCodecLibav library.

getParam method

The getParam(…) method designed to obtain video codec parameter value. Method declaration:

float getParam(VCodecParam id);

Parameter	Description
id	Video codec parameter ID according to VCodecParam enum (see Table 2).

Returns: parameter value or -1 of the parameters doesn’t exist in particular video codec class.

executeCommand method

The executeCommand(…) method designed to execute video codec command. Method declaration:

bool executeCommand(VCodecCommand id);

Parameter	Description
id	Video codec command ID according to VCodecCommand enum.

Returns: TRUE if command was executed or FALSE if not.

VCodec.h file of VCodec library defines IDs for parameters (VCodecParam enum) and IDs for commands (VCodecCommand enum). VCodecCommand declaration:

enum class VCodecCommand
{
    /// Reset.
    RESET = 1,
    /// Generate key frame. For H264 and H265 codecs.
    MAKE_KEY_FRAME
};

Table 3 - Video codec commands description. Some commands maybe unsupported by particular video codec class.

Command	Description
RESET	Reset codec.
MAKE_KEY_FRAME	Generate key frame for H264 and HEVC codecs. Next frame will be key frame for H264 and HEVC codecs.

Build and connect to your project

Before compiling you have to install FFmpeg packages for your system.

Installation on Linux

sudo apt-get install -y build-essential cmake ffmpeg libavcodec-dev libavutil-dev libavformat-dev libavdevice-dev libavfilter-dev libcurl4

Installation on Windows

To install FFmpeg package in Windows make follow:

1. Go to link latest build of FFmpeg of FFmpeg and download ffmpeg-master-latest-win64-lgpl-shared.zip (64 bit libraries under LGPL license without software encoders / decoders, only hardware encoders / decoders) or download ffmpeg-master-latest-win64-gpl-shared.zip (64 bit libraries under GPL license with software and hardware encoders / decoders).
2. Extract zip file to any folder (recommended path C:/libs/ffmpeg/ffmpeg-master-latest-win64-lgpl-shared or C:/libs/ffmpeg/ffmpeg-master-latest-win64-gpl-shared). Inside folder must be folders: bin, include, lib etc.
3. Add path to FFmpeg libs to Windows system variables. To do it go to (Windows 11): Settings -> System -> Advanced system settings -> Environment Variables.
4. Create new variable in System variables section with name FFmpeg_DIR and path to your FFmpeg files (for example C:/libs/ffmpeg/ffmpeg-master-latest-win64-gpl-shared) and save changes. If system variable FFmpeg_DIR not set the library will try find FFmpeg libraries in C:/libs/ffmpeg/ffmpeg-master-latest-win64-gpl-shared folder.
5. Sometimes it requires Windows reboot to implement changes.

Build and connect to project

Typical commands to build VCodecLibav library:

cd VCodecLibav
mkdir build
cd build
cmake ..
make

If you want connect VCodecLibav library to your CMake project as source code you can make follow. For example, if your repository has structure:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp

Create folder 3rdparty in your repository and copy VCodecLibav repository folder to 3rdparty folder. New structure of your repository:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    VCodecLibav

Create CMakeLists.txt file in 3rdparty folder. CMakeLists.txt should contain:

cmake_minimum_required(VERSION 3.13)

################################################################################
## 3RD-PARTY
## dependencies for the project
################################################################################
project(3rdparty LANGUAGES CXX)

################################################################################
## SETTINGS
## basic 3rd-party settings before use
################################################################################
# To inherit the top-level architecture when the project is used as a submodule.
SET(PARENT ${PARENT}_YOUR_PROJECT_3RDPARTY)
# Disable self-overwriting of parameters inside included subdirectories.
SET(${PARENT}_SUBMODULE_CACHE_OVERWRITE OFF CACHE BOOL "" FORCE)

################################################################################
## CONFIGURATION
## 3rd-party submodules configuration
################################################################################
SET(${PARENT}_SUBMODULE_VCODEC_LIBAV                    ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VCODEC_LIBAV)
    SET(${PARENT}_VCODEC_LIBAV                          ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_VCODEC_LIBAV_TEST                     OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_VCODEC_LIBAV_EXAMPLE                  OFF CACHE BOOL "" FORCE)
endif()

################################################################################
## INCLUDING SUBDIRECTORIES
## Adding subdirectories according to the 3rd-party configuration
################################################################################
if (${PARENT}_SUBMODULE_VCODEC_LIBAV)
    add_subdirectory(VCodecLibav)
endif()

File 3rdparty/CMakeLists.txt adds folder VCodecLibav to your project and excluded test application and example (by default test application and example excluded from compiling if VCodecLibav included as sub-repository). Your repository new structure will be:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    CMakeLists.txt
    VCodecLibav

Next you need include folder 3rdparty in main CMakeLists.txt file of your repository. Add string at the end of your main CMakeLists.txt:

add_subdirectory(3rdparty)

Next you have to include VCodecLibav library in your src/CMakeLists.txt file:

target_link_libraries(${PROJECT_NAME} VCodecLibav)

Done!

Simple example

Simple example opens video file with OpenCV, captures video from file, convert frame to NV12 pixel format (required for codec) and encodes. Source code:

#include <iostream>
#include <opencv2/opencv.hpp>
#include "VCodecLibav.h"

int main(void)
{
    // Create codec object and set params.
    cr::video::VCodec* encoder = new cr::video::VCodecLibav();
    encoder->setParam(cr::video::VCodecParam::BITRATE_KBPS, 10000);
    encoder->setParam(cr::video::VCodecParam::FPS, 20);
    encoder->setParam(cr::video::VCodecParam::GOP, 30);
    encoder->setParam(cr::video::VCodecParam::H264_PROFILE, 0);

    // Open video file with OpenCV.
    cv::VideoCapture videoSource;
    if (!videoSource.open("test.mp4"))
        return -1;

    // Get frame size from video source.
    int width = (int)videoSource.get(cv::CAP_PROP_FRAME_WIDTH);
    int height = (int)videoSource.get(cv::CAP_PROP_FRAME_HEIGHT);

    // Init frames.
    cv::Mat inputFrameBgr(height, width, CV_8UC3);
    cv::Mat inputFrameYuv(height, width, CV_8UC3);
    cr::video::Frame nv12Frame(width, height, cr::video::Fourcc::NV12);
    cr::video::Frame h264Frame(width, height, cr::video::Fourcc::H264);

    // Main loop.
    while (true)
    {
        // Capture next video frame.
        videoSource >> inputFrameBgr;
        if (inputFrameBgr.empty())
        {
            // Set first video frame position.
            videoSource.set(cv::CAP_PROP_POS_FRAMES, 1);
            continue;
        }

        // Convert BGR to YUV.
        cvtColor(inputFrameBgr, inputFrameYuv, cv::COLOR_BGR2YUV);

        // Convert YUV to NV12 (replacing pixels). You can use something else.
        size_t p = height;
        nv12Frame.frameId++; // Just to show unique info.
        for (size_t i = 0; i < (size_t)height; i = i + 2)
        {
            for (size_t j = 0; j < (size_t)width; j = j + 2)
            {
                nv12Frame.data[i * (size_t)width + j] =
                inputFrameYuv.data[i * (size_t)width * 3 + j * 3];
                nv12Frame.data[i * (size_t)width + j + 1] =
                inputFrameYuv.data[i * (size_t)width * 3 + j * 3 + 3];
                nv12Frame.data[(i + 1) * (size_t)width + j] =
                inputFrameYuv.data[(i + 1) * (size_t)width * 3 + j * 3];
                nv12Frame.data[(i + 1) * (size_t)width + j + 1] =
                inputFrameYuv.data[(i + 1) * (size_t)width * 3 + j * 3 + 3];
                nv12Frame.data[p * width + j] =
                inputFrameYuv.data[i * (size_t)width * 3 + j * 3 + 1];
                nv12Frame.data[p * width + j + 1] =
                inputFrameYuv.data[i * (size_t)width * 3 + j * 3 + 2];
            }
            ++p;
        }

        // Encode data.
        if (!encoder->transcode(nv12Frame, h264Frame))
        {
            std::cout << "Can't encode frame" << std::endl;
            continue;
        }

        // Show info.
        std::cout << "[" << h264Frame.frameId << "] Size " <<
        h264Frame.size << " Compression ratio : %" <<
        (int)(100.0f * ((float)h264Frame.size / (float)nv12Frame.size)) <<
        std::endl;
    }
}

Test application

Test application (VCodecLibav/test/main.cpp) for VCodecLibav C++ library shows how library works on Intel platform. Test application generates artificial video, compresses it according to user’s parameters (codec type, bitrate or JPEG quality, GOP size and H264 profile), writes results to binary file “out.h264”, “out.hevc” or “out.mjpeg” and decompresses. The application shows encoding and decoding time for each video frame. To run application perform commands on Linux or just run VCodecOneVplTest.exe on Windows:

cd <application folder>
sudo chmod +x VCodecLibavTest
./VCodecLibav

After start you will see output:

====================================
VCodecLibav v1.3.2 test
====================================

Enter Encoder type (0 - JPEG, 1 - H264, 2 - HEVC) :

Chose codec type (0 - JPEG, 1 - H264). If H264 codec chosen you will see message:

====================================
VCodecLibav v1.3.0 test
====================================

Enter Encoder type (0 - JPEG, 1 - H264, 2 - HEVC) : 1
Enter backend type (0 - hardware, 1 - software): 1
Default params:
Bitrate, kbps 5000
Minimum bitrate, kbps 3000
Maximum bitrate, kbps 7000
Bitrate mode (0 - constant, 1 - variable) 0
FPS: 30
GOP size: 30
Video width 1280
Video height 720
H264 Profile: 0
Use default params (0 - no, 1 - yes) : 1
Set number frames: 10  
Set noise mode (0 - no noise, 1 - noise) : 0

User can set custom parameters. When params chosen test application will start writing encoded frames until stop (if user set number of frames). During encoding the application shows encoded data size and encoding time and decoding time:

Data size 1382400 / 11081   encoding time  34183 us / decoding time 8062 us
Data size 1382400 / 299   encoding time  3389 us / decoding time 1828 us
Data size 1382400 / 2704   encoding time  2878 us / decoding time 1056 us
Data size 1382400 / 6504   encoding time  2452 us / decoding time 1084 us
Data size 1382400 / 348   encoding time  2064 us / decoding time 559 us
Data size 1382400 / 329   encoding time  1957 us / decoding time 861 us
Data size 1382400 / 332   encoding time  2384 us / decoding time 662 us
Data size 1382400 / 20070   encoding time  2414 us / decoding time 1044 us
Data size 1382400 / 20833   encoding time  2297 us / decoding time 656 us
Data size 1382400 / 20834   encoding time  1952 us / decoding time 670 us