ged_web_logo

Ged C++ library

v2.2.4

Table of contents

Overview

C++ Ged (Changes detector) library is designed for automatic detection of subtle changes on videos. The library is implemented in C++ (C++17 standard) and utilizes the OpenMP library (2.0 standard and newer, built in C++ compiler) to facilitate parallel computation. Besides the OpenMP runtime of the compiler it uses only the ObjectDetector (interface), Frame and ConfigReader libraries, which are separate repositories (see Build and connect to your project). The library works real-time on low-power CPU. The library is compatible with any processors and operating systems that support the C++ compiler (C++17 standard), provided the compiler has built-in support for the OpenMP (2.0 standard and newer) parallel computing language. The library processes video of different camera types (daylight, SWIR, MWIR and LWIR) and detects small-size and low-contrast objects against complex backgrounds. Each instance of the Ged C++ class performs frame-by-frame processing of one video data stream: the result of a frame also depends on the previous frames of the stream. The library inherits its interface from the ObjectDetector (provides interface for object detector) class, offering flexible and customizable parameters. The library is designed mainly for fixed cameras and for PTZ cameras while pan, tilt and zoom do not change. The library can also be used for object detection after the camera has been moved by an external command. The library is compatible with low-power CPUs.

Ged is general-purpose software: it is not tied to any sector or application, and the developer of a program decides what the program does with it.

Versions

Table 1 - Library versions.

Version Release date What’s new
1.0.0 29.03.2021 First version.
2.0.0 26.09.2023 - Interface changed to ObjectDetector.
- Motion mask calculation algorithm optimized.
- Added convenient demo, test and example applications.
- Added applying detection mask feature.
2.0.1 13.11.2023 - ObjectDetector class updated.
2.0.2 16.11.2023 - Fixed deleting equal object mechanism.
2.0.3 29.12.2023 - Demo application updated.
- ObjectDetector class updated.
- Code optimized.
2.1.0 09.01.2024 - Updated with new interface of ObjectDetector with class names in parameters structure.
2.2.0 24.04.2024 - Dependency from OpenCV removed.
- Added unique object ID.
- Documentation updated.
2.2.1 20.05.2024 - Submodules updated.
- Documentation updated.
2.2.2 23.07.2024 - CMake structure updated.
2.2.3 12.10.2026 - Submodules updated.
2.2.4 19.10.2026 - Sub-repositories removed: the dependencies are in the folders next to the library (stand alone build) or are added by the parent project.
- Frames, detection masks, output frames of getMotionMask(…), param values and commands are checked (frame size from 24x24 to 8192x8192 pixels, data size, NaN values, exact command size).
- Fixed memory errors for YUYV frames with SCALE_FACTOR > 1, for frames higher than wide and for frames with 2 rows or columns after scaling.
- Fixed rectangle of merged objects, type of objects of lost tracks (0), frameId of objects and reuse of the output frame in getMotionMask(…).
- getMotionMask(…) and param changes are thread-safe during frame processing.
- Number of objects of a frame (8192) and number of tracks (16384) limited.
- OpenMP flags passed to the programs that link the library, compiler warnings and hardening flags, the same floating point results on all architectures.
- Example and demo application read the video file with the VSourceFile library, the demo application uses the SimpleUI library for the window and the VideoRecorderMp4 library for the video recording, the benchmark draws the test frames with the SimpleUI library: the applications do not use OpenCV.
- Documentation updated.

Library files

The library is supplied as source code or compiled version. The user is provided with a set of files in the form of a CMake project (repository). The repository structure is shown below:

CMakeLists.txt ------------ Main CMake file of the library.
src ----------------------- Folder with library source code.
    CMakeLists.txt -------- CMake file of the library.
    Ged.h ----------------- Main library header file.
    GedBoxTracker.h ------- Header file for box tracker library.
    GedBoxTracker.cpp ----- C++ implementation file.
    GedVersion.h ---------- Header file with library version.
    GedVersion.h.in ------- Service CMake file to generate version header.
    Ged.cpp --------------- C++ implementation file.
demo ---------------------- Folder for demo application.
    CMakeLists.txt -------- CMake file for demo application.
    README.md ------------- Demo application user manual.
    main.cpp -------------- Source C++ file of demo application.
example ------------------- Folder for example application.
    CMakeLists.txt -------- CMake file of example application.
    main.cpp -------------- Source C++ file of example application.
benchmark ----------------- Folder with benchmark application.
    CMakeLists.txt -------- CMake file of benchmark application.
    main.cpp -------------- Source C++ file of benchmark application.
static -------------------- Folder with images for the documentation.

The library depends on the ObjectDetector, Frame and ConfigReader libraries (separate repositories, see Build and connect to your project) and on the OpenMP runtime of the compiler. Demo application depends on the SimpleFileDialog library which provides file dialog function on the SimpleUI library which provides the window, the sliders and the drawing functions and on the VideoRecorderMp4 library (with VEncoderJPEG) which records MP4 files. The demo application and the example read and decode the video file with the VSourceFile library (separate repository) and its dependencies VSource, VCodec, VDecoderH264, VDecoderH265, VDecoderJPEG and FormatConverter. The benchmark application draws its test frames with the SimpleUI library.

Key features and capabilities

Table 2 - Key features and capabilities.

Parameter and feature Description
Programming language C++ (standard C++17) using the OpenMP library (version 2.0 and higher).
Supported OS Compatible with any operating system that supports the C++ compiler (C++17 standard) and the OpenMP library (version 2.0 and higher).
Shape of detected objects The library is able to detect objects of any shape. The minimum and maximum height and width of the objects to be detected are set by the user in the library parameters.
Supported pixel formats RGB24, BGR24, GRAY, YUV24, YUYV, UYVY, NV12, NV21, YV12, YU12. The library uses the GRAY format for video processing. If the pixel format of the image is different from GRAY, the library pre-converts the pixel formats to GRAY.
Maximum and minimum video frame size The minimum size of video frames to be processed is 24x24 pixels, and the maximum size is 8192x8192 pixels. The size of the video frames to be processed has a significant impact on the computation speed.
Coordinate system The algorithm uses a window coordinate system with the zero point in the upper left corner of the video frame.
Calculation speed The processing time per video frame depends on the computing platform used. The processing time per video frame can be estimated with the demo application. It is possible to scale video frames to provide higher calculation speed.
Discreteness of computation of coordinates The algorithm calculates the object bounding box for each detected object. The increment for calculation of position and parameters of the bounding box is 1 pixel.
Detection of changes A binary motion mask (motion or no motion for each pixel) is calculated for every processed frame, the objects are found on the motion mask and keep their ID from frame to frame (see Library principles).
Working conditions Algorithms implemented in the library are designed to work on fixed cameras mainly. It is possible to work with slight camera movement depending on the background. It is recommended to use a demo application to evaluate the quality of the algorithms in specific situations.

Supported pixel formats

Frame library (dependency of the Ged library) contains Fourcc enum, which defines supported pixel formats (Frame.h file). Ged library supports RAW pixel formats. For internal processing the library uses GRAY pixel format. If the pixel format of the image is different from GRAY, the library pre-converts the pixel formats to GRAY. Fourcc enum declaration:

enum class Fourcc
{
    /// RGB 24bit pixel format.
    RGB24 = MAKE_FOURCC_CODE('R', 'G', 'B', '3'),
    /// BGR 24bit pixel format.
    BGR24 = MAKE_FOURCC_CODE('B', 'G', 'R', '3'),
    /// YUYV 16bits per pixel format.
    YUYV  = MAKE_FOURCC_CODE('Y', 'U', 'Y', 'V'),
    /// UYVY 16bits per pixel format.
    UYVY  = MAKE_FOURCC_CODE('U', 'Y', 'V', 'Y'),
    /// Grayscale 8bit.
    GRAY  = MAKE_FOURCC_CODE('G', 'R', 'A', 'Y'),
    /// YUV 24bit per pixel format.
    YUV24  = MAKE_FOURCC_CODE('Y', 'U', 'V', '3'),
    /// NV12 pixel format.
    NV12  = MAKE_FOURCC_CODE('N', 'V', '1', '2'),
    /// NV21 pixel format.
    NV21  = MAKE_FOURCC_CODE('N', 'V', '2', '1'),
    /// YU12 (YUV420) - Planar pixel format.
    YU12 = MAKE_FOURCC_CODE('Y', 'U', '1', '2'),
    /// YV12 (YVU420) - Planar pixel format.
    YV12 = MAKE_FOURCC_CODE('Y', 'V', '1', '2'),
    /// JPEG compressed format.
    JPEG  = MAKE_FOURCC_CODE('J', 'P', 'E', 'G'),
    /// H264 compressed format.
    H264  = MAKE_FOURCC_CODE('H', '2', '6', '4'),
    /// HEVC compressed format.
    HEVC  = MAKE_FOURCC_CODE('H', 'E', 'V', 'C')
};

Table 3 - Bytes layout of supported RAW pixel formats. Example of 4x4 pixels image.

rgbRGB24 bgrBGR24
yuvYUV24 grayGRAY
yuyvYUYV uyvyUYVY
nv12NV12 nv21NV21
yu12YU12 yv12YV12

Library principles

The library processes the video frames in the following steps:

  1. The frame is converted to GRAY format (grayscale) and reduced SCALE_FACTOR times horizontally and vertically (processing size). The pixel values of FRAME_BUFFER_SIZE frames are summed up. The next steps are performed on every FRAME_BUFFER_SIZE-th frame (processed frame: the frame which completes the frame buffer) with the mean pixel values of the frame buffer; after the other frames the list of detected objects does not change.
  2. The binary motion mask is calculated at the processing size (0 - no motion, 255 - motion). SENSITIVITY sets the brightness threshold, pixels where the detection mask is 0 have no motion. This is the most time-consuming step, it can be calculated in several threads (NUM_THREADS). The motion mask of the last processed frame is given by the getMotionMask(…) method.
  3. The objects are found on the motion mask (motion pixels which are close to each other give one object) and filtered by size (MIN_OBJECT_WIDTH, MAX_OBJECT_WIDTH, MIN_OBJECT_HEIGHT and MAX_OBJECT_HEIGHT, compared at the processing size). Objects whose rectangles intersect or are up to 2 pixels apart are merged.
  4. The objects are matched with the objects of the previous processed frames, so an object keeps its ID from frame to frame.
  5. An object is reported after it has been found on X_DETECTION_CRITERIA processed frames (X_DETECTION_CRITERIA and Y_DETECTION_CRITERIA set one common value). After that it is reported with the same ID on every processed frame, also if it is not found (with a predicted rectangle). An object that is not found on RESET_CRITERIA processed frames in a row is dropped: when it is found again, it gets a new ID.
  6. The objects of the last processed frame can be retrieved using the getObjects() method.

The library is available as source code only. To utilize the library as source code, developers must include the library’s files into their project. The usage sequence for the library is as follows:

  1. Include the library files in the project.
  2. Create an instance of the Ged C++ class.
  3. If necessary, modify the default library parameters using the setParam() method.
  4. Create Frame class object for input frame and a vector of Objects.
  5. Call the detect(…) method to identify objects.
  6. Fill the created vector of Objects using the getObjects() method.

Ged class description

Ged class declaration

Ged.h file contains Ged class declaration. Ged class inherits interface from ObjectDetector interface class. Class declaration:

class Ged : public ObjectDetector
{
public:
    /// Get string of current library version.
    static std::string getVersion();

    /// Class constructor.
    Ged();

    /// Class destructor.
    ~Ged() override;

    /// Copy and move are not allowed (the object holds mutexes).
    Ged(const Ged&) = delete;
    Ged(Ged&&) = delete;
    Ged& operator= (const Ged&) = delete;
    Ged& operator= (Ged&&) = delete;

    /// Init object detector.
    bool initObjectDetector(ObjectDetectorParams& params) override;

    /// Set object detector param.
    bool setParam(ObjectDetectorParam id, float value) override;

    /// Get object detector param value.
    float getParam(ObjectDetectorParam id) override;

    ///  Get object detector params including list of detected objects.
    void getParams(ObjectDetectorParams& params) override;

    /// Get list of detected objects.
    std::vector<Object> getObjects() override;

    /// Execute command.
    bool executeCommand(ObjectDetectorCommand id) override;

    /// Perform detection.
    bool detect(rapidpixel::Frame& frame) override;

    /// Set detection mask.
    bool setMask(rapidpixel::Frame mask) override;

    /// Decode and execute command.
    bool decodeAndExecuteCommand(uint8_t* data, int size) override;

    /// This method retrieves the motion detection binary mask.
    bool getMotionMask(rapidpixel::Frame& mask);
};

getVersion method

The getVersion() method returns string of current version of Ged class. Method declaration:

static std::string getVersion();

Method can be used without Ged class instance. Example:

std::cout << "Ged version: " << rapidpixel::Ged::getVersion();

Console output:

Ged version: 2.2.4

initObjectDetector method

The initObjectDetector(…) method initializes detector. Method declaration:

bool initObjectDetector(ObjectDetectorParams& params) override;
Parameter Value
params Object detector parameters class. Object detector should initialize all parameters listed in ObjectDetectorParams. The library takes into account only following parameters from ObjectDetectorParams class (set with setParam(…)):
frameBufferSize (Default value 30)
minObjectWidth (Default value 2)
maxObjectWidth (Default value 128)
minObjectHeight (Default value 2)
maxObjectHeight (Default value 128)
sensitivity (Default value 10)
scaleFactor (Default value 1)
numThreads (Default value 1)
enable (Default value true)
xDetectionCriteria (Default value 1)
yDetectionCriteria (Default value 1)
resetCriteria (Default value 6)
If particular parameter is out of valid range (see ObjectDetectorParam enum) it is not set: the parameter keeps its current value (the default value for a new object). xDetectionCriteria and yDetectionCriteria set one common value: yDetectionCriteria is set last.

Returns: always returns TRUE.

setParam method

The setParam(…) method designed to set new Ged parameter value. setParam(…) is thread-safe method. This means that the setParam(…) method can be safely called from any thread (also during frame processing: a frame is processed with the parameters which were set before the detect(…) method was called). Method declaration:

bool setParam(ObjectDetectorParam id, float value) override;
Parameter Description
id Parameter ID according to ObjectDetectorParam enum. The library supports not all parameters from ObjectDetectorParam enum.
value Parameter value. Value depends on parameter ID (valid values in Table 5). Integer parameters take the integer part of the value. NaN is not accepted.

Returns: TRUE if the parameter was set or FALSE if not (not supported or out of valid range). After FALSE the parameter keeps its value.

getParam method

The getParam(…) method designed to obtain object detector parameter value. getParam(…) is thread-safe method. This means that the getParam(…) method can be safely called from any thread. Method declaration:

float getParam(ObjectDetectorParam id) override;
Parameter Description
id Parameter ID according to ObjectDetectorParam enum. The library supports not all parameters from ObjectDetectorParam enum. If the library doesn’t support particular parameter the method will return -1.

Returns: parameter value or -1.0f if the parameter is not supported by Ged library.

getParams method

The getParams(…) method designed to obtain object detector params structures as well a list of detected objects (included in ObjectDetectorParams class). getParams(…) is thread-safe method. This means that the getParams(…) method can be safely called from any thread. Method declaration:

void getParams(ObjectDetectorParams& params) override;
Parameter Description
params Object detector params ObjectDetectorParams. If the library doesn’t support particular parameter it will be not changeable and will have default value.

getObjects method

The getObjects() method returns list of detected objects. User can obtain object list of detected objects via getParams(…) method as well. getObjects() is thread-safe method. This means that the getObjects() method can be safely called from any thread. Method declaration:

std::vector<Object> getObjects() override;

Returns: list of detected objects (see Object class description). If no detected object the list will be empty.

executeCommand method

The executeCommand(…) method designed to execute object detector command. executeCommand(…) is thread-safe method. This means that the executeCommand(…) method can be safely called from any thread. Method declaration:

bool executeCommand(ObjectDetectorCommand id) override;
Parameter Description
id Command ID according to ObjectDetectorCommand enum.

Returns: TRUE if the command was executed or FALSE if not (only if command ID not valid).

detect method

The detect(…) performs detection algorithm. The library sums up FRAME_BUFFER_SIZE frames and then calculates the motion mask and the list of objects. detect(…) is thread-safe method: calls of detect(…) and getMotionMask(…) from different threads wait for each other. At most 8192 objects are found in one frame (the objects at the top of the frame first) and at most 16384 tracks are kept (the tracks which were not detected for the longest time are removed first). Method declaration:

bool detect(rapidpixel::Frame& frame) override;
Parameter Description
frame Video frame for processing. Object detector processes only RAW pixel formats (BGR24, RGB24, GRAY, YUV24, YUYV, UYVY, NV12, NV21, YV12, YU12, see Frame class description). The library uses the GRAY format for video processing. If the pixel format of the image is different from GRAY, the library pre-converts the pixel formats to GRAY. Width and height must be from 24 to 8192 pixels, data must not be nullptr and size must be at least the size of the data that the library reads: width * height (GRAY and the luma plane of NV12, NV21, YV12 and YU12), width * height * 2 (YUYV, UYVY) or width * height * 3 (BGR24, RGB24, YUV24). The library can not check that data points to size valid bytes. frameId of the frame is copied to the detected objects.

Returns: TRUE if the video frame was processed or FALSE if not (frame not valid or not enough memory for the processing buffers). If object detector disabled the method will return TRUE for a valid frame. After FALSE the list of detected objects is empty.

setMask method

The setMask(…) method designed to set detection mask. The user can disable detection in any areas of the video frame. For this purpose the user can create an image of any size and configuration with GRAY pixel format. Mask image pixel values equal to 0 prohibit detection of objects in the corresponding place of video frames. Any other mask pixel value other than 0 allows detection of objects at the corresponding location of video frames. The mask is used for detection algorithms to compute a binary motion mask. The method can be called either before video frame processing or during video frame processing (setMask(…) is thread-safe method). Method declaration:

bool setMask(rapidpixel::Frame mask) override;
Parameter Description
mask Detection mask is Frame object with GRAY, NV12, NV21, YV12 or YU12 pixel format (only the luma plane is used). Width and height must be > 0, data must not be nullptr and size must be at least width * height. The mask is passed by value: the library checks the copy that the copy constructor of the Frame class makes (it copies a mask with data nullptr and size > 0 as a mask of zeros). Detector omits image segments, where detection mask pixel values equal 0. Mask can have any resolution. If resolution of mask (width and height) not equal to video frame resolution the library will scale this mask up to original processed video resolution.

Returns: TRUE if the mask is accepted or FALSE if not (not valid pixel format, empty, data size too small or not enough memory for the copy). After FALSE the previous mask is used.

getMotionMask method

The getMotionMask(…) method retrieves the motion detection binary mask (255 - motion, 0 - no motion), that is utilized by the detector to identify objects in the video stream. The mask has the size of the processed video frames divided by SCALE_FACTOR (640x480 before the first frame is processed). Method not included in ObjectDetector interface. getMotionMask(…) is thread-safe method: it waits for the end of the processing of a frame. Method declaration:

bool getMotionMask(rapidpixel::Frame& mask);
Parameter Description
mask Image of motion mask. If the frame has the size of the motion mask, GRAY (preferable), NV12, NV21, YV12 or YU12 pixel format, data and enough size, the mask is copied to its luma plane. Otherwise (for example a not initialized Frame object) the library replaces the frame by a new GRAY frame of the mask size.

Returns: TRUE if the mask image is filled or FALSE if not (not enough memory for the new frame, or no mask because the last detect(…) call could not allocate the processing buffers).

decodeAndExecuteCommand method

The decodeAndExecuteCommand(…) method decodes and executes command which encoded by encodeSetParamCommand(…) or encodeCommand(…) method of ObjectDetector class. decodeAndExecuteCommand(…) is thread-safe method. This means that the decodeAndExecuteCommand(…) method can be safely called from any thread. Method declaration:

bool decodeAndExecuteCommand(uint8_t* data, int size) override;
Parameter Description
data Pointer to input command. Must not be nullptr.
size Size of command. Must be exactly 11 bytes for SET_PARAM or 7 bytes for COMMAND.

Returns: TRUE if command decoded (SET_PARAM or COMMAND) and executed (action command or set param command) or FALSE if not (data nullptr, wrong size, wrong header, unknown ID or not valid parameter value).

encodeSetParamCommand method of ObjectDetector class

The encodeSetParamCommand(…) static method of the ObjectDetector interface designed to encode command to change any parameter for remote object detector (including motion detectors). To control object detector remotely, the developer has to design his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the ObjectDetector class contains static methods for encoding the control command. The ObjectDetector class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeSetParamCommand(…) designed to encode SET_PARAM command. Method declaration:

static void encodeSetParamCommand(uint8_t* data, int& size, ObjectDetectorParam id, float value);
Parameter Description
data Pointer to data buffer for encoded command. Must have size >= 11.
size Size of encoded data. Will be 11 bytes.
id Parameter ID according to ObjectDetectorParam enum.
value Parameter value. Value depends on parameter ID.

encodeSetParamCommand(…) is static and used without ObjectDetector class instance. This method used on client side (control system). Command encoding example:

// Buffer for encoded data.
uint8_t data[11];
// Size of encoded data.
int size = 0;
// Parameter value.
const float outValue = 10.0f;
// Encode command.
rapidpixel::ObjectDetector::encodeSetParamCommand(
    data, size, rapidpixel::ObjectDetectorParam::MIN_OBJECT_WIDTH, outValue);

encodeCommand method of ObjectDetector class

The encodeCommand(…) static method of the ObjectDetector interface designed to encode command for remote object detector (including motion detectors). To control object detector remotely, the developer has to design his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the ObjectDetector class contains static methods for encoding the control command. The ObjectDetector class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeCommand(…) designed to encode COMMAND (action command). Method declaration:

static void encodeCommand(uint8_t* data, int& size, ObjectDetectorCommand id);
Parameter Description
data Pointer to data buffer for encoded command. Must have size >= 7.
size Size of encoded data. Will be 7 bytes.
id Command ID according to ObjectDetectorCommand enum.

encodeCommand(…) is static and used without ObjectDetector class instance. This method used on client side (control system). Command encoding example:

// Buffer for encoded data.
uint8_t data[11];
// Size of encoded data.
int size = 0;
// Encode command.
rapidpixel::ObjectDetector::encodeCommand(
    data, size, rapidpixel::ObjectDetectorCommand::RESET);

decodeCommand method of ObjectDetector class

The decodeCommand(…) static method of the ObjectDetector interface designed to decode command on object detector side (edge device) encoded by encodeSetParamCommand(…) or encodeCommand(…) method of ObjectDetector class. Method declaration:

static int decodeCommand(uint8_t* data, int size, ObjectDetectorParam& paramId, ObjectDetectorCommand& commandId, float& value);
Parameter Description
data Pointer to input command.
size Size of command: 11 bytes for SET_PARAM command or 7 bytes for COMMAND.
paramId Parameter ID according to ObjectDetectorParam enum. After decoding SET_PARAM command the method will return parameter ID.
commandId Command ID according to ObjectDetectorCommand enum. After decoding COMMAND the method will return command ID.
value Parameter value (after decoding SET_PARAM command).

Returns: 0 - in case decoding COMMAND, 1 - in case decoding SET_PARAM command or -1 in case errors (data is nullptr, the size is not exactly 11 bytes for SET_PARAM or 7 bytes for COMMAND, wrong version or data type, ID outside of the range of the enum). The method does not check the parameter value.

Data structures

ObjectDetectorCommand enum

Enum declaration:

enum class ObjectDetectorCommand
{
    /// Reset.
    RESET = 1,
    /// Enable.
    ON,
    /// Disable.
    OFF
};

Table 4 - Object detector commands description. Some commands may be unsupported by particular object detector class.

Command Description
RESET Reset algorithm. Clears the list of detected objects and resets all internal filters. Reset command will be applied only after processing new video frame.
ON Enable object detector. If the detector is not activated, frame processing is not performed - the list of detected objects will always be empty.
OFF Disable object detector. If the detector is not activated, frame processing is not performed - the list of detected objects will always be empty.

ObjectDetectorParam enum

Enum declaration:

enum class ObjectDetectorParam
{
    /// Logging mode. Values: 0 - Disable, 1 - Only file,
    /// 2 - Only terminal (console), 3 - File and terminal (console).
    LOG_MODE = 1,
    /// Frame buffer size. Depends on implementation.
    FRAME_BUFFER_SIZE,
    /// Minimum object width to be detected, pixels. To be detected object's
    /// width must be >= MIN_OBJECT_WIDTH.
    MIN_OBJECT_WIDTH,
    /// Maximum object width to be detected, pixels. To be detected object's
    /// width must be <= MAX_OBJECT_WIDTH.
    MAX_OBJECT_WIDTH,
    /// Minimum object height to be detected, pixels. To be detected object's
    /// height must be >= MIN_OBJECT_HEIGHT.
    MIN_OBJECT_HEIGHT,
    /// Maximum object height to be detected, pixels. To be detected object's
    /// height must be <= MAX_OBJECT_HEIGHT.
    MAX_OBJECT_HEIGHT,
    /// Minimum object's horizontal speed to be detected, pixels/frame. To be
    /// detected object's horizontal speed must be >= MIN_X_SPEED.
    MIN_X_SPEED,
    /// Maximum object's horizontal speed to be detected, pixels/frame. To be
    /// detected object's horizontal speed must be <= MAX_X_SPEED.
    MAX_X_SPEED,
    /// Minimum object's vertical speed to be detected, pixels/frame. To be
    /// detected object's vertical speed must be >= MIN_Y_SPEED.
    MIN_Y_SPEED,
    /// Maximum object's vertical speed to be detected, pixels/frame. To be
    /// detected object's vertical speed must be <= MAX_Y_SPEED.
    MAX_Y_SPEED,
    /// Probability threshold from 0 to 1. To be detected object detection
    /// probability must be >= MIN_DETECTION_PROBABILITY.
    MIN_DETECTION_PROBABILITY,
    /// Horizontal track detection criteria, frames. By default shows how many
    /// frames the objects must move in any(+/-) horizontal direction to be
    /// detected.
    X_DETECTION_CRITERIA,
    /// Vertical track detection criteria, frames. By default shows how many
    /// frames the objects must move in any(+/-) vertical direction to be
    /// detected.
    Y_DETECTION_CRITERIA,
    /// Track reset criteria, frames. By default shows how many
    /// frames the objects should be not detected to be excluded from results.
    RESET_CRITERIA,
    /// Detection sensitivity. Depends on implementation. Default from 0 to 1.
    SENSITIVITY,
    /// Frame scaling factor for processing purposes. Reduce the image size by
    /// scaleFactor times horizontally and vertically for faster processing.
    SCALE_FACTOR,
    /// Num threads. Number of threads for parallel computing.
    NUM_THREADS,
    /// Processing time of last frame in microseconds.
    PROCESSING_TIME_MCS,
    /// Algorithm type. Depends on implementation.
    TYPE,
    /// Mode. Default: 0 - Off, 1 - On.
    MODE,
    /// Custom parameter. Depends on implementation.
    CUSTOM_1,
    /// Custom parameter. Depends on implementation.
    CUSTOM_2,
    /// Custom parameter. Depends on implementation.
    CUSTOM_3
};

Table 5 - Ged class params description (from ObjectDetector interface class). Some params are not supported by Ged class.

Parameter Access Description
LOG_MODE not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
FRAME_BUFFER_SIZE read / write Frame buffer size. Valid values from 1 to 128. Defines how many frames are summed up before motion mask calculation. Default value is 30.
MIN_OBJECT_WIDTH read / write Minimum object width to be detected, pixels. Valid values from 1 to 8192. Should be < MAX_OBJECT_WIDTH (the library does not check it). To be detected object’s width must be >= MIN_OBJECT_WIDTH. Default value is 2.
MAX_OBJECT_WIDTH read / write Maximum object width to be detected, pixels. Valid values from 1 to 8192. Should be > MIN_OBJECT_WIDTH (the library does not check it). To be detected object’s width must be <= MAX_OBJECT_WIDTH. Default value is 128.
MIN_OBJECT_HEIGHT read / write Minimum object height to be detected, pixels. Valid values from 1 to 8192. Should be < MAX_OBJECT_HEIGHT (the library does not check it). To be detected object’s height must be >= MIN_OBJECT_HEIGHT. Default value is 2.
MAX_OBJECT_HEIGHT read / write Maximum object height to be detected, pixels. Valid values from 1 to 8192. Should be > MIN_OBJECT_HEIGHT (the library does not check it). To be detected object’s height must be <= MAX_OBJECT_HEIGHT. Default value is 128.
MIN_X_SPEED not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
MAX_X_SPEED not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
MIN_Y_SPEED not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
MAX_Y_SPEED not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
MIN_DETECTION_PROBABILITY not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
X_DETECTION_CRITERIA read / write Track detection criteria, frames. Shows how many frames the objects must be detected to be included into results. Valid values from 1 to 8192. Default value is 1. X_DETECTION_CRITERIA and Y_DETECTION_CRITERIA set and return one common value.
Y_DETECTION_CRITERIA read / write Track detection criteria, frames. Shows how many frames the objects must be detected to be included into results. Valid values from 1 to 8192. Default value is 1. X_DETECTION_CRITERIA and Y_DETECTION_CRITERIA set and return one common value.
RESET_CRITERIA read / write Track reset criteria, frames. Shows how many frames the objects should be not detected to be excluded from results. Valid values from 1 to 8192. Default value is 6.
SENSITIVITY read / write Detection sensitivity. Valid values from 0 to 255. Default value is 10. For Ged library this parameter means pixel brightness deviation threshold from 0 to 255 for calculation motion mask. The first processing step is the computation of the binary motion mask. The algorithm makes a decision about the presence of motion in a particular pixel also on the basis of changes in pixel brightness. The brightness threshold is determined by the SENSITIVITY parameter. The lower this parameter is, the less contrasty objects will be detected, but there will be more influence of video noise. The larger this parameter is, the more contrast objects must have in order to be detected. The recommended value for thermal cameras is 15. The recommended value for daylight cameras is 10.
SCALE_FACTOR read / write Frame scaling factor for processing purposes. Reduce the image size by scaleFactor times horizontally and vertically for faster processing. Allows to increase calculation speed but slightly reduces sensitivity. Valid values from 1 to 9. Default value is 1.
NUM_THREADS read / write Num threads. Number of threads for parallel computing. Multiple threads are used to compute the binary motion mask. Valid values from 0 to 64 (0 and 1 - one thread). Default value is 1.
PROCESSING_TIME_MCS read only Processing time of last frame in microseconds.
TYPE not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
MODE read / write Mode: 0 - Off, any other value - On. Default value is 1 (On). If the detector is not activated, frame processing is not performed - the list of detected objects will always be empty.
CUSTOM_1 not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
CUSTOM_2 not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.
CUSTOM_3 not supported Not used. setParam(…) returns FALSE, getParam(…) returns -1.

Object structure

Object structure used to describe detected object. Object structure declared in ObjectDetector.h file. Structure declaration:

struct Object
{
    /// Object ID. Must be unique for particular object.
    int id{ 0 };
    /// Frame ID. Must be the same as frame ID of processed video frame.
    int frameId{ 0 };
    /// Object type. Depends on implementation.
    int type{ 0 };
    /// Object rectangle width, pixels.
    int width{ 0 };
    /// Object rectangle height, pixels.
    int height{ 0 };
    /// Object rectangle top-left horizontal coordinate, pixels.
    int x{ 0 };
    /// Object rectangle top-left vertical coordinate, pixels.
    int y{ 0 };
    /// Horizontal component of object velocity, +-pixels/frame.
    float vX{ 0.0f };
    /// Vertical component of object velocity, +-pixels/frame.
    float vY{ 0.0f };
    /// Detection probability from 0 (minimum) to 1 (maximum).
    float p{ 0.0f };
};

Table 6 - Object structure fields description.

Field Type Description
id int Object ID. Unique for particular object. The library assigns a unique ID to the frame of a new detected object and does not change it from frame to frame. If the object disappears and reappears, the library can assign a new ID. This is necessary for control algorithms to distinguish different objects from frame to frame. After a reset of the detector the objects get new IDs. The IDs are counted from 0 to 65535 and then from 0 again.
frameId int Frame ID. Will be the same as frame ID of processed video frame (the frame which completes the frame buffer).
type int Not used. Will have value 0.
width int Object rectangle width, pixels. The library detects objects with MIN_OBJECT_WIDTH <= width <= MAX_OBJECT_WIDTH (see ObjectDetectorParam enum description; the limits are compared after integer division by SCALE_FACTOR). Merged objects (rectangles which intersect) and predicted rectangles of objects which are not detected in the frame can be larger.
height int Object rectangle height, pixels. The library detects objects with MIN_OBJECT_HEIGHT <= height <= MAX_OBJECT_HEIGHT (see ObjectDetectorParam enum description; the limits are compared after integer division by SCALE_FACTOR). Merged objects (rectangles which intersect) and predicted rectangles of objects which are not detected in the frame can be larger.
x int Object rectangle top-left horizontal coordinate, pixels. The predicted rectangle of an object which is not detected in the frame can be outside of the frame.
y int Object rectangle top-left vertical coordinate, pixels. The predicted rectangle of an object which is not detected in the frame can be outside of the frame.
vX float Estimated horizontal speed, pixels / frame.
vY float Estimated vertical speed, pixels / frame.
p float Not used. Will have value 0.0.

ObjectDetectorParams class description

ObjectDetectorParams class declaration

ObjectDetectorParams class used for object detector initialization (initObjectDetector(…) method) or to get all actual params (getParams() method) including list of detected objects. Also ObjectDetectorParams provides structure to write/read params from JSON files (JSON_READABLE macro, see ConfigReader class description) and provide methods to encode and decode params. Class declaration:

class ObjectDetectorParams
{
public:
    /// Init string. Depends on implementation.
    std::string initString{ "" };
    /// Logging mode. Values:
    /// 0 - Disable,
    /// 1 - Only file,
    /// 2 - Only terminal (console),
    /// 3 - File and terminal (console).
    int logMode{ 0 };
    /// Frame buffer size. Depends on implementation.
    int frameBufferSize{ 1 };
    /// Minimum object width to be detected, pixels. To be detected object's
    /// width must be >= minObjectWidth.
    int minObjectWidth{ 4 };
    /// Maximum object width to be detected, pixels. To be detected object's
    /// width must be <= maxObjectWidth.
    int maxObjectWidth{ 128 };
    /// Minimum object height to be detected, pixels. To be detected object's
    /// height must be >= minObjectHeight.
    int minObjectHeight{ 4 };
    /// Maximum object height to be detected, pixels. To be detected object's
    /// height must be <= maxObjectHeight.
    int maxObjectHeight{ 128 };
    /// Minimum object's horizontal speed to be detected, pixels/frame. To be
    /// detected object's horizontal speed must be >= minXSpeed.
    float minXSpeed{ 0.0f };
    /// Maximum object's horizontal speed to be detected, pixels/frame. To be
    /// detected object's horizontal speed must be <= maxXSpeed.
    float maxXSpeed{ 30.0f };
    /// Minimum object's vertical speed to be detected, pixels/frame. To be
    /// detected object's vertical speed must be >= minYSpeed.
    float minYSpeed{ 0.0f };
    /// Maximum object's vertical speed to be detected, pixels/frame. To be
    /// detected object's vertical speed must be <= maxYSpeed.
    float maxYSpeed{ 30.0f };
    /// Probability threshold from 0 to 1. To be detected object detection
    /// probability must be >= minDetectionProbability.
    float minDetectionProbability{ 0.5f };
    /// Horizontal track detection criteria, frames. By default shows how many
    /// frames the objects must move in any(+/-) horizontal direction to be
    /// detected.
    int xDetectionCriteria{1};
    /// Vertical track detection criteria, frames. By default shows how many
    /// frames the objects must move in any(+/-) vertical direction to be
    /// detected.
    int yDetectionCriteria{ 1 };
    /// Track reset criteria, frames. By default shows how many
    /// frames the objects should be not detected to be excluded from results.
    int resetCriteria{ 1 };
    /// Detection sensitivity. Depends on implementation. Default from 0 to 1.
    float sensitivity{ 0.04f };
    /// Frame scaling factor for processing purposes. Reduce the image size by
    /// scaleFactor times horizontally and vertically for faster processing.
    int scaleFactor{ 1 };
    /// Num threads. Number of threads for parallel computing.
    int numThreads{ 1 };
    /// Processing time of last frame in microseconds.
    int processingTimeMcs{ 0 };
    /// Algorithm / backend type. Depends on implementation.
    int type{ 0 };
    /// Mode. Default: false - Off, on - On.
    bool enable{ true };
    /// Custom parameter. Depends on implementation.
    float custom1{ 0.0f };
    /// Custom parameter. Depends on implementation.
    float custom2{ 0.0f };
    /// Custom parameter. Depends on implementation.
    float custom3{ 0.0f };
    /// List of detected objects.
    std::vector<Object> objects;
    // A list of object class names used in detectors that recognize different
    // object classes. Detected objects have an attribute called 'type.'
    // If a detector doesn't support object class recognition or can't determine
    // the object type, the 'type' field must be set to 0. Otherwise, the 'type'
    // should correspond to the ordinal number of the class name from the
    // 'classNames' list (if the list was set in params), starting from 1
    // (where the first element in the list has 'type == 1').
    std::vector<std::string> classNames{ "" };

    JSON_READABLE(ObjectDetectorParams, initString, logMode, frameBufferSize,
                  minObjectWidth, maxObjectWidth, minObjectHeight,
                  maxObjectHeight, minXSpeed, maxXSpeed, minYSpeed, maxYSpeed,
                  minDetectionProbability, xDetectionCriteria,
                  yDetectionCriteria, resetCriteria, sensitivity, scaleFactor,
                  numThreads, type, enable, custom1, custom2, custom3,
                  classNames);

    /// Default constructor.
    ObjectDetectorParams() = default;

    /// Copy constructor.
    ObjectDetectorParams(const ObjectDetectorParams& src) = default;

    /// operator =
    ObjectDetectorParams& operator= (const ObjectDetectorParams& src) = default;

    /// Encode params.
    bool encode(uint8_t* data, int bufferSize, int& size,
                ObjectDetectorParamsMask* mask = nullptr);

    /// Decode params.
    bool decode(uint8_t* data, int dataSize);
};

Table 7 - ObjectDetectorParams class fields description.

Field Type Description
initString string Not used. Can have any value.
logMode int Not used. Can have any value.
frameBufferSize int Frame buffer size. Valid values from 1 to 128. Defines how many frames are summed up before motion mask calculation.
minObjectWidth int Minimum object width to be detected, pixels. To be detected object’s width must be >= minObjectWidth.
maxObjectWidth int Maximum object width to be detected, pixels. To be detected object’s width must be <= maxObjectWidth.
minObjectHeight int Minimum object height to be detected, pixels. To be detected object’s height must be >= minObjectHeight.
maxObjectHeight int Maximum object height to be detected, pixels. To be detected object’s height must be <= maxObjectHeight.
minXSpeed float Not used. Can have any values.
maxXSpeed float Not used. Can have any values.
minYSpeed float Not used. Can have any values.
maxYSpeed float Not used. Can have any values.
minDetectionProbability float Not used. Can have any values.
xDetectionCriteria int Track detection criteria, frames. Shows how many frames the objects must be detected to be detected.
yDetectionCriteria int Track detection criteria, frames. Shows how many frames the objects must be detected to be detected.
resetCriteria int Track reset criteria, frames. Shows how many frames the objects should be not detected to be excluded from results.
sensitivity float Detection sensitivity. For Ged library this parameter means pixel brightness deviation threshold from 0 to 255 for calculation motion mask. The first processing step is the computation of the binary motion mask. The algorithm makes a decision about the presence of motion in a particular pixel also on the basis of changes in pixel brightness. The brightness threshold is determined by the SENSITIVITY parameter. The lower this parameter is, the less contrasty objects will be detected, but there will be more influence of video noise. The larger this parameter is, the more contrast objects must have in order to be detected. The recommended value for thermal cameras is 15. The recommended value for daylight cameras is 10.
scaleFactor int Frame scaling factor for processing purposes. Reduce the image size by scaleFactor times horizontally and vertically for faster processing. Allows to increase calculation speed but slightly reduces sensitivity.
numThreads int Number of threads for parallel computing. Multiple threads are used to compute the binary motion mask.
processingTimeMcs int Processing time of last frame in microseconds.
type int Not used. Can have any value.
enable bool Mode: false - Off, true - On.
custom1 float Not used. Can have any values.
custom2 float Not used. Can have any values.
custom3 float Not used. Can have any values.
objects std::vector List of detected objects.
classNames std::vector A list of object class names used in detectors that recognize different object classes. Detected objects have an attribute called ‘type.’ If a detector doesn’t support object class recognition or can’t determine the object type, the ‘type’ field must be set to 0. Otherwise, the ‘type’ should correspond to the ordinal number of the class name from the ‘classNames’ list (if the list was set in params), starting from 1 (where the first element in the list has ‘type == 1’).

Note: ObjectDetectorParams class fields listed in Table 7 must reflect params set/get by methods setParam(…) and getParam(…).

Serialize object detector params

ObjectDetectorParams class provides method encode(…) to serialize object detector params (fields of ObjectDetectorParams class, see Table 7). Serialization of object detector params necessary in case when you need to send params via communication channels. Method provides options to exclude particular parameters from serialization. To do this method inserts binary mask (4 bytes) where each bit represents particular parameter and decode(…) method recognizes it. Method declaration:

bool encode(uint8_t* data, int bufferSize, int& size, ObjectDetectorParamsMask* mask = nullptr);
Parameter Value
data Pointer to data buffer.
bufferSize Size of data buffer. The buffer must hold all selected fields except the objects: 102 bytes for the default params without objects (the size depends on the length of initString and classNames), otherwise the method returns FALSE. If the buffer is not large enough for all detected objects (40 bytes per object), only the objects which fit into the buffer are encoded.
size Size of encoded data. 102 bytes for the default params without objects.
mask Parameters mask - pointer to ObjectDetectorParamsMask structure. ObjectDetectorParamsMask (declared in ObjectDetector.h file) determines flags for each field (parameter) declared in ObjectDetectorParams class. If the user wants to exclude any parameters from serialization, he can put a pointer to the mask. If the user wants to exclude a particular parameter from serialization, he should set the corresponding flag in the ObjectDetectorParamsMask structure to false.

ObjectDetectorParamsMask structure declaration:

struct ObjectDetectorParamsMask
{
    bool initString{ true };
    bool logMode{ true };
    bool frameBufferSize{ true };
    bool minObjectWidth{ true };
    bool maxObjectWidth{ true };
    bool minObjectHeight{ true };
    bool maxObjectHeight{ true };
    bool minXSpeed{ true };
    bool maxXSpeed{ true };
    bool minYSpeed{ true };
    bool maxYSpeed{ true };
    bool minDetectionProbability{ true };
    bool xDetectionCriteria{ true };
    bool yDetectionCriteria{ true };
    bool resetCriteria{ true };
    bool sensitivity{ true };
    bool scaleFactor{ true };
    bool numThreads{ true };
    bool processingTimeMcs{ true };
    bool type{ true };
    bool enable{ true };
    bool custom1{ true };
    bool custom2{ true };
    bool custom3{ true };
    bool objects{ true };
    bool classNames{ true };
};

Example without parameters mask:

// Prepare params.
rapidpixel::ObjectDetectorParams in;
in.logMode = 111;
in.classNames = { "apple", "banana", "orange", "pineapple", "strawberry",
                  "watermelon", "lemon", "peach", "pear", "plum" };
for (int i = 0; i < 5; ++i)
{
    rapidpixel::Object obj;
    obj.id = i;
    obj.type = i % 3;
    obj.width = 10 + i;
    obj.height = 20 + i;
    obj.x = 100 + 10 * i;
    obj.y = 200 + 10 * i;
    obj.vX = 0.5f * static_cast<float>(i);
    obj.vY = -0.5f * static_cast<float>(i);
    obj.p = 0.9f;
    in.objects.push_back(obj);
}

// Encode data.
uint8_t data[1024];
int size = 0;
if (!in.encode(data, 1024, size))
{
    std::cout << "Can't encode data" << std::endl;
    return false;
}
std::cout << "Encoded data size: " << size << " bytes" << std::endl;

Example with parameters mask:

// Prepare params.
rapidpixel::ObjectDetectorParams in;
in.logMode = 111;
in.classNames = { "apple", "banana", "orange", "pineapple", "strawberry",
                  "watermelon", "lemon", "peach", "pear", "plum" };
for (int i = 0; i < 5; ++i)
{
    rapidpixel::Object obj;
    obj.id = i;
    obj.type = i % 3;
    obj.width = 10 + i;
    obj.height = 20 + i;
    obj.x = 100 + 10 * i;
    obj.y = 200 + 10 * i;
    obj.vX = 0.5f * static_cast<float>(i);
    obj.vY = -0.5f * static_cast<float>(i);
    obj.p = 0.9f;
    in.objects.push_back(obj);
}

// Prepare mask.
rapidpixel::ObjectDetectorParamsMask mask;
mask.logMode = false;

// Encode data.
uint8_t data[1024];
int size = 0;
if (!in.encode(data, 1024, size, &mask))
{
    std::cout << "Can't encode data" << std::endl;
    return false;
}
std::cout << "Encoded data size: " << size << " bytes" << std::endl;

Deserialize object detector params

ObjectDetectorParams class provides method decode(…) to deserialize params (fields of ObjectDetectorParams class, see Table 7). Deserialization of params necessary in case when you need to receive params via communication channels. Method automatically recognizes which parameters were serialized by encode(…) method. Method declaration:

bool decode(uint8_t* data, int dataSize);
Parameter Value
data Pointer to encoded data.
dataSize Size of encoded data.

Returns: TRUE if data decoded (deserialized) or FALSE if not.

Example:

// Prepare params.
rapidpixel::ObjectDetectorParams in;
in.logMode = 111;
in.classNames = { "apple", "banana", "orange", "pineapple", "strawberry",
                  "watermelon", "lemon", "peach", "pear", "plum" };
for (int i = 0; i < 5; ++i)
{
    rapidpixel::Object obj;
    obj.id = i;
    obj.type = i % 3;
    obj.width = 10 + i;
    obj.height = 20 + i;
    obj.x = 100 + 10 * i;
    obj.y = 200 + 10 * i;
    obj.vX = 0.5f * static_cast<float>(i);
    obj.vY = -0.5f * static_cast<float>(i);
    obj.p = 0.9f;
    in.objects.push_back(obj);
}

// Encode data.
uint8_t data[1024];
int size = 0;
if (!in.encode(data, 1024, size))
{
    std::cout << "Can't encode data" << std::endl;
    return false;
}

// Decode data.
rapidpixel::ObjectDetectorParams out;
if (!out.decode(data, size))
{
    std::cout << "Can't decode data" << std::endl;
    return false;
}

Read params from JSON file and write to JSON file

ObjectDetector library depends on ConfigReader library which provides method to read params from JSON file and to write params to JSON file. Example of writing and reading params to JSON file:

// Prepare params.
rapidpixel::ObjectDetectorParams in;
in.logMode = 111;
in.classNames = { "apple", "banana", "orange", "pineapple", "strawberry",
                  "watermelon", "lemon", "peach", "pear", "plum" };
for (int i = 0; i < 5; ++i)
{
    rapidpixel::Object obj;
    obj.id = i;
    obj.type = i % 3;
    obj.width = 10 + i;
    obj.height = 20 + i;
    obj.x = 100 + 10 * i;
    obj.y = 200 + 10 * i;
    obj.vX = 0.5f * static_cast<float>(i);
    obj.vY = -0.5f * static_cast<float>(i);
    obj.p = 0.9f;
    in.objects.push_back(obj);
}

// Write params to file.
rapidpixel::ConfigReader inConfig;
if (!inConfig.set(in, "ObjectDetectorParams") ||
    !inConfig.writeToFile("ObjectDetectorParams.json"))
{
    std::cout << "Can't write config file" << std::endl;
    return false;
}

// Read params from file.
rapidpixel::ConfigReader outConfig;
if (!outConfig.readFromFile("ObjectDetectorParams.json"))
{
    std::cout << "Can't open config file" << std::endl;
    return false;
}

rapidpixel::ObjectDetectorParams out;
if (!outConfig.get(out, "ObjectDetectorParams"))
{
    std::cout << "Can't read params from file" << std::endl;
    return false;
}

ObjectDetectorParams.json will look like:

{
    "ObjectDetectorParams": {
        "initString": "",
        "logMode": 111,
        "frameBufferSize": 1,
        "minObjectWidth": 4,
        "maxObjectWidth": 128,
        "minObjectHeight": 4,
        "maxObjectHeight": 128,
        "minXSpeed": 0.0,
        "maxXSpeed": 30.0,
        "minYSpeed": 0.0,
        "maxYSpeed": 30.0,
        "minDetectionProbability": 0.5,
        "xDetectionCriteria": 1,
        "yDetectionCriteria": 1,
        "resetCriteria": 1,
        "sensitivity": 0.039999999105930328,
        "scaleFactor": 1,
        "numThreads": 1,
        "type": 0,
        "enable": true,
        "custom1": 0.0,
        "custom2": 0.0,
        "custom3": 0.0,
        "classNames": [
            "apple",
            "banana",
            "orange",
            "pineapple",
            "strawberry",
            "watermelon",
            "lemon",
            "peach",
            "pear",
            "plum"
        ]
    }
}

Build and connect to your project

Build stand alone

The repository does not contain sub-repositories. The libraries Ged depends on (Frame, ConfigReader and ObjectDetector) must be cloned into the same folder as Ged. The benchmark and the demo application also need the SimpleUI library in the same folder, the demo application also needs the SimpleFileDialog, VEncoderJPEG and VideoRecorderMp4 libraries, the demo application and the example need the VSourceFile library and the libraries it depends on (VSource, VCodec, VDecoderH264, VDecoderH265, VDecoderJPEG and FormatConverter). The folder structure must be:

<work folder>
    Ged
    Frame
    ConfigReader
    ObjectDetector
    SimpleFileDialog
    SimpleUI
    VSource
    VCodec
    VDecoderH264
    VDecoderH265
    VDecoderJPEG
    FormatConverter
    VSourceFile
    VEncoderJPEG
    VideoRecorderMp4

The library needs a C++17 compiler with OpenMP support. On Linux the window of the demo application needs the X11 development packages (libx11-dev and libxext-dev); without them the SimpleUI library builds a window which shows nothing. Typical commands to build the library, the example, the benchmark and the demo application (use the URLs of the repositories provided to you):

git clone <Ged repository URL>
git clone <Frame repository URL>
git clone <ConfigReader repository URL>
git clone <ObjectDetector repository URL>
git clone <SimpleFileDialog repository URL>
git clone <SimpleUI repository URL>
git clone <VSource repository URL>
git clone <VCodec repository URL>
git clone <VDecoderH264 repository URL>
git clone <VDecoderH265 repository URL>
git clone <VDecoderJPEG repository URL>
git clone <FormatConverter repository URL>
git clone <VSourceFile repository URL>
git clone <VEncoderJPEG repository URL>
git clone <VideoRecorderMp4 repository URL>
cd Ged
mkdir build
cd build
cmake ..
make

On Windows use cmake --build . --config Release instead of make. If Ged is the top-level CMake project, the main CMakeLists.txt file adds the folders ../Frame, ../ConfigReader and ../ObjectDetector (in this order) before the library itself, the folder ../SimpleUI before the benchmark and the demo application, the folders ../VSource, ../VCodec, ../VDecoderH264, ../VDecoderH265, ../VDecoderJPEG, ../FormatConverter and ../VSourceFile (in this order) before the demo application and the example, and the folders ../SimpleFileDialog, ../VEncoderJPEG and ../VideoRecorderMp4 before the demo application. If a folder does not exist CMake stops with an error. To build only the library switch off the applications: cmake .. -D_SUBMODULE_CACHE_OVERWRITE=OFF -D_GED_BENCHMARK=OFF -D_GED_DEMO=OFF -D_GED_EXAMPLE=OFF.

CMake options

Table 8 - CMake options of the repository. In the options below ${PARENT} is the prefix defined by the top-level project (empty if Ged is the top-level project).

Option Default Description
${PARENT}_GED ON Build the library.
${PARENT}_GED_BENCHMARK ON (stand alone), OFF (included in other project) Build the benchmark application.
${PARENT}_GED_DEMO ON (stand alone), OFF (included in other project) Build the demo application.
${PARENT}_GED_EXAMPLE ON (stand alone), OFF (included in other project) Build the example.
${PARENT}_SUBMODULE_CACHE_OVERWRITE ON Forced rewriting of the options above in the CMake cache. The top-level project sets it to OFF when it includes Ged as a sub-repository.
CMAKE_BUILD_TYPE Release (if not specified) Build type.

Connect to your project

If you want to connect Ged library to your CMake project as source code, you can do the following. For example, if your repository has structure:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp

Add the repositories of Ged and of the libraries it depends on as sub-repositories of your repository (use the URLs of the repositories provided to you):

cd <your repository folder>
git submodule add <Frame repository URL> 3rdparty/Frame
git submodule add <ConfigReader repository URL> 3rdparty/ConfigReader
git submodule add <ObjectDetector repository URL> 3rdparty/ObjectDetector
git submodule add <Ged repository URL> 3rdparty/Ged

In your repository folder the 3rdparty/Frame, 3rdparty/ConfigReader, 3rdparty/ObjectDetector and 3rdparty/Ged folders will be created. Create CMakeLists.txt file in 3rdparty folder. The file adds the libraries in the order of their dependencies (Frame, ConfigReader, ObjectDetector, Ged). CMakeLists.txt should be containing:

cmake_minimum_required(VERSION 3.13)

################################################################################
## 3RD-PARTY
## dependencies for the project
################################################################################
project(3rdparty LANGUAGES CXX)

################################################################################
## SETTINGS
## basic 3rd-party settings before use
################################################################################
# To inherit the top-level architecture when the project is used as a submodule.
SET(PARENT ${PARENT}_YOUR_PROJECT_3RDPARTY)
# Disable self-overwriting of parameters inside included subdirectories.
SET(${PARENT}_SUBMODULE_CACHE_OVERWRITE OFF CACHE BOOL "" FORCE)

################################################################################
## CONFIGURATION
## 3rd-party submodules configuration
################################################################################
SET(${PARENT}_SUBMODULE_FRAME                           ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_FRAME)
    SET(${PARENT}_FRAME                                 ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_FRAME_TEST                            OFF CACHE BOOL "" FORCE)
endif()

SET(${PARENT}_SUBMODULE_CONFIG_READER                   ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_CONFIG_READER)
    SET(${PARENT}_CONFIG_READER                         ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_CONFIG_READER_EXAMPLES                OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_CONFIG_READER_TESTS                   OFF CACHE BOOL "" FORCE)
endif()

SET(${PARENT}_SUBMODULE_OBJECT_DETECTOR                 ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_OBJECT_DETECTOR)
    SET(${PARENT}_OBJECT_DETECTOR                       ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_OBJECT_DETECTOR_TEST                  OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_OBJECT_DETECTOR_EXAMPLE               OFF CACHE BOOL "" FORCE)
endif()

SET(${PARENT}_SUBMODULE_GED                             ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_GED)
    SET(${PARENT}_GED                                   ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_GED_BENCHMARK                         OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_GED_DEMO                              OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_GED_EXAMPLE                           OFF CACHE BOOL "" FORCE)
endif()

################################################################################
## INCLUDING SUBDIRECTORIES
## Adding subdirectories according to the 3rd-party configuration
################################################################################
if (${PARENT}_SUBMODULE_FRAME)
    add_subdirectory(Frame)
endif()

if (${PARENT}_SUBMODULE_CONFIG_READER)
    add_subdirectory(ConfigReader)
endif()

if (${PARENT}_SUBMODULE_OBJECT_DETECTOR)
    add_subdirectory(ObjectDetector)
endif()

if (${PARENT}_SUBMODULE_GED)
    add_subdirectory(Ged)
endif()

File 3rdparty/CMakeLists.txt adds the folders to your project and excludes the test applications, examples, the benchmark and the demo application from compiling (by default they are excluded from compiling if the libraries are included as sub-repositories). The new structure of your repository:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    CMakeLists.txt
    Frame
    ConfigReader
    ObjectDetector
    Ged

Next, you need to include the ‘3rdparty’ folder in the main CMakeLists.txt file of your repository. Add the following string at the end of your main CMakeLists.txt:

add_subdirectory(3rdparty)

Next, you have to include Ged library in your src/CMakeLists.txt file:

target_link_libraries(${PROJECT_NAME} Ged)

The library target passes its include folders, the libraries it depends on and the OpenMP flags to your target. Ged does not add the libraries it depends on when it is included in another project: your project must add Frame, ConfigReader and ObjectDetector (as the 3rdparty/CMakeLists.txt file above does). Without them the build stops with a compiler error like ObjectDetector.h: No such file or directory.

Portability

The library does not use SIMD instructions and does not serialize data. The motion mask is calculated in parallel threads with OpenMP. The library compiles the floating point calculations without fused multiply-add (GCC and Clang option -ffp-contract=off): the results are the same on x86-64 and on ARM64. The library, the example, the benchmark and the demo application are built without warnings by GCC 12 and 13, Clang 15 and 18 (Linux x86-64), GCC 13 and Clang 18 (Linux aarch64) and MSVC 19.51 (Windows x86-64 and x86). The results of the library were compared on Linux x86-64 (GCC, Clang), Linux aarch64 (GCC, Clang, qemu) and Windows x86-64 and x86 (MSVC): they are the same. On 32-bit platforms the processing buffers of the largest frames (for example 8192x8192 pixels) do not fit in the address space: detect(…) returns FALSE.

Example

A simple application shows how to use the Ged library. The application reads the video file “test.mp4” with the VSourceFile library, which decodes the video frames (H264, HEVC, JPEG; MP4, AVI or elementary stream files) and plays the file in a loop. The application performs object detection in every frame (NV12 pixel format, the output format of the decoders) and prints the detected objects. The application stops with exit code -1 if a call fails (for example the file can not be opened); stop it with Ctrl+C.

#include <cstdint>
#include <iostream>
#include <string>
#include <vector>
#include "Ged.h"
#include "VSourceFile.h"

int main()
{
    // Open video file "test.mp4". The video source library decodes the video
    // frames and plays the file in a loop.
    rapidpixel::VSourceFile videoSource;
    std::string initString = "test.mp4";
    if (!videoSource.openVSource(initString))
    {
        std::cout << "Can't open video file " << initString << std::endl;
        return -1;
    }

    // Create detector and set params.
    rapidpixel::Ged detector;
    if (!detector.setParam(rapidpixel::ObjectDetectorParam::MIN_OBJECT_WIDTH, 4) ||
        !detector.setParam(rapidpixel::ObjectDetectorParam::MAX_OBJECT_WIDTH, 96) ||
        !detector.setParam(rapidpixel::ObjectDetectorParam::MIN_OBJECT_HEIGHT, 4) ||
        !detector.setParam(rapidpixel::ObjectDetectorParam::MAX_OBJECT_HEIGHT, 96) ||
        !detector.setParam(rapidpixel::ObjectDetectorParam::SENSITIVITY, 10))
    {
        return -1;
    }

    // Video frame. NV12 is the pixel format of the decoders of the video source
    // library: the decoded frames are not converted.
    rapidpixel::Frame frame;
    frame.fourcc = rapidpixel::Fourcc::NV12;
    // Maximum time to wait for a video frame, milliseconds.
    const int32_t frameTimeoutMsec = 5000;

    // Main loop. The application stops if a call fails (stop it with Ctrl+C).
    while (true)
    {
        // Wait for the next video frame.
        if (!videoSource.getFrame(frame, frameTimeoutMsec))
        {
            std::cout << "No video frame" << std::endl;
            return -1;
        }

        // Detect objects.
        if (!detector.detect(frame))
        {
            std::cout << "Can't process video frame" << std::endl;
            return -1;
        }

        // Print detected objects.
        const std::vector<rapidpixel::Object> objects = detector.getObjects();
        std::cout << "Frame " << frame.frameId << ": " << objects.size()
                  << " objects" << std::endl;
        for (const rapidpixel::Object& object : objects)
        {
            std::cout << "    ID " << object.id << " x " << object.x << " y "
                      << object.y << " width " << object.width << " height "
                      << object.height << std::endl;
        }
    }
}

Benchmark

Benchmark application located in /benchmark folder and is intended to check performance (processing time per frame) on particular platform. After start user should enter detector params (an invalid value stops the application). The benchmark application will show processing time per frame until it is stopped (Ctrl+C). Benchmark output example:

=================================================
Ged v2.2.4 test application
=================================================

Image size - 1280
Object size - 720
Number of objects - 5
Random background(y/n) - y
Default detector parameters:
FRAME_BUFFER_SIZE = 30
SENSITIVITY = 10
X_DETECTION_CRITERIA = 1
Y_DETECTION_CRITERIA = 1
RESET_CRITERIA = 6
SCALE_FACTOR = 1
NUM_THREADS = 1
---------------------------
Default params (y/n) - n
FRAME_BUFFER_SIZE - 30
SENSITIVITY - 10
X_DETECTION_CRITERIA - 1
Y_DETECTION_CRITERIA - 1
RESET_CRITERIA - 6
SCALE_FACTOR - 1
NUM_THREADS - 1
Time Ged 65.7 msec
Time Ged 1.51 msec
Time Ged 1.485 msec
Time Ged 4.59 msec

Demo application

Demo application overview

The demo application is intended to evaluate the performance of the Ged C++ library. User can check performance of Ged library on any video file with H264, HEVC or JPEG video (MP4, MOV, AVI or elementary stream files). It serves as an example of using the Ged library. The application uses the VSourceFile library to read and decode the video file, the SimpleUI library for the window, the parameter sliders and the drawing and the VideoRecorderMp4 library to record MP4 video files.

Launch and user interface

The demo application does not require installation. The demo application compiled for Windows OS x64 (Windows 10 and newer). The demo application for Linux OS can be provided upon request. Table 9 shows list of files of demo application.

Table 9 - List of files of demo application (example for Windows OS).

File Description
GedDemo.exe Demo application executable file for windows OS.
GedDemo.json Demo application config file.
VC_redist.x64 Installer of necessary system libraries for Windows x64.
src/main.cpp Source code.
test.mp4 Test video file.

To launch demo application run GedDemo.exe executable file on Windows x64 OS or run commands on Linux:

sudo chmod +x GedDemo
./GedDemo

If a message about missing system libraries appears (on Windows OS) when launching the application, you must install the VC_redist.x64.exe program, which will install the system libraries required for operation. Config file GedDemo.json included video capture params (videoSource field) and object detector parameters (all others). If the demo application does not find the file after startup, it will create it with default parameters. Config file content:

{
    "Params": {
        "videoSource": "file dialog",
        "objectDetector": {
            "initString": "",
            "logMode": 0,
            "frameBufferSize": 30,
            "minObjectWidth": 2,
            "maxObjectWidth": 128,
            "minObjectHeight": 2,
            "maxObjectHeight": 128,
            "minXSpeed": -1.0,
            "maxXSpeed": -1.0,
            "minYSpeed": -1.0,
            "maxYSpeed": -1.0,
            "minDetectionProbability": -1.0,
            "xDetectionCriteria": 1,
            "yDetectionCriteria": 1,
            "resetCriteria": 6,
            "sensitivity": 10.0,
            "scaleFactor": 1,
            "numThreads": 1,
            "type": -1,
            "enable": true,
            "custom1": -1.0,
            "custom2": -1.0,
            "custom3": -1.0,
            "classNames": []
        }
    }
}

Table 10 - Parameters description.

Field Type Description
videoSource string Video source initialization string. If the parameter is set to “file dialog”, then after start the program will offer to select video file via file selection dialog. Otherwise the parameter is a full file name (e.g. “test.mp4”): H264, HEVC or JPEG video in MP4, MOV, M4V or AVI files or in elementary stream files (.h264, .264, .h265, .hevc, .jpg, .jpeg, .mjpg, .mjpeg). The software plays the video with repetition i.e. when the end of the video file is reached, playback starts again.
classNames list of strings Not used. A list of object class names used in detectors that recognize different object classes. Ged does not recognize object classes: the ‘type’ field of the objects is 0.
custom1 float Not used. Can have any value.
custom2 float Not used. Can have any value.
custom3 float Not used. Can have any value.
enable bool enable / disable detector.
frameBufferSize int Frame buffer size. Valid values from 1 to 128. If the frame buffer size is 1, the library converts the frame format to GRAY and processes it. If the buffer size is greater than 1, the library will calculate the average value of brightness (converted to GRAY format) of each pixel of the frame over several frames (FRAME_BUFFER_SIZE) and use it as input data. In this way, the original video is time smoothed. This allows you to detect objects in the video with high noise.
initString string Not used. Can have any value.
logMode int Not used. Can have any value.
maxObjectHeight int Maximum object height to be detected, pixels. To be detected object’s height must be <= maxObjectHeight. Default value 128.
maxObjectWidth int Maximum object width to be detected, pixels. To be detected object’s width must be <= maxObjectWidth. Default value 128.
minXSpeed float Not used. Can have any value.
maxXSpeed float Not used. Can have any value.
minYSpeed float Not used. Can have any value.
maxYSpeed float Not used. Can have any value.
minDetectionProbability float Not used. Can have any value.
minObjectHeight int Minimum object height to be detected, pixels. To be detected object’s height must be >= minObjectHeight. Default value 2.
minObjectWidth int Minimum object width to be detected, pixels. To be detected object’s width must be >= minObjectWidth. Default value 2.
numThreads int Num threads. Number of threads for parallel computing. Multiple threads are used to compute the binary motion mask.
resetCriteria int The number of consecutive video frames in which the object was not detected, so it will be excluded from the internal object list. The internal object list is used to associate newly detected objects with objects detected in the previous video frame. Internal objects list allow to library keep static object ID from frame to frame. If object not detected during number of frames < resetCriteria next time of detection will not change of object ID.
scaleFactor int Frame scaling factor for processing purposes. Reduce the image size by scaleFactor times horizontally and vertically for faster processing. Allows to increase calculation speed but slightly reduces sensitivity.
sensitivity float Detection sensitivity. For Ged library this parameter means pixel brightness deviation threshold from 0 to 255 for calculation motion mask. The first processing step is the computation of the binary motion mask. The algorithm makes a decision about the presence of motion in a particular pixel also on the basis of changes in pixel brightness. The brightness threshold is determined by the SENSITIVITY parameter. The lower this parameter is, the less contrasty objects will be detected, but there will be more influence of video noise. The larger this parameter is, the more contrast objects must have in order to be detected. The normal value for thermal cameras is 15. The normal value for daylight cameras is 10.
type int Not used. Can have any value.
xDetectionCriteria int Track detection criteria, frames. Shows how many frames the objects must be detected to be included into results. xDetectionCriteria and yDetectionCriteria set one common value (yDetectionCriteria is set last).
yDetectionCriteria int Track detection criteria, frames. Shows how many frames the objects must be detected to be included into results. xDetectionCriteria and yDetectionCriteria set one common value (yDetectionCriteria is set last).

After starting the application (running the executable file) the user should select the video file in the dialog box (if parameter “videoSource” in config file is set to “file dialog”). After that the user will see the user interface as shown below. The window shows the sliders of the detector parameters (top), the original video with detection results (left) and binary motion mask (right). Each detected object is represented by red rectangle with the object ID above it. The window can be resized: the video is scaled to the window.

ged_demo_application_interface

Control

To control the application, it is necessary that the main video display window was active (in focus), and also it is necessary that the English keyboard layout was activated without CapsLock mode. The program is controlled by the keyboard, the sliders and the mouse (detection ROI control).

Table 11 - Control buttons.

Button Description
ESC Exit the application.
SPACE Reset motion detector.
R Start/stop video recording. When video recording is enabled, the files dst_[date and time].mp4 (source video with detected objects), mask_[date and time].mp4 (binary motion mask) and mix_[date and time].mp4 (source video and binary motion mask) are created in the working directory of the application. The videos have the size of the window image and the frame rate of the video file. To stop the recording, press the R key again. During video recording, the application shows a warning message.

The sliders above the video change the detector parameters: frame buffer size (1 - 128), sensitivity (0 - 255), detection criteria (X and Y, 1 - 100), reset criteria (1 - 1000), minimum and maximum object width and height (1 - 1024) and scale factor (1 - 9). The sliders start from the values of the config file. If a value of the config file is out of the range of a slider, the slider shows the nearest end of its range and the detector keeps the value of the config file until the slider is moved.

The user can set the detection mask (mark a rectangular area where objects are to be detected). In order to set a rectangular detection area it is necessary to draw a line with the mouse with the left button pressed from the left-top corner of the required detection area to the right-bottom corner. The detection area is marked in cyan color as shown in the image.

Table of contents