fmd_web_logo

Fmd C++ library

v1.0.0

Overview
Versions
Library files
Key features and capabilities
Hardware and platform support
Supported pixel formats
Library principles
Fmd class description
Data structures
ObjectDetectorParams class description
Build and connect to your project
Example
Test application

Overview

Fmd (Flow Motion Detector) C++ library is designed for automatic detection of moving objects on videos captured by a moving camera. The library is implemented in C++ (C++17 standard). It is based on dense optical flow with automatic backend selection (OpenCV Farneback CUDA preferred, with NVIDIA VPI OFA / NVENC as a Jetson hardware fallback and OpenCV Farneback CPU as the universal fallback), 4-DOF affine ego-motion compensation, two parallel anomaly detectors over the residual flow (local N-sigma and two-window local contrast), an ensemble step with direction-coherence verification, and an IoU/centroid temporal tracker with hysteresis confirmation. Each instance of the Fmd C++ class performs frame-by-frame processing of a video data stream, processing each video frame independently. The library inherits its interface from the ObjectDetector (provides interface for object detector, source code included, Apache 2.0 license) class, offering flexible and customizable parameters. The library depends on open source OpenCV 4.x (linked, Apache 2.0 license), optionally NVIDIA VPI 3.x (linked when found, hardware-accelerated dense optical flow on Jetson platforms), and open source Logger library (source code included, Apache 2.0 license). The library is designed mainly for moving cameras (handheld, vehicle-mounted, gimbal) where conventional background-subtraction approaches fail because the background is non-stationary. It is suitable for detection of independently-moving foreground objects against complex, panning / translating backgrounds (drone detection from a moving platform, surveillance from a moving vehicle, target search from a moving aerial platform). The library uses C++17 standard.

Versions

Table 1 - Library versions.

Version	Release date	What’s new
1.0.0	06.06.2026	First version.

Library files

The library is supplied as source code. The user is provided with a set of files in the form of a CMake project (repository). The repository structure is shown below:

CMakeLists.txt --------------------- Main CMake file of the library.
3rdparty --------------------------- Folder with third-party libraries.
    CMakeLists.txt ----------------- CMake file to include third-party libraries.
    ObjectDetector ----------------- Folder with ObjectDetector library source code.
    Logger ------------------------- Folder with Logger library source code.
src -------------------------------- Folder with library source code.
    CMakeLists.txt ----------------- CMake file of the library.
    Fmd.h -------------------------- Main library header file.
    FmdVersion.h ------------------- Header file with library version.
    FmdVersion.h.in ---------------- Service CMake file to generate version header.
    Fmd.cpp ------------------------ C++ implementation file (pImpl wrapper).
    impl --------------------------- Folder with pipeline implementation.
        FmdImpl.h ------------------ Orchestrator class header.
        FmdImpl.cpp ---------------- Orchestrator implementation.
        FlowEngine.h --------------- Optical flow backend abstraction header.
        FlowEngine.cpp ------------- VPI / Farneback backend selection logic.
        MotionCompensator.h -------- 4-DOF affine ego-motion compensation header.
        MotionCompensator.cpp ------ Ego-motion compensation implementation.
        AnomalyNSigma.h ------------ Local N-sigma anomaly detector header.
        AnomalyNSigma.cpp ---------- Local N-sigma anomaly detector implementation.
        AnomalyLocalContrast.h ----- Local-contrast anomaly detector header.
        AnomalyLocalContrast.cpp --- Local-contrast anomaly detector implementation.
        Ensemble.h ----------------- Ensemble verification and scoring header.
        Ensemble.cpp --------------- Ensemble verification and scoring implementation.
        Clustering.h --------------- Hierarchical motion-feature clustering header.
        Clustering.cpp ------------- Hierarchical clustering implementation.
        TemporalTracker.h ---------- IoU/centroid temporal tracker header.
        TemporalTracker.cpp -------- Temporal tracker implementation.
        FlowUtils.h ---------------- Shared flow utilities header.
        FlowUtils.cpp -------------- Shared flow utilities implementation.
test ------------------------------- Folder for test application.
    CMakeLists.txt ----------------- CMake file for test application.
    FmdTest.cpp -------------------- Source C++ file of test application.
example ---------------------------- Folder for example application.
    CMakeLists.txt ----------------- CMake file of example application.
    main.cpp ----------------------- Source C++ file of example application.
static ----------------------------- Folder with static assets (logo, etc.).

Key features and capabilities

Table 2 - Key features and capabilities.

Parameter and feature	Description
Programming language	C++ (standard C++17).
Supported OS	Linux (Jetson and x86), Windows. The optical-flow backend chosen at runtime depends on hardware availability — see Hardware and platform support.
Shape of detected objects	The library is able to detect moving objects of any shape. The minimum and maximum height and width of the objects to be detected are set by the user in the library parameters.
Supported pixel formats	RGB24, BGR24, GRAY, YUV24, YUYV, UYVY, NV12, NV21, YV12, YU12. The library uses the GRAY format for processing. If the pixel format of the input frame is different from GRAY, the library pre-converts it to GRAY.
Maximum and minimum video frame size	The maximum size of video frames to be processed is 8192x8192 pixels. The size of the video frames has a significant impact on the computation time.
Coordinate system	The algorithm uses a window coordinate system with the zero point in the upper left corner of the video frame.
Calculation speed	The processing time per video frame depends on the computing platform and the selected optical-flow backend. On Jetson Orin with the VPI OFA backend dense optical flow at 1080p runs in single-digit milliseconds; on a CPU-only x86 system Farneback at the same resolution typically takes tens of milliseconds.
Discreteness of computed coordinates	The algorithm calculates the object bounding box for each detected object. The increment for position and size of the bounding box is 1 pixel.
Detection algorithm	Dense optical flow → 4-DOF affine ego-motion compensation → two parallel anomaly detectors over the residual flow → ensemble verification (direction coherence + magnitude uniformity) → IoU/centroid temporal tracker with hysteresis confirmation. Object velocity is reported in a camera-stabilized reference frame (ego-motion removed).
Working conditions	Designed mainly for moving cameras (handheld, vehicle, gimbal, drone). The 4-DOF (translation + rotation + uniform scale) ego-motion model handles typical camera pan / roll / forward-flight motion. Rapidly changing camera depth (strong zooming, fast forward translation through cluttered scene) reduces accuracy. It is recommended to use the example application to evaluate the quality of the algorithm in specific situations.

Hardware and platform support

The library compiles and runs on any platform that provides a C++17 toolchain and OpenCV 4.x. Optical-flow performance, however, depends on which hardware-accelerated backends are available at build time. The CMake configuration detects them automatically and sets the corresponding preprocessor flags inside the Fmd target:

CMake check	Preprocessor flag	Code paths enabled
`find_package(vpi 3.0)`	`FMD_HAVE_VPI=1`	VPI OFA / NVENC dense optical flow.
`TARGET opencv_cudaoptflow`	`FMD_HAVE_OPENCV_CUDA=1`	`cv::cuda::FarnebackOpticalFlow` (GPU).
(none required)	—	`cv::calcOpticalFlowFarneback` (CPU).

When a flag is not defined the corresponding backend code is replaced with a stub that returns “backend not compiled in”, so the same source tree builds cleanly on any platform — the runtime selection logic simply skips unavailable paths.

The optical-flow backend is selected automatically on the first detect() call in the following preference order:

Priority	Backend	Notes
1	Farneback CUDA	OpenCV CUDA Farneback. Preferred when `opencv_cudaoptflow` is built — it dominates VPI on both speed and quality.
2	VPI OFA	Jetson Orin dedicated Optical Flow Accelerator. No-CUDA fallback.
3	VPI NVENC	Encoder-based dense optical flow (older Jetson). No-CUDA fallback.
4	Farneback CPU	Universal fallback. Always available.

Table — typical platform matrix.

Platform	VPI	CUDA Farneback	CPU Farneback	Auto-pick
NVIDIA Jetson Orin — JetPack 6	OFA (HW)	optional, requires custom OpenCV build with CUDA	yes	Farneback CUDA if built, else VPI OFA
Older Jetson (Xavier, Nano-Maxwell) — JetPack 5	NVENC (HW)	optional	yes	Farneback CUDA if built, else VPI NVENC
x86 Linux + NVIDIA GPU + OpenCV CUDA build	no	yes	yes	Farneback CUDA
x86 Linux + system OpenCV (no CUDA)	no	no	yes	Farneback CPU
Windows + NVIDIA GPU + OpenCV CUDA build	no	yes	yes	Farneback CUDA
Windows + prebuilt OpenCV	no	no	yes	Farneback CPU
macOS / ARM Linux (non-Jetson)	no	no	yes	Farneback CPU

To force a specific backend, set the TYPE parameter through setParam:

TYPE value	Meaning
-1	Auto (default).
0	VPI (OFA / NVENC, whichever works).
1	OpenCV Farneback (CPU).
2	OpenCV Farneback (CUDA).

getParam(TYPE) returns the actually selected backend after the first detect() call: 0 for any VPI backend, 1 for Farneback CPU, 2 for Farneback CUDA, -1 if the engine has not been initialized yet.

Supported pixel formats

The Frame library (vendored through the ObjectDetector submodule) contains the Fourcc enum which defines supported pixel formats (Frame.h file). Fmd processes RAW pixel formats only (the compressed JPEG, H264 and HEVC entries are not supported). The library uses the intensity (GRAY) channel for processing; if the input frame is in another supported format it is pre-converted to GRAY. Fourcc enum declaration:

enum class Fourcc
{
    /// RGB 24bit pixel format.
    RGB24 = MAKE_FOURCC_CODE('R', 'G', 'B', '3'),
    /// BGR 24bit pixel format.
    BGR24 = MAKE_FOURCC_CODE('B', 'G', 'R', '3'),
    /// YUYV 16bits per pixel format.
    YUYV  = MAKE_FOURCC_CODE('Y', 'U', 'Y', 'V'),
    /// UYVY 16bits per pixel format.
    UYVY  = MAKE_FOURCC_CODE('U', 'Y', 'V', 'Y'),
    /// Grayscale 8bit.
    GRAY  = MAKE_FOURCC_CODE('G', 'R', 'A', 'Y'),
    /// YUV 24bit per pixel format.
    YUV24  = MAKE_FOURCC_CODE('Y', 'U', 'V', '3'),
    /// NV12 pixel format.
    NV12  = MAKE_FOURCC_CODE('N', 'V', '1', '2'),
    /// NV21 pixel format.
    NV21  = MAKE_FOURCC_CODE('N', 'V', '2', '1'),
    /// YU12 (YUV420) - Planar pixel format.
    YU12 = MAKE_FOURCC_CODE('Y', 'U', '1', '2'),
    /// YV12 (YVU420) - Planar pixel format.
    YV12 = MAKE_FOURCC_CODE('Y', 'V', '1', '2'),
    /// JPEG compressed format.
    JPEG  = MAKE_FOURCC_CODE('J', 'P', 'E', 'G'),
    /// H264 compressed format.
    H264  = MAKE_FOURCC_CODE('H', '2', '6', '4'),
    /// HEVC compressed format.
    HEVC  = MAKE_FOURCC_CODE('H', 'E', 'V', 'C')
};

Bytes layout of supported RAW pixel formats. Example of a 4x4 pixels image.

YUV24	GRAY
YUYV	UYVY
NV12	NV21
YU12	YV12

For planar luminance formats (GRAY, NV12, NV21, YV12, YU12) Fmd reads the Y plane directly without copy or color conversion — these are the most efficient input formats.

Library principles

The object detection algorithm consists of the following sequential steps executed on every detect() call:

Acquire the source video frame and convert it to GRAY format (grayscale).
Compute dense optical flow between the previous and the current grayscale frame. The backend is selected automatically (VPI OFA / NVENC / Farneback CUDA / Farneback CPU).
Estimate a 4-DOF affine model (translation + rotation + uniform scale) of the camera ego-motion. The model is fit on spatially-distributed CR feature points with two-stage outlier rejection (exclusion regions from confirmed tracks + textured-background median flow).
Compute the residual flow — per-pixel optical flow with the affine prediction subtracted. Pixels belonging to the static background have near-zero residual; independently moving objects produce non-zero residual.
Run two anomaly detectors over the residual flow in parallel:
- Local N-sigma: per-pixel adaptive threshold over a sliding window of the residual magnitude.
- Local Contrast: two-window approach — magnitude minus its local mean, then an adaptive threshold on the resulting contrast map.
Combine the two detector masks via an ensemble step: per-component verification by flow direction coherence and magnitude uniformity (hard filter), followed by additive scoring (NSigma weight + LocalContrast weight × cluster factor) and merge of co-directional overlapping detections.
Pass the verified detections to an IoU/centroid temporal tracker with hysteresis confirmation and a hard motion check. Positions are accumulated in a camera-stabilized reference frame derived from the per-frame affine ego-motion, so velocity and displacement measurements isolate object motion from camera motion.
The resulting confirmed objects for the current frame can be retrieved using the getObjects() method.

The library is available as source code. To use the library as source code, developers must incorporate the library files into their project. The usage sequence is:

Include the library files in the project.
Create an instance of the Fmd class. For multiple parallel cameras, create multiple instances.
If necessary, modify the default library parameters using the setParam(…) method.
Create a Frame class object for the input frame.
Call the detect(…) method to identify objects on each frame.
Retrieve the detected objects using the getObjects() method.

Fmd class description

Fmd class declaration

Fmd class declared in Fmd.h file. Fmd inherits from the ObjectDetector interface and lives in the cr::detector namespace. Class declaration:

class Fmd : public ObjectDetector
{
public:

    /// Get library version.
    static std::string getVersion();

    /// Class constructor.
    Fmd();

    /// Class destructor.
    ~Fmd();

    /// Init object detector with the given parameters.
    bool initObjectDetector(ObjectDetectorParams& params) override;

    /// Set parameter value.
    bool setParam(ObjectDetectorParam id, float value) override;

    /// Get parameter value.
    float getParam(ObjectDetectorParam id) override;

    /// Get full parameters snapshot.
    void getParams(ObjectDetectorParams& params) override;

    /// Get list of detected objects from the last detect() call.
    std::vector<Object> getObjects() override;

    /// Execute command.
    bool executeCommand(ObjectDetectorCommand id) override;

    /// Perform detection on a single video frame.
    bool detect(cr::video::Frame& frame) override;

    /// Set detection mask. Pixels with value 0 are ignored.
    bool setMask(cr::video::Frame mask) override;

    /// Decode binary command and execute it.
    bool decodeAndExecuteCommand(uint8_t* data, int size) override;
};

getVersion method

The getVersion() method returns string of current version of Fmd class. Method declaration:

static std::string getVersion();

Method can be used without Fmd class instance:

std::cout << "Fmd class version: " << cr::detector::Fmd::getVersion() << std::endl;

Console output:

Fmd class version: 1.0.0

initObjectDetector method

The initObjectDetector(…) method initializes the detector with a complete ObjectDetectorParams structure. Internally it calls setParam(...) for each supported field so the same validation rules apply. Method declaration:

bool initObjectDetector(ObjectDetectorParams& params) override;

Parameter	Description
params	ObjectDetectorParams structure (see ObjectDetectorParams class description).

Returns: TRUE if the detector was initialized, FALSE otherwise.

setParam method

The setParam(…) method sets a new parameter value. Method declaration:

bool setParam(ObjectDetectorParam id, float value) override;

Parameter	Description
id	Parameter ID according to the ObjectDetectorParam enum (see ObjectDetectorParam enum).
value	Parameter value. Valid range depends on the parameter ID — see Table 3 below.

Returns: TRUE if the parameter was set, FALSE if the id is unsupported, the value is out of range, or the parameter is read-only (PROCESSING_TIME_MCS).

Table 3 - Fmd parameter mapping. Some standard fields of ObjectDetectorParams have a Fmd-specific interpretation (noted in the Notes column).

Parameter	Access	Valid range	Default	Notes
LOG_MODE	read / write	0..3	0	Logger destination. 0 - disable, 1 - file only, 2 - console only, 3 - console + file.
FRAME_BUFFER_SIZE	read / write	1..	1	Accepted for interface compatibility; Fmd processes optical flow between two consecutive frames and does not maintain a multi-frame buffer.
MIN_OBJECT_WIDTH	read / write	1..8192	4	Output filter: drop detected objects with width < value.
MAX_OBJECT_WIDTH	read / write	1..8192	128	Output filter: drop detected objects with width > value.
MIN_OBJECT_HEIGHT	read / write	1..8192	4	Output filter: drop detected objects with height < value.
MAX_OBJECT_HEIGHT	read / write	1..8192	128	Output filter: drop detected objects with height > value.
MIN_X_SPEED	read / write	0.0..	0.0	Output filter: drop detected objects with `\\|vX\\|` < value (pixels/frame, in stabilized reference frame).
MAX_X_SPEED	read / write	0.0..	30.0	Output filter: drop detected objects with `\\|vX\\|` > value.
MIN_Y_SPEED	read / write	0.0..	0.0	Output filter: drop detected objects with `\\|vY\\|` < value.
MAX_Y_SPEED	read / write	0.0..	30.0	Output filter: drop detected objects with `\\|vY\\|` > value.
MIN_DETECTION_PROBABILITY	read / write	0.0..1.0	0.0	Stored but currently not applied: Fmd reports `Object::p = 1.0` for confirmed tracks.
X_DETECTION_CRITERIA	read / write	1..256	4	Score-sum threshold to confirm a track. The tracker’s internal `confirmThreshold` is set to `max(X_DETECTION_CRITERIA, Y_DETECTION_CRITERIA)`.
Y_DETECTION_CRITERIA	read / write	1..256	4	See X_DETECTION_CRITERIA.
RESET_CRITERIA	read / write	1..256	3	Number of consecutive unmatched frames after which a track is dropped (tracker’s `maxMissed`).
SENSITIVITY	read / write	0.1..50.0	3.0	Fmd-specific meaning: N-sigma multiplier shared by both anomaly detectors. Lower value = more sensitive (more false positives). Typical range 2.0..5.0.
SCALE_FACTOR	read / write	1	1	Fmd does not downsample internally. Only the value `1` is accepted; use upstream resizing if you need lower-resolution processing.
NUM_THREADS	read / write	0..	1	Accepted for interface compatibility; Fmd relies on OpenCV / VPI internal threading, so this value is not consulted directly.
PROCESSING_TIME_MCS	read only	—	—	Last `detect()` call duration, microseconds.
TYPE	read / write	-1..2	-1	Fmd-specific meaning: optical-flow backend. -1 - auto, 0 - VPI, 1 - Farneback CPU, 2 - Farneback CUDA. Reading this parameter returns the actually selected backend after the first `detect()` call.
MODE	read / write	0/1	1	0 - off (detect returns true with empty objects), 1 - on. Switching off triggers a tracker reset.
CUSTOM_1 / CUSTOM_2 / CUSTOM_3	read / write	any	0.0	Fmd-specific tuning levers, each applied only when set > 0 (otherwise the internal default is kept): CUSTOM_1 — minimum net displacement, the stabilized-frame motion floor a track must exceed before it can confirm (suppresses quasi-stationary clutter); CUSTOM_2 — cluster reference area used by the co-moving-group penalty (smaller = stronger penalty); CUSTOM_3 — score-history window length (longer rewards sustained detections over brief flickers).

getParam method

The getParam(…) method retrieves a parameter value. Method declaration:

float getParam(ObjectDetectorParam id) override;

Parameter	Description
id	Parameter ID according to ObjectDetectorParam enum.

Returns: the parameter value, or -1 if the id is unsupported.

getParams method

The getParams(…) method copies the full ObjectDetectorParams snapshot, including the most recent objects list. Method declaration:

void getParams(ObjectDetectorParams& params) override;

Parameter	Description
params	Output ObjectDetectorParams structure.

executeCommand method

The executeCommand(…) method executes a control command. Method declaration:

bool executeCommand(ObjectDetectorCommand id) override;

Parameter	Description
id	Command ID according to ObjectDetectorCommand enum (see ObjectDetectorCommand enum).

Returns: TRUE if the command was accepted, FALSE otherwise.

Table 4 - Command behavior.

Command	Description
RESET	Resets the optical-flow engine, the ego-motion compensator, the temporal tracker, and the cached previous frame. The next detect() call will be a priming call (returns TRUE with an empty objects list).
ON	Enables the detector (equivalent to `setParam(MODE, 1)`).
OFF	Disables the detector (equivalent to `setParam(MODE, 0)`). While disabled, detect() returns TRUE but reports an empty objects list; a tracker reset is also scheduled so that re-enabling produces a clean state.

detect method

The detect(…) method performs detection on a single video frame. Method declaration:

bool detect(cr::video::Frame& frame) override;

Parameter	Description
frame	Source video frame (Frame class). Supported pixel formats: GRAY, NV12, NV21, YV12, YU12, BGR24, RGB24, YUV24, UYVY, YUYV.

Returns: TRUE if the frame was processed successfully (note: returns TRUE on the very first frame too, with an empty objects list — that frame is used to prime the optical-flow buffer). Returns FALSE on an invalid frame or an unsupported pixel format.

After detect() the list of confirmed objects is available via getObjects().

setMask method

The setMask(…) method sets a detection mask. Pixels of the mask equal to 0 are ignored — detections whose centers fall inside those regions are dropped before tracking. Method declaration:

Note. The detector processes the entire frame; it does not crop or inset any border internally. Masking frame-edge artifacts (overlays, OSD, fixed borders) or other regions that produce false positives is the caller’s responsibility — use this mask to exclude them.

bool setMask(cr::video::Frame mask) override;

Parameter	Description
mask	Binary mask (Frame object). Supported pixel formats: GRAY, NV12, NV21, YV12, YU12. Only the Y plane is read.

Returns: TRUE if the mask was accepted, FALSE on an invalid frame or an unsupported pixel format.

decodeAndExecuteCommand method

The decodeAndExecuteCommand(…) method decodes a binary command (produced by ObjectDetector::encodeSetParamCommand(...) or ObjectDetector::encodeCommand(...)) and executes it via setParam or executeCommand. Method declaration:

bool decodeAndExecuteCommand(uint8_t* data, int size) override;

Parameter	Description
data	Pointer to a buffer with the encoded command.
size	Size of the buffer in bytes.

Returns: TRUE if the command was decoded and executed, FALSE otherwise.

Data structures

ObjectDetectorCommand enum

enum class ObjectDetectorCommand
{
    RESET = 1,
    ON,
    OFF
};

See executeCommand method.

ObjectDetectorParam enum

enum class ObjectDetectorParam
{
    LOG_MODE = 1,
    FRAME_BUFFER_SIZE,
    MIN_OBJECT_WIDTH,
    MAX_OBJECT_WIDTH,
    MIN_OBJECT_HEIGHT,
    MAX_OBJECT_HEIGHT,
    MIN_X_SPEED,
    MAX_X_SPEED,
    MIN_Y_SPEED,
    MAX_Y_SPEED,
    MIN_DETECTION_PROBABILITY,
    X_DETECTION_CRITERIA,
    Y_DETECTION_CRITERIA,
    RESET_CRITERIA,
    SENSITIVITY,
    SCALE_FACTOR,
    NUM_THREADS,
    PROCESSING_TIME_MCS,
    TYPE,
    MODE,
    CUSTOM_1,
    CUSTOM_2,
    CUSTOM_3
};

See Table 3 for the Fmd-specific meaning of each parameter.

Object structure

struct Object
{
    int id{0};            // stable track ID
    int frameId{0};       // input frame.frameId of the frame the object was detected on
    int type{0};          // object class — always 0 for Fmd (no classifier)
    int width{0};         // bounding box width, pixels
    int height{0};        // bounding box height, pixels
    int x{0};             // bounding box top-left x, pixels (image coords)
    int y{0};             // bounding box top-left y, pixels
    float vX{0.0f};       // horizontal velocity, pixels/frame, stabilized
    float vY{0.0f};       // vertical velocity, pixels/frame, stabilized
    float p{0.0f};        // detection probability (always 1.0 for Fmd confirmed tracks)
};

The vX and vY components are measured in a camera-stabilized reference frame: the per-frame affine ego-motion estimated by the detector is removed before differencing positions, so they reflect motion of the object relative to the static scene, not relative to the moving camera.

ObjectDetectorParams class description

ObjectDetectorParams is defined by the ObjectDetector interface library. See its documentation for the complete declaration, the JSON_READABLE macro, and the encode() / decode() serialization methods. Fmd maps the subset of fields listed in Table 3 onto its internal pipeline knobs; fields that are not in that table are stored verbatim but do not influence the algorithm.

Build and connect to your project

Before compiling you have to install OpenCV for your system. VPI is optional and is detected automatically at configure time when its CMake config is on the search path.

Installation on Linux:

sudo apt-get install -y build-essential cmake libopencv-dev

For hardware-accelerated dense optical flow on NVIDIA Jetson Orin platforms, install VPI 3.x from the JetPack repository. On x86 Linux desktops VPI is typically not used for dense flow, so the library falls back to Farneback automatically.

Installation on Windows:

Download a prebuilt OpenCV 4.x release for Windows and extract it to any folder (recommended path C:/libs/openCV_4.x.x).
Add path to OpenCV to Windows system variables. Settings → System → Advanced system settings → Environment Variables.
Create a new variable in System variables with name OpenCV_DIR and path to your OpenCV build folder (for example C:/libs/openCV_4.12.0/build) and save changes.
Optionally add %OpenCV_DIR%/x64/vc16/bin to PATH so applications find opencv_*.dll at runtime.
A Windows reboot is sometimes required.

VPI is not officially available on Windows; on Windows the library always uses one of the Farneback backends.

Enabling the CUDA optical-flow backend (recommended):

The production backend is OpenCV Farneback CUDA (opencv_cudaoptflow module). Stock distributions — apt libopencv-dev and the prebuilt Windows release — do not include the CUDA modules, so with them the library silently falls back to CPU Farneback. To get the GPU backend, build OpenCV ≥ 4.9.0 (4.8 fails to build the CUDA modules against CUDA 12.x) together with opencv_contrib and the cudaoptflow module.

Prerequisites: an NVIDIA GPU, a recent NVIDIA driver, and the CUDA Toolkit 12.x (nvcc). Set CUDA_ARCH_BIN to the GPU’s compute capability:

GPU	`CUDA_ARCH_BIN`
RTX 20xx (Turing)	7.5
RTX 30xx (Ampere)	8.6
RTX 40xx (Ada)	8.9
Jetson Orin	8.7

Linux — a helper script is provided (scripts/build_opencv_cuda.sh). Stage the sources, then run it with ARCH set to your CUDA_ARCH_BIN:

git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv.git         ~/opencv_src/opencv
git clone --branch 4.10.0 --depth 1 https://github.com/opencv/opencv_contrib.git ~/opencv_src/opencv_contrib
ARCH=8.9 bash scripts/build_opencv_cuda.sh

It installs a self-contained OpenCV to ~/opencv-cuda and leaves the system OpenCV untouched. Point Fmd at it:

cmake -S . -B build -DOpenCV_DIR=$HOME/opencv-cuda/lib/cmake/opencv4

Windows — build OpenCV + opencv_contrib with CMake and Visual Studio:

cmake -S opencv -B opencv_build -G "Visual Studio 17 2022" -A x64 ^
  -D CMAKE_INSTALL_PREFIX=C:/opencv-cuda ^
  -D OPENCV_EXTRA_MODULES_PATH=opencv_contrib/modules ^
  -D WITH_CUDA=ON -D CUDA_ARCH_BIN=8.9 ^
  -D BUILD_LIST=core,imgproc,imgcodecs,videoio,highgui,video,calib3d,features2d,flann,photo,cudev,cudaarithm,cudawarping,cudaoptflow ^
  -D WITH_GSTREAMER=OFF
cmake --build opencv_build --config Release --target INSTALL -j 8

WITH_GSTREAMER=OFF avoids a glib alloca.h build break under MSVC; FFmpeg (bundled automatically) covers video I/O. Then point Fmd at the install:

cmake -S . -B build -G "Visual Studio 17 2022" -A x64 -D OpenCV_DIR=C:/opencv-cuda/x64/vc17/lib

When configured correctly, CMake prints Fmd: opencv_cudaoptflow available — CUDA Farneback enabled, and getParam(TYPE) returns 2 (Farneback CUDA) after the first detect() call.

Build:

cd Fmd
mkdir build
cd build
cmake ..
cmake --build . --config Release

If you want connect Fmd library to your CMake project as source code you can make follow. For example, if your repository has structure:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp

Create folder 3rdparty in your repository and copy Fmd repository folder to 3rdparty folder. New structure:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    Fmd

Create 3rdparty/CMakeLists.txt with:

cmake_minimum_required(VERSION 3.13)

################################################################################
## 3RD-PARTY
## dependencies for the project
################################################################################
project(3rdparty LANGUAGES CXX)

################################################################################
## SETTINGS
################################################################################
SET(PARENT ${PARENT}_YOUR_PROJECT_3RDPARTY)
SET(${PARENT}_SUBMODULE_CACHE_OVERWRITE OFF CACHE BOOL "" FORCE)

################################################################################
## CONFIGURATION
################################################################################
SET(${PARENT}_SUBMODULE_FMD                             ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_FMD)
    SET(${PARENT}_FMD                                   ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_FMD_TEST                              OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_FMD_EXAMPLE                           OFF CACHE BOOL "" FORCE)
endif()

################################################################################
## INCLUDING SUBDIRECTORIES
################################################################################
if (${PARENT}_SUBMODULE_FMD)
    add_subdirectory(Fmd)
endif()

Include the 3rdparty folder in your main CMakeLists.txt:

add_subdirectory(3rdparty)

Link Fmd in your src/CMakeLists.txt:

target_link_libraries(${PROJECT_NAME} Fmd)

The ObjectDetector interface (with Frame and ConfigReader) is transitively pulled in as a dependency of Fmd.

Example

The example application opens a video file with OpenCV, wraps each frame in a cr::video::Frame, runs the Fmd detector through the ObjectDetector interface and draws the bounding boxes of the detected moving objects. Source code:

#include <opencv2/opencv.hpp>
#include <Fmd.h>

int main(void)
{
    // Open video file "test.mp4".
    cv::VideoCapture videoSource;
    if (!videoSource.open("test.mp4"))
        return -1;

    // Create detector and configure.
    cr::detector::Fmd detector;
    detector.setParam(cr::detector::ObjectDetectorParam::MIN_OBJECT_WIDTH,  4);
    detector.setParam(cr::detector::ObjectDetectorParam::MAX_OBJECT_WIDTH,  128);
    detector.setParam(cr::detector::ObjectDetectorParam::MIN_OBJECT_HEIGHT, 4);
    detector.setParam(cr::detector::ObjectDetectorParam::MAX_OBJECT_HEIGHT, 128);
    detector.setParam(cr::detector::ObjectDetectorParam::SENSITIVITY,       3.0f);

    cv::Mat frameBgrOpenCv;
    int frameIdx = 0;

    // Main loop.
    while (true)
    {
        videoSource >> frameBgrOpenCv;
        if (frameBgrOpenCv.empty())
        {
            detector.executeCommand(cr::detector::ObjectDetectorCommand::RESET);
            videoSource.set(cv::CAP_PROP_POS_FRAMES, 0);
            continue;
        }

        // Wrap the OpenCV frame in cr::video::Frame.
        cr::video::Frame bgrFrame;
        bgrFrame.width   = frameBgrOpenCv.cols;
        bgrFrame.height  = frameBgrOpenCv.rows;
        bgrFrame.size    = bgrFrame.width * bgrFrame.height * 3;
        bgrFrame.data    = frameBgrOpenCv.data;
        bgrFrame.fourcc  = cr::video::Fourcc::BGR24;
        bgrFrame.frameId = static_cast<uint32_t>(frameIdx++);

        // Run detection.
        detector.detect(bgrFrame);

        // Retrieve and draw confirmed objects.
        std::vector<cr::detector::Object> objects = detector.getObjects();
        for (const auto& obj : objects)
        {
            cv::rectangle(frameBgrOpenCv,
                          cv::Rect(obj.x, obj.y, obj.width, obj.height),
                          cv::Scalar(0, 255, 0), 2);
        }

        cv::imshow("Fmd", frameBgrOpenCv);
        if (cv::waitKey(1) == 27)
            return 0;
    }
    return 0;
}

Test application

The test application (Fmd/test/FmdTest.cpp) is a smoke test for the Fmd library. It exercises construction, the getVersion() call, the setParam / getParam round-trip on representative parameters, the read-only-parameter contract for PROCESSING_TIME_MCS, the priming and frame-size semantics of detect() on synthetic frames containing a translating bright square, the OFF / ON mode toggle, and the RESET command. The test forces the Farneback CPU backend so it runs on any platform regardless of hardware availability. To run on Linux or Windows:

cd <build folder>/bin
./FmdTest          # Linux
FmdTest.exe        # Windows

After start you will see output similar to:

Fmd version: 1.0.0
Backend used: 1 object_count_last_frame=1
All checks passed.

A non-zero exit code indicates one or more checks failed; failing lines are printed to stderr before the summary.