mmag_web_logo

Motion Magnificator C++ library

v5.0.0

Overview
Versions
Library files
Key features and capabilities
Supported pixel formats
Library principles
MotionMagnificator class description
Data structures
- VFilterCommand enum
- VFilterParam enum
VFilterParams class description
Build and connect to your project
Portability
Example
Benchmark
Demo application

Overview

MotionMagnificator C++ library computes the amplitude of the motion of every pixel of digital video frames and gives it as a separate grayscale image or shows it in the intensity channel of the frames, so that motion which is not visible to the naked eye in the original video becomes visible. When the intensity channel shows the motion, the colour channels of the frames can show the original intensity of the scene with a palette (Arctic, Ironbow or Rainbow), so that the scene of cameras without colour (for example thermal cameras) stays recognisable; the colours do not depend on the motion. The library is implemented in C++ (C++17 standard) and uses OpenMP (version >= 2.0) for multithreading if it is available on the particular platform. The library is suitable for various types of cameras (daylight, SWIR, MWIR and LWIR). Each instance of the MotionMagnificator C++ class performs frame-by-frame processing of one video stream. The library is intended for fixed (or slowly moving) cameras and for PTZ cameras while pan, tilt and zoom do not change. It works on different platforms: it detects at build time whether OpenMP is available (without OpenMP the processing runs on one CPU core, with OpenMP the user chooses the number of threads) and selects at run time the SIMD instructions (AVX2, SSSE3, NEON) the processor supports. The library depends on the VFilter library (provides the interface and the data structures of video filters), which depends on the Frame and ConfigReader libraries. Additionally the demo application depends on the SimpleFileDialog (provides a dialog to open files), SimpleUI (provides a window to display video with a simple user interface), VSourceFile (provides reading and decoding of video files) and VideoRecorderMp4 (provides recording of video to MP4 files) libraries, and the example depends on SimpleUI and VSourceFile; see Build stand alone.

MotionMagnificator is general-purpose software: it is not tied to any sector or application, and the developer of a program decides what the program does with it.

Versions

Table 1 - Library versions.

Version	Release date	What’s new
1.0.0	13.09.2021	- First version.
2.0.0	23.10.2023	- New interface created. - New algorithm implemented.
2.0.1	13.11.2023	- Frame class updated.
2.0.2	28.12.2023	- Demo application updated. - Examples updated. - Benchmark added.
2.1.0	23.02.2024	- Interface update
3.0.0	27.02.2024	- Interface changed according to VFilter interface class.
3.0.1	24.03.2024	- VFilter class updated. - SimpleFileDialog class updated. - Documentation updated.
3.1.0	15.04.2024	- Performance improved. - Documentation updated.
3.1.1	21.05.2024	- Submodules updated. - Documentation updated.
3.2.0	18.07.2024	- Added support for OpenMP. - Added support for OpenCL. - CMake architecture updated.
3.2.1	15.12.2024	- Fix OpenMP linkage in Linux.
3.2.2	13.08.2025	- Fix bug with building on Windows.
3.2.3	01.10.2026	- Remove OpenCV from example and demo applications.
4.0.0	06.10.2026	- New algorithm implemented (with SIMD optimization). - OpenCL support removed. - getDevices() method removed. - Number of threads is set by CUSTOM_1 parameter.
4.1.0	07.10.2026	- Implementation restructured (own image kernels, helper libraries ImgProc and MathProc removed); the processing result is bit-identical to the 4.0.0 Release builds made with GCC or Clang. - SIMD level (AVX2 + FMA, SSSE3, NEON, portable) selected at run time: one binary runs on every x86-64 processor; no -march=native / fast-math builds. - Input checks: frame and mask size fields, NaN and infinite parameter values, invalid initVFilter() parameters change nothing. - processFrame() calls on one object are serialised; the class cannot be copied. - Build hardening, zero compiler warnings, generated header in the build folder. - Example, demo and benchmark fixed (no endless loops, input checks). - Documentation updated.
4.1.1	12.10.2026	- Submodules updated.
4.1.2	19.10.2026	- Git sub-repositories removed: the libraries that the repository depends on (VFilter, Frame, ConfigReader and the libraries of the demo application and of the example) are expected in the folders next to the repository when it is built stand alone. - CMake files, documentation and code comments updated. - The processing result is bit-identical to 4.1.1.
5.0.0	26.10.2026	- New processing implemented. - TYPE parameter: 0 - the frame is not changed, 1 - the intensity channel of the frame shows the motion (128 plus the motion signal); without a palette the colour channels are not changed. - New getAmplitudes() method: amplitudes of the motion of the last processed frame as a GRAY image. - CUSTOM_2 parameter: palette of the colour channels with TYPE 1 (0 - off, 1 - Arctic, 2 - Ironbow, 3 - Rainbow) computed from the original intensity of the pixels, for cameras without colour (thermal cameras). - Pixels excluded by the mask keep their intensity and get the amplitude 0. - The processing uses at most one thread per processor; the working memory follows the frame size. - Unused FloatImage::copyFrom() method removed; exception handling and frame buffer checks in benchmark, demo and example completed. - Example: TYPE 1, level slider and palette switching (P key). - Demo application: TYPE 1 always (the T key removed), processing switched on and off with the E key, palette switching (P key). - CMake: a hardening flag is used only if the compiler takes it without a warning (32-bit ARM with Clang). - Documentation and code comments updated.

Library files

The library is supplied as source code only. The user is given a set of files in the form of a CMake project (repository). The repository structure is shown below:

CMakeLists.txt ----------------------- Main CMake file of the library.
src ---------------------------------- Folder with library source code.
    CMakeLists.txt ------------------- CMake file of the library.
    MotionMagnificator.h ------------- Main library header file.
    MotionMagnificatorConfig.h.in ---- File for CMake to generate version header and config.
    MotionMagnificator.cpp ----------- C++ class source file.
    impl ----------------------------- Folder with implementation files.
        MotionMagnificatorImpl.h ----- Parameters, commands, mask, amplitudes and frame checks (header).
        MotionMagnificatorImpl.cpp --- Parameters, commands, mask, amplitudes and frame checks.
        MagnificationEngine.h -------- Processing engine (header).
        MagnificationEngine.cpp ------ Processing engine.
        FrameFormats.h --------------- Supported pixel formats and frame checks (header).
        FrameFormats.cpp ------------- Supported pixel formats and frame checks.
        ColourPalette.h -------------- Palettes of the colour samples (header).
        ColourPalette.cpp ------------ Palettes of the colour samples.
        MaskScaler.h ----------------- Scaling of the mask (header).
        MaskScaler.cpp --------------- Scaling of the mask.
        FloatImage.h ----------------- Float image class (header).
        FloatImage.cpp --------------- Float image class.
        kernels ---------------------- Folder with the image processing kernels.
            KernelSet.h -------------- Interface of the kernels of one SIMD level.
            ImageKernels.h ----------- Kernels for any SIMD level (template).
            KernelDispatch.cpp ------- Run-time selection of the SIMD level.
            KernelsPortable.cpp ------ Kernels in plain C++.
            KernelsSsse3.cpp --------- Kernels for x86 processors with SSSE3.
            KernelsAvx2.cpp ---------- Kernels for x86 processors with AVX2 and FMA.
            KernelsNeon.cpp ---------- Kernels for ARM processors with NEON.
demo --------------------------------- Folder for demo application files.
    CMakeLists.txt ------------------- CMake file for demo application.
    main.cpp ------------------------- Source C++ file of demo application.
example ------------------------------ Folder for simple example.
    CMakeLists.txt ------------------- CMake file of example.
    main.cpp ------------------------- Source C++ file of example.
benchmark ---------------------------- Folder with benchmark application.
    CMakeLists.txt ------------------- CMake file of benchmark application.
    main.cpp ------------------------- Source C++ file of benchmark application.
static ------------------------------- Folder with the images of this document.

Key features and capabilities

Table 2 - Key features and capabilities.

Parameter and feature	Description
Programming language	C++ (standard C++17) and OpenMP (version >= 2.0, if supported on the particular platform). The library depends only on the VFilter library (and its dependencies Frame and ConfigReader).
Supported OS	Compatible with any operating system that supports the C++ compiler (C++17 standard). Tested on Linux x86-64 and x86 (GCC 12 and 13, Clang 15 and 18), Linux AArch64, ARMv7 and s390x (emulated) and Windows x86-64 (MinGW-w64, run with Wine).
Movement type	Any kind of movement. The library is best suited to small motion that is hard to see with the unaided eye.
Supported pixel formats	GRAY, YUV24, YUYV, UYVY, NV12, NV21, YV12, YU12. The library uses the pixel intensity for video processing. If the pixel format of the image doesn’t include an intensity channel it should be converted to a proper format before processing.
Maximum and minimum video frame size	From 1x1 to 65536x65536 pixels; the frame data must fit the size field of the Frame class (up to 2^31 - 1 bytes). Very small frames are processed but give small amplitudes. The size of the video frames has a significant impact on the computation speed.
Calculation speed	The processing time per video frame depends on the computing platform used. It can be measured with the benchmark and the demo application.
Output	TYPE 0 (default): the frame is not changed, the amplitudes of the motion are given as a separate GRAY image by the getAmplitudes(…) method. TYPE 1: in addition the intensity channel of the frame shows 128 plus the motion signal (0 - 255); the colour channels are not changed or, with a palette (CUSTOM_2 1 - 3: Arctic, Ironbow, Rainbow), show the original intensity of the scene. The LEVEL parameter sets the scale of the amplitudes; LEVEL and CUSTOM_2 can be changed at any time.
Working conditions	The library is designed for fixed (or slowly moving) cameras with any background conditions.
Supported devices	The library works on CPU in one-thread mode or in multithreading mode (using OpenMP). The number of threads is set by the CUSTOM_1 parameter; at most one thread per processor available to the program is used. SIMD instructions (AVX2 + FMA, SSSE3, NEON) are used when the processor supports them.

Supported pixel formats

The Frame library which the MotionMagnificator library depends on contains the Fourcc enum which defines the pixel formats (Frame.h file). MotionMagnificator supports only the formats with an intensity channel (GRAY, YUV24, YUYV, UYVY, NV12, NV21, YV12, YU12). Only the intensity samples are used for the processing; with TYPE 1 the intensity samples of a frame are changed and the colour samples stay as they are, unless a palette is set (CUSTOM_2, see Library principles). Fourcc enum declaration:

enum class Fourcc
{
    /// RGB 24bit pixel format.
    RGB24 = MAKE_FOURCC_CODE('R', 'G', 'B', '3'),
    /// BGR 24bit pixel format.
    BGR24 = MAKE_FOURCC_CODE('B', 'G', 'R', '3'),
    /// YUYV 16bits per pixel format.
    YUYV  = MAKE_FOURCC_CODE('Y', 'U', 'Y', 'V'),
    /// UYVY 16bits per pixel format.
    UYVY  = MAKE_FOURCC_CODE('U', 'Y', 'V', 'Y'),
    /// Grayscale 8bit.
    GRAY  = MAKE_FOURCC_CODE('G', 'R', 'A', 'Y'),
    /// YUV 24bit per pixel format.
    YUV24  = MAKE_FOURCC_CODE('Y', 'U', 'V', '3'),
    /// NV12 pixel format.
    NV12  = MAKE_FOURCC_CODE('N', 'V', '1', '2'),
    /// NV21 pixel format.
    NV21  = MAKE_FOURCC_CODE('N', 'V', '2', '1'),
    /// YU12 (YUV420) - Planar pixel format.
    YU12 = MAKE_FOURCC_CODE('Y', 'U', '1', '2'),
    /// YV12 (YVU420) - Planar pixel format.
    YV12 = MAKE_FOURCC_CODE('Y', 'V', '1', '2'),
    /// JPEG compressed format.
    JPEG  = MAKE_FOURCC_CODE('J', 'P', 'E', 'G'),
    /// H264 compressed format.
    H264  = MAKE_FOURCC_CODE('H', '2', '6', '4'),
    /// HEVC compressed format.
    HEVC  = MAKE_FOURCC_CODE('H', 'E', 'V', 'C')
};

Table 3 - Bytes layout of supported RAW pixel formats. Example of 4x4 pixels image.

YUV24	GRAY
YUYV	UYVY
NV12	NV21
YU12	YV12

Library principles

The library computes for every pixel of a video frame the amplitude of its motion from the intensity samples of the current and of the previous frames of the stream. The amplitudes of a frame are available in two ways:

As a separate image: after processFrame(…) the getAmplitudes(…) method copies the amplitudes of the frame into a GRAY frame of the same size (0 - 255, 0 means no motion).
In the frame itself, if the TYPE parameter is 1: the intensity samples are replaced by 128 plus the motion signal, clamped to 0 - 255 (128 means no motion). Without a palette (CUSTOM_2 0, default) the colour samples are not changed, so the colours of the scene stay visible. With TYPE 0 (default) the frame is not changed.

Frames of cameras without colour (for example thermal cameras) have no colours to keep: with TYPE 1 they show only the motion. For them the CUSTOM_2 parameter sets a palette: before the intensity samples are replaced, the colour samples of the whole frame are computed from the original intensity of the pixels, so that the colours show the scene. The colours depend only on the original intensity, not on the motion. The palettes are those of the ColorProcessor library (Y is the original intensity, 0 - 255):

0 - no palette (default): the colour samples are not changed.
1 - Arctic: U = 128, V = 255 - Y.
2 - Ironbow: U = 128, V = 2 * Y for Y < 128, otherwise 255.
3 - Rainbow: U = 255 - 2 * Y and V = 0 for Y < 128, otherwise U = 0 and V = 2 * (Y - 128).

Pixels that share colour samples (two neighbouring pixels in YUYV and UYVY, a block of 2 x 2 pixels in NV12, NV21, YU12 and YV12) get the colour of the rounded mean of their intensities. The palette is used only with TYPE 1; GRAY frames have no colour samples and are not coloured. A palette replaces the colours of a colour camera as well.

The LEVEL parameter (0 - 100) sets the scale of the amplitudes: higher values give larger amplitudes. The first frame, the first frame after a RESET or ON command and the first frame of a new frame size start the processing: their amplitudes are 0 (with TYPE 1 the intensity is 128). Where the mask is 0 the amplitude is 0 and the frame keeps its intensity (a palette sets the colour samples of the whole frame).

The library is available as source code only. To use it, include the library files in your project (see Build and connect to your project). The usage sequence for the library is as follows:

Include the library files in the project.
Create an instance of the MotionMagnificator C++ class. For multiple cameras create one MotionMagnificator object per camera.
If necessary, modify the default library parameters (TYPE, LEVEL, CUSTOM_1, CUSTOM_2) using the setParam(…) method.
Create a Frame class object for the input frame and, if the amplitudes are needed as an image, a Frame object for them.
Call the processFrame(…) method for every frame of the video stream and, if needed, the getAmplitudes(…) method after it.

MotionMagnificator class description

MotionMagnificator class declaration

MotionMagnificator.h file contains MotionMagnificator class declaration:

namespace rapidpixel
{
/// Motion magnificator class.
class MotionMagnificator : public VFilter
{
public:

    /// Class constructor: mode on, type 0, level 40, one thread, no palette.
    MotionMagnificator();

    /// Class destructor.
    ~MotionMagnificator() override;

    /// The class cannot be copied or moved.
    MotionMagnificator(MotionMagnificator const&) = delete;
    MotionMagnificator& operator=(MotionMagnificator const&) = delete;
    MotionMagnificator(MotionMagnificator&&) = delete;
    MotionMagnificator& operator=(MotionMagnificator&&) = delete;

    /// Get string of current library version.
    static std::string getVersion();

    /// Initialize motion magnificator.
    bool initVFilter(VFilterParams& params) override;

    /// Set motion magnificator parameter.
    bool setParam(VFilterParam id, float value) override;

    /// Get motion magnificator parameter value.
    float getParam(VFilterParam id) override;

    /// Get all motion magnificator parameters.
    void getParams(VFilterParams& params) override;

    /// Execute action command.
    bool executeCommand(VFilterCommand id) override;

    /// Process frame.
    bool processFrame(rapidpixel::Frame& frame) override;

    /// Get the amplitudes of the last processed frame (not part of VFilter).
    bool getAmplitudes(rapidpixel::Frame& amplitudes);

    /// Set mask.
    bool setMask(rapidpixel::Frame mask) override;

    /// Decode and execute command.
    bool decodeAndExecuteCommand(uint8_t* data, int size) override;
};
}

All methods may be called from any thread. processFrame(…) calls on one object are serialised (a second call waits for the first one); the other methods do not wait for a running processFrame(…) call and take effect with the next frame (getAmplitudes(…) waits only while the amplitudes of a finished frame are stored). The methods do not throw exceptions; only the constructor may throw std::bad_alloc. The argument of setMask(…) is passed by value (VFilter interface): the copy of the mask is made by the calling code before the method runs and may throw std::bad_alloc.

getVersion method

The getVersion() method returns string of current version of MotionMagnificator class. Method declaration:

static std::string getVersion();

Method can be used without MotionMagnificator class instance. Example:

std::cout << "MotionMagnificator v: " << rapidpixel::MotionMagnificator::getVersion();

Console output:

MotionMagnificator v: 5.0.0

initVFilter method

The initVFilter(…) method initializes MotionMagnificator class by VFilterParams object which includes all video filter parameters. The method uses the fields mode, type, level, custom1 (number of threads; values below 1, such as the default 0, keep the current number of threads) and custom2 (palette; the default 0 switches the palette off). All values are checked first: if one of them is invalid the method returns FALSE and changes nothing. Method declaration:

bool initVFilter(VFilterParams& params) override;

Parameter	Value
params	VFilterParams class object.

Returns: TRUE if the video filter initialized or FALSE if not.

setParam method

The setParam(…) method designed to set new MotionMagnificator object parameter value. setParam(…) is thread-safe method. This means that the setParam(…) method can be safely called from any thread. Method declaration:

bool setParam(VFilterParam id, float value) override;

Parameter	Description
id	Parameter ID according to VFilterParam enum.
value	Parameter value. Value depends on parameter ID (see VFilterParam enum). NaN and infinite values are rejected.

Returns: TRUE if the parameter was set or FALSE if the value is out of range. Parameter IDs that the library does not use (CUSTOM_3, PROCESSING_TIME_MCSEC and unknown IDs) are accepted and ignored: the method returns TRUE.

getParam method

The getParam(…) method designed to obtain a parameter value. getParam(…) is thread-safe method. This means that the getParam(…) method can be safely called from any thread. Method declaration:

float getParam(VFilterParam id) override;

Parameter	Description
id	Parameter ID according to VFilterParam enum.

Returns: parameter value or -1 if the parameter is not supported.

getParams method

The getParams(…) method fills a VFilterParams object with the current parameters: mode, type, level, processingTimeMcSec, custom1 (number of threads) and custom2 (palette); the other fields are not changed. getParams(…) is thread-safe method. Method declaration:

void getParams(VFilterParams& params) override;

Parameter	Description
params	VFilterParams class object to fill.

executeCommand method

The executeCommand(…) method designed to execute a command. executeCommand(…) is thread-safe method. This means that the executeCommand(…) method can be safely called from any thread. Method declaration:

bool executeCommand(VFilterCommand id) override;

Parameter	Description
id	Command ID according to VFilterCommand enum.

Returns: TRUE if the command was executed or FALSE if not (unknown command ID).

processFrame method

The processFrame(…) method computes the amplitudes of the motion of the pixels of a frame and, with TYPE 1, replaces the intensity samples of the frame by 128 plus the motion signal (clamped to 0 - 255). With TYPE 1 and a palette (CUSTOM_2 1 - 3) the colour samples of the whole frame are first computed from the original intensity (see Library principles); without a palette they are not changed. With TYPE 0 the frame is not changed. The library provides thread-safe processFrame(…) method call: calls from several threads on one object are processed one after the other. Method declaration:

bool processFrame(rapidpixel::Frame& frame) override;

Parameter	Description
frame	Video Frame object for processing. Only GRAY, YUV24, YUYV, UYVY, NV12, NV21, YV12, YU12 formats are processed. The library uses the pixel intensity for video processing. If the pixel format of the image doesn’t include an intensity channel it should be converted to a proper format before processing. The width and the height must be from 1 to 65536 pixels and the size field must cover the intensity samples (width * height * bytes per pixel for YUV24, YUYV and UYVY, width * height for the other formats). With TYPE 1 and a palette the size field of NV12, NV21, YU12 and YV12 frames must also cover the colour samples: width * height + 2 * ((width + 1) / 2) * ((height + 1) / 2) bytes (the Frame class allocates this size for an even width and height).

Parameter

Description

frame

Video Frame object for processing. Only GRAY, YUV24, YUYV, UYVY, NV12, NV21, YV12, YU12 formats are processed. The library uses the pixel intensity for video processing. If the pixel format of the image doesn’t include an intensity channel it should be converted to a proper format before processing. The width and the height must be from 1 to 65536 pixels and the size field must cover the intensity samples (width * height * bytes per pixel for YUV24, YUYV and UYVY, width * height for the other formats). With TYPE 1 and a palette the size field of NV12, NV21, YU12 and YV12 frames must also cover the colour samples: width * height + 2 * ((width + 1) / 2) * ((height + 1) / 2) bytes (the Frame class allocates this size for an even width and height).

Returns: TRUE if the video frame was processed, FALSE if not (invalid frame or not enough memory; after a memory error the processing restarts with the next frame). The frame is not changed if FALSE is returned. If the filter is off (MODE 0) the method returns TRUE and does not change the frame.

The data pointer of the frame must point to at least size bytes (the library cannot check this); a frame that the method rejects is not accessed. The working memory (about 7.5 bytes per frame pixel, about 8.5 bytes with a mask) is allocated when the frame size changes and is reused for the next frames; the palette needs no working memory. The result is deterministic and does not depend on the number of threads.

getAmplitudes method

The getAmplitudes(…) method copies the amplitudes of the motion of the frame of the last processFrame(…) call into a frame. The method is not part of the VFilter interface. It can be called with any value of the TYPE parameter and from any thread. Method declaration:

bool getAmplitudes(rapidpixel::Frame& amplitudes);

Parameter	Description
amplitudes	Result: GRAY image of the size of the processed frame (data size width * height) with the frameId and sourceId of the processed frame. A pixel value is the amplitude of the motion of the pixel, from 0 (no motion) to 255. It is 0 where the mask is 0 and in a frame that starts the processing. If the frame already holds a GRAY image of the same size its memory is reused, otherwise new memory is allocated.

Parameter

Description

amplitudes

Result: GRAY image of the size of the processed frame (data size width * height) with the frameId and sourceId of the processed frame. A pixel value is the amplitude of the motion of the pixel, from 0 (no motion) to 255. It is 0 where the mask is 0 and in a frame that starts the processing. If the frame already holds a GRAY image of the same size its memory is reused, otherwise new memory is allocated.

Returns: TRUE if the amplitudes were copied. FALSE if the last processFrame(…) call gave no amplitudes (no call yet, the filter was off or the frame was rejected) or memory could not be allocated; the frame is not changed in that case.

Example:

rapidpixel::MotionMagnificator magnificator;
rapidpixel::Frame amplitudes;
...
if (magnificator.processFrame(frame) && magnificator.getAmplitudes(amplitudes))
{
    // amplitudes.data holds amplitudes.width * amplitudes.height bytes.
}

setMask method

The setMask(…) method sets the mask. The user can exclude any areas of the video frame from the processing. For this purpose the user can create an image of any size and configuration with GRAY (preferable), NV12, NV21, YV12 or YU12 pixel format. Mask image pixel values equal to 0 exclude the corresponding place of video frames: there the amplitude is 0 and the intensity of the frame is not changed (a palette sets the colour samples of the whole frame, also there). Any other mask pixel value includes the corresponding location of video frames. The mask is copied by the method and used from the next processFrame(…) call on. The method can be called either before video frame processing or during video frame processing. The argument is passed by value (VFilter interface), so the calling code makes a copy of the mask before the method runs; this copy allocates memory and may throw std::bad_alloc. Method declaration:

bool setMask(rapidpixel::Frame mask) override;

Parameter	Description
mask	Image of the mask (Frame object). Must have GRAY (preferable), NV12, NV21, YV12 or YU12 pixel format and a size field of at least width * height bytes. The size of the mask image can be any. If the size of the mask image differs from the size of processed frames, the mask is scaled by the library (bilinear interpolation; a frame pixel is processed if its scaled mask value is not 0).

Parameter

Description

mask

Image of the mask (Frame object). Must have GRAY (preferable), NV12, NV21, YV12 or YU12 pixel format and a size field of at least width * height bytes. The size of the mask image can be any. If the size of the mask image differs from the size of processed frames, the mask is scaled by the library (bilinear interpolation; a frame pixel is processed if its scaled mask value is not 0).

Returns: TRUE if the mask is accepted or FALSE if not (invalid pixel format, empty or inconsistent size fields, not enough memory); the previous mask stays in use in that case. The data pointer of the mask must point to at least size bytes (the library cannot check this).

encodeSetParamCommand method of VFilter class

The encodeSetParamCommand(…) static method encodes command to change any VFilter parameter value remote. To control any video filter remotely, the developer has to design his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VFilter class contains static methods for encoding the control command. The VFilter class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeSetParamCommand(…) designed to encode SET_PARAM command. Method declaration:

static void encodeSetParamCommand(uint8_t* data, int& size, VFilterParam id, float value);

Parameter	Description
data	Pointer to data buffer for encoded command. Must have size >= 11.
size	Size of encoded data. Size will be 11 bytes.
id	Parameter ID according to VFilterParam enum.
value	Parameter value.

encodeSetParamCommand(…) is static and can be used without VFilter class instance. This method used on client side (control system). Command encoding example:

// Buffer for encoded data.
uint8_t data[11];
// Size of encoded data.
int size = 0;
// Parameter value.
float outValue = 20.0f;
// Encode command.
rapidpixel::VFilter::encodeSetParamCommand(data, size, rapidpixel::VFilterParam::LEVEL, outValue);

encodeCommand method of VFilter class

The encodeCommand(…) static method encodes action command for VFilter remote control. To control any video filter remotely, the developer has to design his own protocol and according to it encode the command and deliver it over the communication channel. To simplify this, the VFilter class contains static methods for encoding the control command. The VFilter class provides two types of commands: a parameter change command (SET_PARAM) and an action command (COMMAND). encodeCommand(…) designed to encode COMMAND (action command). Method declaration:

static void encodeCommand(uint8_t* data, int& size, VFilterCommand id);

Parameter	Description
data	Pointer to data buffer for encoded command. Must have size >= 7.
size	Size of encoded data. Size will be 7 bytes.
id	Command ID according to VFilterCommand enum.

encodeCommand(…) is static and can be used without VFilter class instance. This method used on client side (control system). Command encoding example:

// Buffer for encoded data.
uint8_t data[7];
// Size of encoded data.
int size = 0;
// Encode command.
rapidpixel::VFilter::encodeCommand(data, size, rapidpixel::VFilterCommand::RESET);

decodeCommand method of VFilter class

The decodeCommand(…) static method decodes command on video filter side (on edge device). Method declaration:

static int decodeCommand(uint8_t* data, int size, VFilterParam& paramId, VFilterCommand& commandId, float& value);

Parameter	Description
data	Pointer to input command.
size	Size of command. Must be 11 bytes for SET_PARAM and 7 bytes for COMMAND (action command).
paramId	VFilter parameter ID according to VFilterParam enum. After decoding SET_PARAM command the method will return parameter ID.
commandId	VFilter command ID according to VFilterCommand enum. After decoding COMMAND the method will return command ID.
value	VFilter parameter value (after decoding SET_PARAM command).

Returns: 0 - in case decoding COMMAND (action command), 1 - in case decoding SET_PARAM command or -1 in case errors.

decodeAndExecuteCommand method

The decodeAndExecuteCommand(…) method decodes and executes command encoded by encodeSetParamCommand(…) and encodeCommand(…) methods on video filter side. The library provides thread-safe decodeAndExecuteCommand(…) method call. This means that the decodeAndExecuteCommand(…) method can be safely called from any thread. Method declaration:

bool decodeAndExecuteCommand(uint8_t* data, int size) override;

Parameter	Description
data	Pointer to input command.
size	Size of command. Must be 11 bytes for SET_PARAM or 7 bytes for COMMAND.

Returns: TRUE if command decoded (SET_PARAM or COMMAND) and executed (action command or set param command) successfully, FALSE otherwise (null pointer, size <= 0, damaged command, invalid parameter value or command ID).

Data structures

VFilterCommand enum

Enum declaration:

enum class VFilterCommand
{
    /// Reset video filter algorithm.
    RESET = 1,
    /// Enable filter.
    ON,
    /// Disable filter.
    OFF
};

Table 4 - Action commands description.

Command	Description
RESET	Restart the processing with the next frame (its amplitudes are 0).
ON	Switch the filter on. If it was off, the processing restarts with the next frame.
OFF	Switch the filter off: frames are passed through unchanged and getAmplitudes(…) gives no amplitudes.

VFilterParam enum

Enum declaration:

enum class VFilterParam
{
    /// Filter mode: 0 - off, 1 - on. Depends on implementation.
    MODE = 1,
    /// Enhancement level for particular filter, as a percentage from
    /// 0% to 100%. May have another meaning depends on implementation.
    LEVEL,
    /// Processing time in microseconds. Read only parameter.
    PROCESSING_TIME_MCSEC,
    /// Type of the filter. Depends on the implementation.
    TYPE,
    /// VFilter custom parameter. Custom parameters used when particular image
    /// filter has specific unusual parameter.
    CUSTOM_1,
    /// VFilter custom parameter. Custom parameters used when particular image
    /// filter has specific unusual parameter.
    CUSTOM_2,
    /// VFilter custom parameter. Custom parameters used when particular image
    /// filter has specific unusual parameter.
    CUSTOM_3
};

Table 5 - Params description.

Parameter	Access	Description
MODE	read / write	Mode: 0 - off, 1 - on. Default: 1. The value is truncated towards zero (for example 0.5 means off). If the filter is off, frames are passed through unchanged and no amplitudes are computed. Switching on restarts the processing.
LEVEL	read / write	Level in percent, from 0 to 100. Default: 40. Higher values give larger amplitudes.
PROCESSING_TIME_MCSEC	read only	Processing time of the last processed frame in microseconds. Used to check the performance of the library.
TYPE	read / write	Type of the output: 0 - the frame is not changed (the amplitudes are given by getAmplitudes(…)), 1 - the intensity samples of the frame are replaced by 128 plus the motion signal (clamped to 0 - 255; 128 means no motion), the colour samples are not changed or are set by the palette (CUSTOM_2). Default: 0. The value is truncated towards zero (for example 1.5 means 1). A change of the type keeps the processing state. The amplitudes are available with both types.
CUSTOM_1	read / write	Number of threads for computing, from 1 to 128 (truncated towards zero). Default: 1 (one thread). Multithreading is available only if OpenMP was detected on library compilation step. The processing uses at most one thread per processor available to the program (more threads would not run at the same time); the parameter keeps the value that was set. Processing results do not depend on the number of threads.
CUSTOM_2	read / write	Palette of the colour samples with TYPE 1: 0 - no palette, the colour samples are not changed; 1 - Arctic; 2 - Ironbow; 3 - Rainbow (see Library principles). The colour samples of the whole frame are computed from the original intensity of the pixels before it is replaced; the colours do not depend on the motion. Intended for cameras without colour (thermal cameras). Default: 0. The value is truncated towards zero (for example 2.5 means 2). A change of the palette keeps the processing state. Not used with TYPE 0 and for GRAY frames.
CUSTOM_3	read / write	Not used.

VFilterParams class description

VFilterParams class declaration

VFilterParams class is used to provide video filter parameters structure. Also VFilterParams provides possibility to write/read params from JSON files (JSON_READABLE macro) and provides methods to encode and decode params. VFilterParams interface class declared in VFilter.h file. Class declaration:

namespace rapidpixel
{
/// VFilter parameters class.
class VFilterParams
{
public:

    /// Filter mode: 0 - off, 1 - on. Depends on implementation.
    int mode{1};
    /// Enhancement level for particular filter, as a percentage from
    /// 0% to 100%. May have another meaning depends on implementation.
    float level{0.0f};
    /// Processing time in microseconds. Read only parameter.
    int processingTimeMcSec{0};
    /// Type of the filter. Depends on the implementation.
    int type{0};
    /// VFilter custom parameter. Custom parameters used when particular image
    /// filter has specific unusual parameter.
    float custom1{0.0f};
    /// VFilter custom parameter. Custom parameters used when particular image
    /// filter has specific unusual parameter.
    float custom2{0.0f};
    /// VFilter custom parameter. Custom parameters used when particular image
    /// filter has specific unusual parameter.
    float custom3{0.0f};

    /// Macro from ConfigReader to make params readable / writable from JSON.
    JSON_READABLE(VFilterParams, mode, level, type, custom1, custom2, custom3)

    /// Encode (serialize) params.
    bool encode(uint8_t* data, int bufferSize, int& size,
                VFilterParamsMask* mask = nullptr);

    /// Decode (deserialize) params.
    bool decode(uint8_t* data, int dataSize);
};
}

Table 6 - VFilterParams class fields description is related to VFilterParam enum description.

Field	type	Description
mode	int	Mode: 0 - off, 1 - on. If the filter is off, frames are passed through unchanged.
level	float	Level in percent, from 0 to 100. Note that the default value of the field (0) differs from the default level of the library (40).
processingTimeMcSec	int	Processing time of the last processed frame in microseconds. Read only parameter.
type	int	Type of the output: 0 - the frame is not changed, 1 - the intensity channel shows 128 plus the motion signal (see the TYPE parameter).
custom1	float	Number of threads for computing, from 1 to 128. Values less than 1 are ignored by the initVFilter(…) method (the number of threads does not change). Multithreading is available only if OpenMP was detected on library compilation step.
custom2	float	Palette of the colour samples with TYPE 1: 0 - no palette (default of the field and of the library), 1 - Arctic, 2 - Ironbow, 3 - Rainbow (see the CUSTOM_2 parameter). The initVFilter(…) method always applies it.
custom3	float	Not used.

Note: VFilterParams class fields listed in Table 6 have to reflect params set/get by methods setParam(…) and getParam(…).

Serialize VFilter params

VFilterParams class provides method encode(…) to serialize VFilter params. Serialization of VFilterParams is necessary in case when video filter parameters have to be sent via communication channels. Method provides options to exclude particular parameters from serialization. To do this method inserts binary mask (1 byte) where each bit represents particular parameter and decode(…) method recognizes it. Method declaration:

bool encode(uint8_t* data, int bufferSize, int& size, VFilterParamsMask* mask = nullptr);

Parameter	Value
data	Pointer to data buffer. Buffer size must be >= 32 bytes.
bufferSize	Data buffer size. Buffer size must be >= 32 bytes.
size	Size of encoded data.
mask	Parameters mask - pointer to VFilterParamsMask structure. VFilterParamsMask (declared in VFilter.h file) determines flags for each field (parameter) declared in VFilterParams class. If user wants to exclude any parameters from serialization, he can put a pointer to the mask. If the user wants to exclude a particular parameter from serialization, he should set the corresponding flag in the VFilterParamsMask structure.

Returns: TRUE if params encoded (serialized) or FALSE if not (buffer size < 32).

VFilterParamsMask structure declaration:

struct VFilterParamsMask
{
    bool mode{true};
    bool level{true};
    bool processingTimeMcSec{true};
    bool type{true};
    bool custom1{true};
    bool custom2{true};
    bool custom3{true};
};

Example without parameters mask:

// Prepare parameters.
rapidpixel::VFilterParams params;
params.level = 80.0f;

// Encode (serialize) params.
int bufferSize = 128;
uint8_t buffer[128];
int size = 0;
if (!params.encode(buffer, bufferSize, size))
{
    std::cout << "Can't encode params" << std::endl;
}

Example with parameters mask:

// Prepare parameters.
rapidpixel::VFilterParams params;
params.level = 80.0f;

// Prepare mask.
rapidpixel::VFilterParamsMask mask;
// Exclude level.
mask.level = false;

// Encode (serialize) params.
int bufferSize = 128;
uint8_t buffer[128];
int size = 0;
if (!params.encode(buffer, bufferSize, size, &mask))
{
    std::cout << "Can't encode params" << std::endl;
}

Deserialize VFilter params

VFilterParams class provides method decode(…) to deserialize params. Deserialization of VFilterParams is necessary in case when it is needed to receive params via communication channels. Method automatically recognizes which parameters were serialized by encode(…) method. Method declaration:

bool decode(uint8_t* data, int dataSize);

Parameter	Value
data	Pointer to data buffer with serialized params.
dataSize	Size of command data.

Returns: TRUE if params decoded (deserialized) or FALSE if not.

Example:

// Prepare parameters.
rapidpixel::VFilterParams params1;
params1.level = 80.0f;

// Encode (serialize) params.
int bufferSize = 128;
uint8_t buffer[128];
int size = 0;
if (!params1.encode(buffer, bufferSize, size))
{
    std::cout << "Can't encode params" << std::endl;
}

// Decode (deserialize) params.
rapidpixel::VFilterParams params2;
if (!params2.decode(buffer, size))
{
    std::cout << "Can't decode params" << std::endl;
}

Read params from JSON file and write to JSON file

VFilter depends on the ConfigReader library which provides methods to read params from a JSON file and to write params to a JSON file. Example of writing and reading params to JSON file:

// Prepare params.
rapidpixel::VFilterParams params1;
params1.mode = 1;
params1.level = 10.1f;
params1.processingTimeMcSec = 10;
params1.type = 2;
params1.custom1 = 22.3f;
params1.custom2 = 23.4f;
params1.custom3 = 24.5f;

// Save to JSON.
rapidpixel::ConfigReader configReader1;
if (!configReader1.set(params1, "Params"))
{
    std::cout << "Can't set params" << std::endl;
}
if (!configReader1.writeToFile("VFilterParams.json"))
{
    std::cout << "Can't write to file" << std::endl;
}

// Read params from file.
rapidpixel::ConfigReader configReader2;
if (!configReader2.readFromFile("VFilterParams.json"))
{
    std::cout << "Can't read from file" << std::endl;
}
rapidpixel::VFilterParams params2;
if (!configReader2.get(params2, "Params"))
{
    std::cout << "Can't get params" << std::endl;
}

VFilterParams.json will look like:

{
    "Params": {
        "mode": 1,
        "level": 10.100000381469727,
        "type": 2,
        "custom1": 22.299999237060547,
        "custom2": 23.399999618530273,
        "custom3": 24.5
    }
}

Build and connect to your project

Build stand alone

The repository does not contain sub-repositories. The libraries MotionMagnificator depends on (Frame, ConfigReader and VFilter) must be cloned into the same folder as MotionMagnificator. The demo application and the example need more libraries (SimpleFileDialog, SimpleUI, VCodec, FormatConverter, VSource, VDecoderH264, VDecoderH265, VDecoderJPEG, VSourceFile, VEncoderJPEG and VideoRecorderMp4), which must be next to the library as well. The folder structure must be:

<work folder>
    MotionMagnificator
    Frame
    ConfigReader
    VFilter
    SimpleFileDialog
    SimpleUI
    VCodec
    FormatConverter
    VSource
    VDecoderH264
    VDecoderH265
    VDecoderJPEG
    VSourceFile
    VEncoderJPEG
    VideoRecorderMp4

Typical commands to build the library, the benchmark, the demo application and the example (CMake >= 3.13, a C++17 compiler; use the URLs of the repositories provided to you):

git clone <MotionMagnificator repository URL>
git clone <Frame repository URL>
git clone <ConfigReader repository URL>
git clone <VFilter repository URL>
git clone <SimpleFileDialog repository URL>
git clone <SimpleUI repository URL>
git clone <VCodec repository URL>
git clone <FormatConverter repository URL>
git clone <VSource repository URL>
git clone <VDecoderH264 repository URL>
git clone <VDecoderH265 repository URL>
git clone <VDecoderJPEG repository URL>
git clone <VSourceFile repository URL>
git clone <VEncoderJPEG repository URL>
git clone <VideoRecorderMp4 repository URL>
cd MotionMagnificator
mkdir build
cd build
cmake ..
make

On Windows use cmake --build . --config Release instead of make. If MotionMagnificator is the top-level CMake project, the main CMakeLists.txt file adds the folders ../Frame, ../ConfigReader and ../VFilter (in this order) before the library itself. For the demo application it adds the folders of its libraries before the application, for the example the folders of the libraries of the example (a library that both need is added once). If a folder does not exist CMake stops with an error.

The library and the benchmark need only Frame, ConfigReader and VFilter. A stand alone build switches the demo application and the example off if the options are given on the command line together with the option that stops the forced rewriting of the other options (see CMake options):

cmake .. -D_SUBMODULE_CACHE_OVERWRITE=OFF -D_MOTION_MAGNIFICATOR_DEMO=OFF -D_MOTION_MAGNIFICATOR_EXAMPLE=OFF

CMake options

Table 7 - CMake options and macros of the repository. In the options below ${PARENT} is the prefix defined by the top-level project (empty if MotionMagnificator is the top-level project).

Option / macro	Default	Description
${PARENT}_MOTION_MAGNIFICATOR	ON	Build the library.
${PARENT}_MOTION_MAGNIFICATOR_BENCHMARK	ON (stand alone), OFF (included in other project)	Build the benchmark application.
${PARENT}_MOTION_MAGNIFICATOR_DEMO	ON (stand alone), OFF (included in other project)	Build the demo application.
${PARENT}_MOTION_MAGNIFICATOR_EXAMPLE	ON (stand alone), OFF (included in other project)	Build the example.
${PARENT}_SUBMODULE_CACHE_OVERWRITE	ON	Forced rewriting of the options above in the CMake cache. The top-level project sets it to OFF when it includes MotionMagnificator as a sub-repository.
CMAKE_BUILD_TYPE	Release (if not specified)	Build type.
MOTION_MAGNIFICATOR_ENABLE_SIMD	ON	Build the SIMD kernels. OFF builds only the portable kernels (defines MOTION_MAGNIFICATOR_NO_SIMD).
MOTION_MAGNIFICATOR_NO_SIMD	not defined	Macro: portable kernels only.
MOTION_MAGNIFICATOR_NO_SSSE3	not defined	Macro: leave out the SSSE3 kernels (x86). Defined automatically if the compiler cannot build them.
MOTION_MAGNIFICATOR_NO_AVX2	not defined	Macro: leave out the AVX2 kernels (x86). Defined automatically if the compiler cannot build them.
OPENMP_FOUND_FLAG	1 if OpenMP was found, 0 otherwise	Defined in the generated MotionMagnificatorConfig.h (in the build folder).

The CMake files of the library set the flags the processing needs: -ffp-contract=off (GCC, Clang) or /fp:precise (MSVC) for the whole library, -mssse3 for KernelsSsse3.cpp and -mavx2 -mfma (/arch:AVX2 for MSVC) for KernelsAvx2.cpp only. If you compile the source files with another build system, use the same flags and never compile the library with fast-math options or a global -march / /arch option: the results would then depend on the compiler, and a global AVX2 option makes the program fail on processors without AVX2.

OpenMP is optional (find_package(OpenMP)): if CMake finds it the library links OpenMP::OpenMP_CXX, so a program that links the library also needs the OpenMP runtime of the compiler (for example libgomp and pthread with GCC); CMake adds it to targets that link MotionMagnificator.

Connect to your project

If you want to connect MotionMagnificator to your CMake project as source code you can make follow. For example, if your repository has structure:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp

Clone (or copy) the MotionMagnificator, Frame, ConfigReader and VFilter repositories to 3rdparty folder of your repository. Structure of your repository:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    MotionMagnificator
    Frame
    ConfigReader
    VFilter

MotionMagnificator does not add its dependencies when it is included in another project: the project must add Frame, ConfigReader and VFilter by itself (before or after MotionMagnificator). If they are not added, the build stops with a compiler error like Frame.h: No such file or directory.

Create CMakeLists.txt file in 3rdparty folder. CMakeLists.txt should contain:

cmake_minimum_required(VERSION 3.13)

################################################################################
## 3RD-PARTY
## dependencies for the project
################################################################################
project(3rdparty LANGUAGES CXX)

################################################################################
## SETTINGS
## basic 3rd-party settings before use
################################################################################
# To inherit the top-level architecture when the project is used as a submodule.
SET(PARENT ${PARENT}_YOUR_PROJECT_3RDPARTY)
# Disable self-overwriting of parameters inside included subdirectories.
SET(${PARENT}_SUBMODULE_CACHE_OVERWRITE OFF CACHE BOOL "" FORCE)

################################################################################
## CONFIGURATION
## 3rd-party submodules configuration
################################################################################
SET(${PARENT}_SUBMODULE_FRAME                           ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_FRAME)
    SET(${PARENT}_FRAME                                 ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_FRAME_TEST                            OFF CACHE BOOL "" FORCE)
endif()

SET(${PARENT}_SUBMODULE_CONFIG_READER                   ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_CONFIG_READER)
    SET(${PARENT}_CONFIG_READER                         ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_CONFIG_READER_EXAMPLES                OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_CONFIG_READER_TESTS                   OFF CACHE BOOL "" FORCE)
endif()

SET(${PARENT}_SUBMODULE_VFILTER                         ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_VFILTER)
    SET(${PARENT}_VFILTER                               ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_VFILTER_TEST                          OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_VFILTER_EXAMPLE                       OFF CACHE BOOL "" FORCE)
endif()

SET(${PARENT}_SUBMODULE_MOTION_MAGNIFICATOR             ON  CACHE BOOL "" FORCE)
if (${PARENT}_SUBMODULE_MOTION_MAGNIFICATOR)
    SET(${PARENT}_MOTION_MAGNIFICATOR                   ON  CACHE BOOL "" FORCE)
    SET(${PARENT}_MOTION_MAGNIFICATOR_BENCHMARK         OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_MOTION_MAGNIFICATOR_DEMO              OFF CACHE BOOL "" FORCE)
    SET(${PARENT}_MOTION_MAGNIFICATOR_EXAMPLE           OFF CACHE BOOL "" FORCE)
endif()

################################################################################
## INCLUDING SUBDIRECTORIES
## Adding subdirectories according to the 3rd-party configuration
################################################################################
if (${PARENT}_SUBMODULE_FRAME)
    add_subdirectory(Frame)
endif()

if (${PARENT}_SUBMODULE_CONFIG_READER)
    add_subdirectory(ConfigReader)
endif()

if (${PARENT}_SUBMODULE_VFILTER)
    add_subdirectory(VFilter)
endif()

if (${PARENT}_SUBMODULE_MOTION_MAGNIFICATOR)
    add_subdirectory(MotionMagnificator)
endif()

File 3rdparty/CMakeLists.txt adds folders Frame, ConfigReader, VFilter and MotionMagnificator to your project and excludes the benchmark, the demo application and the example from compiling (by default they are excluded from compiling if MotionMagnificator is included as sub-repository). The structure of your repository will be:

CMakeLists.txt
src
    CMakeList.txt
    yourLib.h
    yourLib.cpp
3rdparty
    CMakeLists.txt
    MotionMagnificator
    Frame
    ConfigReader
    VFilter

Next, you need to include the 3rdparty folder in the main CMakeLists.txt file of your repository. Add the following string at the end of your main CMakeLists.txt:

add_subdirectory(3rdparty)

Next, you have to include MotionMagnificator library in your src/CMakeLists.txt file:

target_link_libraries(${PROJECT_NAME} MotionMagnificator)

Done!

Portability

The image processing kernels exist for several instruction set levels. Each level is compiled in its own source file with its own compiler flags, and the library chooses the best level for the processor once per process. The CMake options and the macros that switch the levels are listed in Table 7.

Table 8 - SIMD levels.

Level	Processors	Selection
AVX2 + FMA	x86 processors with AVX2 and FMA	At run time, if the processor and the operating system support AVX2 and FMA.
SSSE3	x86 processors with SSSE3	At run time, if AVX2 + FMA is not available.
NEON	AArch64, 32-bit ARM compiled with NEON	At compile time.
Portable	Every processor	If no other level is available or SIMD is switched off.

One binary runs on every x86-64 processor: no global AVX2 option is used. Within one level the results do not depend on the build type or on the number of threads. The AVX2 + FMA level and the NEON level on AArch64 use fused multiply-add, so the last bit of some intermediate values (and rarely an output value, by one) differs from the SSSE3, portable and 32-bit ARM NEON levels. Within each of the two groups the results are identical, also on big-endian processors.

Example

A simple application shows how to use the MotionMagnificator library.

#include <iostream>
#include <string>
#include "MotionMagnificator.h"
#include "SimpleUi.h"
#include "VSourceFile.h"

// Link namespaces.
using namespace std;
using namespace rapidpixel;


// Entry point: the example plays test.MOV from the working directory.
int main()
{
    // Open the video file. The file replays itself at its end.
    VSourceFile videoSource;
    string fileName{"test.mp4"};
    if (!videoSource.openVSource(fileName))
        return 1;

    // Source frame. Request for YUV24.
    Frame frame;
    frame.fourcc = Fourcc::YUV24;

    // Init MotionMagnificator and set params.
    MotionMagnificator filter;
    filter.setParam(VFilterParam::MODE, 1); // Enable filtering.
    filter.setParam(VFilterParam::CUSTOM_1, 2); // Number of threads.
    filter.setParam(VFilterParam::LEVEL, 100); // Max level.
    filter.setParam(VFilterParam::TYPE, 1); // Enable effect in video.

    // Create windows to display frame.
    rapidpixel::SimpleUiWindow window{"VIDEO"};
    window.show();

    // Main loop until the window is closed or ESC is pressed.
    while (window.isOpen())
    {
        // Get next frame.
        if (!videoSource.getFrame(frame, 1000))
            continue;

        // Process the frame: the library changes only the frames given to
        // processFrame(). With TYPE 1 the intensity channel shows the motion.
        if (!filter.processFrame(frame))
            cout << "Frame not processed" << endl;

        // Display the frame in the window.
        window.render(frame);

        // Process keyboard events.
        if (window.getKey(1) == 27)
            break;
    }

    return 0;
}

Benchmark

Benchmark allows you to evaluate the processing speed on different platforms. This is a console application that asks for the frame width and height (32 - 8192 pixels) and, if OpenMP was detected on library compilation step, for the number of threads (1 - 128). Then it processes 1000 synthetic YUV24 frames with a small moving object with TYPE 1 and no palette (amplitudes and intensity samples are computed; the first frame, which starts the processing, is not measured) and prints the minimum, average and maximum processing time per frame for every 100 frames, and ends. An invalid input ends the benchmark with an error message. Console output for one thread (the times depend on the processor and its load):

=================================================
MotionMagnificator v5.0.0 benchmark
Average processing time per frame
=================================================

Set frame width: 1920
Set frame height: 1080
Set number of threads: 1
Time 2.632 -> 3.47482 -> 5.55 msec
Time 2.631 -> 3.61158 -> 11.312 msec
Time 2.651 -> 3.7512 -> 6.172 msec
...

Console output for 4 threads:

Set number of threads: 4
Time 1.336 -> 1.80191 -> 2.72 msec
Time 1.315 -> 2.1853 -> 5.091 msec
...

Demo application

The demo application is intended to evaluate the performance of the MotionMagnificator C++ library. The application allows you to evaluate the library with a chosen video file. It is a console application and can be used as an example of MotionMagnificator library usage. The application uses the SimpleUI library for displaying video and forming a simple user interface, the VSourceFile library for reading video files and the VideoRecorderMp4 library for recording video. The demo application does not require installation. The demo application is compiled for Windows OS x64 (Windows 10 and newer) as well as for Linux OS (few distros; for a specific Linux distro send a request). Table 9 shows the list of files of the demo application.

Table 9 - List of files of demo application (example for Windows OS).

File	Description
MotionMagnificatorDemo.exe	Demo application executable file for windows OS.
VC_redist.x64	Installer of necessary system libraries for Windows x64.
src	Folder with application source code.
test.mp4	Test video file.

To launch demo application run MotionMagnificatorDemo.exe executable file on Windows x64 OS or run commands on Linux:

sudo chmod +x MotionMagnificatorDemo
./MotionMagnificatorDemo

If a message about missing system libraries appears (on Windows OS) when launching the application, you must install the VC_redist.x64.exe program, which will install the system libraries required for operation. After start user must set number of threads (only if OpenMP was detected on library compilation step; an invalid number keeps one thread):

=================================================
MotionMagnificator v5.0.0 demo application
=================================================

Set number of threads: 4
Open file dialog? (y/n)

After that the user chooses between the file dialog and entering the video file name. If the user does not choose the file dialog the application asks to enter the video file name. Video files are read by the VSourceFile library (supported files: .h264, .264, .h265, .hevc, .jpg, .jpeg, .mjpg, .mjpeg, .mp4, .mov, .m4v, .avi), played with the FPS of the file and replayed from the beginning at the end of the file. If the file gives no frame for 10 seconds the application ends. After that the user will see the user interface as shown below.

motionmagnificator_demo_interface

To control the application, the main video display window must be active (in focus). Letter keys don’t depend on keyboard layout and CapsLock mode. The program is controlled by the keyboard, by the slider (level) and by the mouse: a rectangle drawn with the left mouse button becomes the mask (the frame is processed only inside the rectangle). The demo always processes the frames with TYPE 1 (the type cannot be changed): the intensity channel of the frame shows the motion, the colour channels show the colours of the video or, with a palette (P key), the original intensity. The processing (MODE) is on at the start and can be switched off and on with the E key; when it is off the video is shown as it is. The window shows the frame or, after the A key, the amplitudes of the frame as a grey image.

Table 10 - Control buttons.

Button	Description
ESC	Exit the application. If video recording is active, it will be stopped.
SPACE	Restart the processing (RESET command).
R	Start/stop video recording. When video recording is enabled, a file dst_[date and time].mp4 (result video, MJPEG codec) is created in the working directory of the application. The shown image (frame or amplitudes) is recorded before the information text is drawn. To stop the recording, press the R key again. During video recording, the application shows a warning message.
E	Switch the processing on / off (MODE).
P	Switch the palette (CUSTOM_2) in a circle: off (0, the colours of the video are kept), Arctic (1), Ironbow (2), Rainbow (3), off and so on.
A	Show the amplitudes or the frame.
N / M	Supported only if OpenMP was detected on library compilation step - ‘N’ key decreases and ‘M’ key increases number of threads used to compute (by 1 thread, in range from 1 to 128).

Motion Magnificator C++ library

Table of contents

Overview

Versions

Library files

Key features and capabilities

Supported pixel formats

Library principles

MotionMagnificator class description

MotionMagnificator class declaration

getVersion method

initVFilter method

setParam method

getParam method

getParams method

executeCommand method

processFrame method

getAmplitudes method

setMask method

encodeSetParamCommand method of VFilter class

encodeCommand method of VFilter class

decodeCommand method of VFilter class

decodeAndExecuteCommand method

Data structures

VFilterCommand enum

VFilterParam enum

VFilterParams class description

VFilterParams class declaration

Serialize VFilter params

Deserialize VFilter params

Read params from JSON file and write to JSON file

Build and connect to your project

Build stand alone

CMake options

Connect to your project

Portability

Example

Benchmark

Demo application

Table of contents