RpiOnboardControl application

v1.0.3

Table of contents

• Overview
• Versions
• Source code files
• Config file
• Build application
• Run application
• How to send tracker commands
• Source code

Overview

The RpiOnboardControl is template control application that works with RpiTracker. It provides remote control for RpiTracker as well as showing real time stream that comes from RpiTracker. The application combines libraries: VTracker (for preparing tracker commands and decoding tracker results) and UdpSocket (for communication with RpiTracker). The application shows how to control tracker and receive real time stream from remote RpiTracker application. It is built for both Linux and WINDOWS.

After start, application reads JSON config file which includes communication port, ip and rtsp input (such as rtsp://192.168.0.2:8554/live). If there is no config file the application will create new one with default parameters.

The application uses OpenCV for receiving and displaying the real-time stream and UdpSocket for sending tracker commands to RpiTracker.

Versions

Table 1 - Application versions.

Version	Release date	What’s new
1.0.0	01.01.2024	- First version.
1.0.1	13.03.2024	- Code reviewed. - Documentation added.
1.0.2	24.05.2024	- Documentation added.
1.0.3	06.08.2024	- Submodules updated.

Source code files

The application is supplied by source code only. The user is given a set of files in the form of a CMake project (repository). The repository structure is shown below:

CMakeLists.txt --------------------- Main CMake file.
3rdparty --------------------------- Folder with third-party libraries.
    CMakeLists.txt ----------------- CMake file to include third-party libraries.
    UdpSocket ---------------------- Source code of UdpSocket library.
    VTracker ----------------------- Source code of VTracker library.
src -------------------------------- Folder with application source code.
    CMakeLists.txt ----------------- CMake file.
    RpiOnboardControlVersion.h ----- Header file with application version.
    RpiOnboardControlVersion.h.in -- File for CMake to generate version header.
    main.cpp ----------------------- Application source code file.

Config file

RpiOnboardControl application reads config file RpiOnboardControl.json in the same folder with application executable file. Config file content:

{
    "Params": {
        "controlChannel": {
            "controlUdpPort": 50020,
            "dstIp": "192.168.0.2"
        },
        "videoSource": {
            "initString": "rtsp://192.168.0.2:8554/live"
        }
    }
}

Table 2 - Config file parameters description.

Parameter	type	Description
controlChannel:
dataUdpPort	int	UDP port number to send commands for tracker and get video stream.
dstIp	string	Ip of server (system that RpiTracker runs)
videoSource:
initString	string	Rtsp stream link that runs on RpiTracker.

Build application

Application is built using CMake. Below typical build commands:

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make

Run application

• Copy application (RpiOnboardControl executable and RpiOnboardControl.json) to any folder.

• Run :

./RpiOnboardControl

When the application is started for the first time, it creates a configuration file named RpiOnboardControl.json if file does not exist (refer to the Config file section). If the application is run as a superuser using sudo, the file will be owned by the root user. Therefore, to modify the configuration file, superuser privileges will be necessary.

Please note that the application will link to the OpenCV library. Either the library is not installed on the system or the application cannot find it, the application will not start. To fix this issue, install the OpenCV library on the system and/or set the path to the library in the system environment variables.

How to send tracker commands

The RpiOnboardControl application transmits various tracker commands to the RpiTracker application, some of which are specifically related to keyboard and mouse functions. Below is a list of these tracker commands that pertain to keyboard and mouse operations.

• Main tracker commands.

  • Start tracking : Click mouse left button.

  • Reset tracker : Click mouse right button.

• Tracking rectangle dimension commands.

  • Increase tracking rectangle width : Press D on keyboard.

  • Decrease tracking rectangle width : Press A on keyboard.

  • Increase tracking rectangle height : Press W on keyboard.

  • Decrease tracking rectangle height : Press S on keyboard.

• Move tracking rectangle by keyboard.

  • Move right : Press H on keyboard.

  • Mode left : Press F on keyboard.

  • Move up : Press T on keyboard.

  • Move down : Press G on keyboard.

Source code

Bellow source code of main.cpp file:

#include <iostream>
#include <string>
#include <opencv2/opencv.hpp>
#include "VTracker.h"
#include "UdpSocket.h"
#include "RpiOnboardControlVersion.h"



/// Max command size, bytes.
constexpr int MAX_COMMAND_SIZE = 32;
/// Max telemetry size, bytes.
constexpr int MAX_TELEMETRY_SIZE = 512;
/// Config file name.
#define CONFIG_FILE_NAME "RpiOnboardControl.json"



/// Application parameters.
struct Params
{
public:
    /// Video source params.
    struct VideoSource
    {
    public:
        /// Init string.
        std::string initString{"rtsp://192.168.0.2:8554/live"};

        JSON_READABLE(VideoSource, initString)
    };

    /// Control channel.
    struct ControlChannel
    {
    public:
        /// Input UDP port.
        int controlUdpPort{50020};
        /// Destination IP.
        std::string dstIp{"192.168.0.2"};

        JSON_READABLE(ControlChannel, controlUdpPort, dstIp)
    };

    /// Video source.
    VideoSource videoSource;
    /// Control channel.
    ControlChannel controlChannel;

    JSON_READABLE(Params, videoSource, controlChannel)
};



/// Application params.
Params g_params;
/// UDP data client.
cr::clib::UdpSocket g_client;



/// Mouse callback function.
void mouseCallBackFunc(int event, int x, int y, int flags, void *userdata);



int main(int argc, char **argv)
{
    std::cout << "RpiOnboardControl application v" << RPI_ONBOARD_CONTROL_VERSION << std::endl;
    
    // Open config json file (if does not exist - create new and exit)
    cr::utils::ConfigReader config;
    if (config.readFromFile(CONFIG_FILE_NAME))
    {
        // Read values and set to params.
        if (!config.get(g_params, "Params"))
        {
            std::cerr << "ERROR: Params were not read." << std::endl;
            return -1;
        }
    }
    else
    {
        // Set default params.
        config.set(g_params, "Params");

        // Save config file.
        if (!config.writeToFile(CONFIG_FILE_NAME))
        {
            std::cerr << "Config file created." << std::endl;
            return -1;
        }
    }

    // Init UDP data client.
    if (!g_client.open(g_params.controlChannel.controlUdpPort, false, g_params.controlChannel.dstIp))
    {
        std::cerr << "ERROR: Can't initialize client." << std::endl;
        return -1;
    }

    // Open video source. If not open will try again.
    cv::VideoCapture videoSource;
    while (!videoSource.open(g_params.videoSource.initString))
    {
        std::cout << "ERROR: Can't open video source" << std::endl;
        std::this_thread::sleep_for(std::chrono::seconds(1));
    }

    // Create window.
    cv::namedWindow("Control application", cv::WINDOW_AUTOSIZE);
    // Set mouse callback.
    cv::setMouseCallback("Control application", mouseCallBackFunc, nullptr);

    // Init input image.
    cv::Mat frame;

    // Main loop.
    while (true)
    {
        // Wait new frame.
        videoSource >> frame;
        if (frame.empty())
        {
            // Reconnect.
            videoSource.release();
            while (!videoSource.open(g_params.videoSource.initString))
            {
                std::cerr << "ERROR: Can't open video source." << std::endl;
                std::this_thread::sleep_for(std::chrono::seconds(1));
            }
            continue;
        }

        // Show video.
        cv::imshow("Control application", frame);

        // Prepare command.
        uint8_t command[32]{};
        int size = 0;

        // Process keyboard events.
        switch (cv::waitKey(1))
        {
        // ESC - Exit.
        case 27:
        {
            cv::destroyAllWindows();
            return 1;
        }
        // W - Increase tracking rectangle height.
        case 119:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::CHANGE_RECT_SIZE, 0.0f, 4.0f);
            break;
        }
        // S - Decrease tracking rectangle height.
        case 115:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::CHANGE_RECT_SIZE, 0.0f, -4.0f);
            break;
        }
        // D - Increase tracking rectangle width.
        case 100:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::CHANGE_RECT_SIZE, 4.0f, 0.0f);
            break;
        }
        // A - Decrease tracking rectangle width.
        case 97:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::CHANGE_RECT_SIZE, -4.0f, 0.0f);
            break;
        }
        // T - Move strobe UP (change position in TRACKING mode).
        case 116:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::MOVE_RECT, 0.0f, 4.0f);
            break;
        }
        // G - Move strobe DOWN (change position in TRACKING mode).
        case 103:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::MOVE_RECT, 0.0f, -4.0f);
            break;
        }
        // H - Move strobe RIGHT (change position in TRACKING mode).
        case 104:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::MOVE_RECT, -4.0f, 0.0f);
            break;
        }
        // F - Move strobe LEFT (change position in TRACKING mode).
        case 102:
        {
            cr::vtracker::VTracker::encodeCommand(
                command, size, cr::vtracker::VTrackerCommand::MOVE_RECT, 4.0f, 0.0f);
            break;
        }
        default:
            continue; // There is no command to send. So continue.
        }

        // Send command.
        g_client.send(command, size);
    }

    return 1;
}



void mouseCallBackFunc(int event, int x, int y, int flags, void *userdata)
{
    // Prepare command.
    uint8_t command[32]{};
    int size = 0;

    switch (event)
    {
    /// Capture object on current coordinates.
    case cv::EVENT_LBUTTONDOWN:
    {
        cr::vtracker::VTracker::encodeCommand(
            command, size, cr::vtracker::VTrackerCommand::CAPTURE, -1.0f, -1.0f, -1.0f);
        break;
    }
    /// Reset tracker.
    case cv::EVENT_RBUTTONDOWN:
    {
        cr::vtracker::VTracker::encodeCommand(
            command, size, cr::vtracker::VTrackerCommand::RESET);
        break;
    }
    case cv::EVENT_MOUSEMOVE:
    {
        cr::vtracker::VTracker::encodeCommand(
            command, size, cr::vtracker::VTrackerCommand::SET_RECT_POSITION, static_cast<float>(x), static_cast<float>(y));
        break;
    }
    default: // No command to send. So return.
        return;
    }

    // Send command.
    g_client.send(command, size);
}