yolopreparation_web_logo

YoloDnnPreparationExample

v1.0.0

Table of contents

Overview

YoloDnnPreparationExample is a comprehensive set of software tools designed for training, evaluating, and exporting neural network models, with a focus on object detection tasks. These tools provide a workflow for setting up a model, preparing datasets, optimizing performance, and exporting the trained model. The training set used in this process comes from the FLIR thermal dataset, enabling the model to specialize in detecting objects in thermal imagery. Additionally, the user can test trained model using demo application or improve model by annotating new data.

Versions

Table 1 - Library versions.

Version Release date What’s new
1.0.0 20.12.2024 First version.

Software tools files

The software tools are supplied as source code only. The user is provided with a set of files (repository). The repository structure is shown below:

helpers
  merge_data.py ---------- Script to import new data to main dataset.
  nms.py ----------------- Non Maximum Suppression algorithm implementation.
  onnx_yolov8.py --------- Onnx based object detector implementation.
  prepare_dataset.py ----- Tool for training dataset building.
  split_dataset.py ------- Script to create train / validation / test subsets of data
  verify.py -------------- Script check and modify main dataset.
dataset_config.yaml -------- Configuration file for setting up a dataset.
demo.py -------------------- Script for detector model demo run
eval.py -------------------- Model evaluation script
export_model.py ------------ Script for pytorch model export to onnx | openvino format
finetune.py ---------------- Model training script.
organize_data.py ----------- Script for YOLO data preparation.
requirements.txt ----------- List of 3rd party python packages, required for the tool
weights -------------------- Folder containing pre-trained yolov8 models
  yolov8l.pt -------------- Weights file of Yolov8 L model
  yolov8m.pt -------------- Weights file of Yolov8 M model
  yolov8s.pt -------------- Weights file of Yolov8 S model
  yolov8n.pt -------------- Weights file of Yolov8 N model
yolo8.yaml------------------ YOLO file example.

How to prepare YOLO model based on ready dataset

This section explains the process of preparing a YOLO model using a pre-labeled dataset. It covers the organization of data, configuration adjustments, and the setup required to begin training. This ensures the model is ready to learn effectively from the provided dataset. The whole process results in network.onnx file, which can be successfully used in DnnOpenVinoDetector library for object detection purposes.

Install environment

The tools are created to work under Ubuntu OS (implemented and tested with Ubuntu 22.04). To start user has to copy this repository to local YoloDnnPreparationExample folder.

The whole process is performed with Python. Python can be installed on Linux with the command line:

sudo apt update
sudo apt install python3

To install all the dependencies run the command bellow:

cd YoloDnnPreparationExample
pip3 install -r requirements.txt

Configure dataset

The first step to prepaThis tutorial operates on FLIR Thermal Dataset for Algorithm Training, which can be downloaded from the website. To correctly configure the dataset, editdataset_config.yaml. This file should be updated with all the necessary paths including where dataset was downloaded. Follow the given template:

  • dataset_root - root folder of the Dataset (where to keep it) ; for example - /data/test/datasets/annotated_dataset/thermal/ground`);
  • split_dataset_root - root folder where to keep a copy of the Dataset, that will be split to train / val / test parts (for example - /data/test/datasets/annotated_dataset/thermal/ground/split);
  • flir_train_root - location of the train part of the FLIR dataset (for example - /data/test/datasets/thermal_vision/external_datasets/FLIR_ADAS_v2/images_thermal_train);
  • flir_val_root - location of the validation part of the FLIR dataset (for example - /data/test/datasets/thermal_vision/external_datasets/FLIR_ADAS_v2/images_thermal_val);
  • flir_test_root - location of the test part of the FLIR dataset (for example - /data/test/datasets/thermal_vision/external_datasets/FLIR_ADAS_v2/video_thermal_test);
  • classes - list class names to train (for example classes: - bus - car - person);
  • yolo_dataset_root - path where to keep dataset built for YOLO training (specify root destination folder; for example /data/datasets/thermal_vision/transformed/yolo_training);
  • temp_folder - folder to use for temporal extraction of data (will be removed after end of usage; for example - /tmp/test/cvat_export).

Note: Remember that yolo_dataset_root path has to start with /data/datasets/... because of the Ultralytics settings.

To configure and verify the dataset for YOLO8 structure use configure_data.py script. With this command the dataset will be configured correctly for YOLO8 structure:

python3 organize_data.py -op make-yolo

If all the paths are specified correctly, the script will display information about configured data, for instance:

DATA STATS:
Within annotation file 7 classnames are defined: {1: 'person', 2: 'car', 3: 'truck', 4: 'animal', 5: 'ship', 6: 'uav', 7: 'bird'}
Within annotation file found 6028 images (6028 unique filepaths)
Within annotation file found 81464 objects  which belong to 6028 images
ANNOTATED OBJECTS AMOUNTS:
person:  81464 samples presented onto 6028 images
VERIFICATION
---   1. Found 2 subfolders and 2 files in data folder:   ---
bvk_clip_10	is folder, contains 3014 files
bvk_clip_10.json
export_test	is folder, contains 3014 files
export_test.json
---   2. Annotation file defines next data subfolders:   ---
Annotated subfolder bvk_clip_10  --  CORRECT: defines 3014 images presented in data folder
Annotated subfolder export_test  --  CORRECT: defines 3014 images presented in data folder
Job completed successfully

Now all the data is configured correctly for further processing.

Train the model

To run transfer learning (model training on new data with new object classes) run following command:

python3 finetune.py \
  -d=<path to data config .yaml file built while dataset configuration> \
  -m=<pretrained yolo weights>

Example of running the command:

python3 finetune.py \
  -d=/data/test/datasets/thermal_vision/transformed/yolo_training/dataset_3-classes.yaml \
  -m=weights/yolov8n.pt

The script will save resulting model to the training_results/train/weights/best.pt. Use that during following evaluation and export steps.

Evaluate the model

To run model evaluation (optional step for model accuracy estimation) run following command:

python3 eval.py \
  -d=<path to data config .yaml file built while dataset configuration> \
  -m=<path to trained model>

Example of running the command:

python3 eval.py \
  -d=/data/test/datasets/thermal_vision/transformed/yolo_training/dataset_3-classes.yaml \
  -m=training_results/train/weights/best.pt

This command results in useful information like:

  • precision - correctly predicted positive detections out of all detections
  • recall - proportion of correctly predicted positive detections,
  • mAP50 - mean average precision at 50% Intersection over Union (IoU) threshold,
  • mAP50-95 - average of the mean average precision calculated at varying IoU thresholds, ranging from 0.50 to 0.95
  • fitness - combination of metrics used to evaluate the overall quality of the model
  • F1 score - the harmonic mean of precision and recall.

Export the model

To export trained model to .onnx or openvino format run the command below:

python3 export_model.py \
  -m=<path to trained model> \
  -o <path to resulting .onnx file (extention must be '.onnx')

Example of running the command:

python3 export_model.py \
  -m=training_results/train/weights/best.pt \
  -o=training_results/train/weights/best.onnx

Now the model is exported with name best.onnx and can be used in DnnOpenVinoDetector library.

Try the model with demo application

There is a demo.py tool which runs detector model against image(-s) or video and displays detections frame by frame or creates a video clip with drawn detections. To run it, use the following command:

python3 demo.py \
  -m=<path to the model weights file ([.pt | .onnx]) \
  -i <path(-s) to image(-s) or path to video file> \
  -o <path to resulting video clip (optional)>

To display detections real time frame by frame do not provide -o parameter. To create video clip with detections drawn do provide -o parameter (path to resulting .avi file)

python3 demo.py \
  -m=training_results/train/weights/best.pt \
  -i=/data/test/demo/clip_demo.mp4

or

python3 demo.py \
  -m=training_results/train/weights/best.pt \
  -i=/data/test/demo/clip_demo.mp4 \
  -o=/data/test/demo/clip_demo_detections.avi

How to improve model using new data

The accuracy of trained model on the dataset above can be improved by new data. The process starts with manual annotation using the tool called CVAT and exporting the data in COCO v1.0 format. The data should be packed in .zip archive and include two folders, images and annotations with correct data inside.

The configure_dataset.py script includes option to include additional data and merge it into base dataset. To execute new data import process run the command below providing path to COCO export .zip archives:

python3 organize_data.py -op merge -i <path to exported COCO .zip file>

For example, following command will merge export_test.zip to the main dataset:

python3 organize_data.py -op merge -i /data/test/export_test.zip

It is also possible to run merging multiple datasets in one command, specifying paths one after another.

python3 organize_data.py -op merge -i <path(-s) to exported COCO .zip file(-s)>

For example, following command will merge export_test.zip and export_test_two.zip to the main dataset.

python3 organize_data.py -op merge -i /data/test/export_test.zip /data/test/export_test_two.zip

As an execution result updated dataset statistics will be printed.

Now the dataset is updated with the new data and all the steps for training can be repeated for training new model, starting from Train the model.