README.md

# Video Analyser

[![MPAI CAE-ARP](https://img.shields.io/badge/MPAI%20CAE--ARP-gray?style=for-the-badge&logo=AppleMusic&logoColor=cyan&link=https://mpai.community/standards/mpai-cae/about-mpai-cae/)](https://mpai.community/standards/mpai-cae/about-mpai-cae/)

## Description
Implements the Technical Specification of [MPAI CAE-ARP](https://mpai.community/standards/mpai-cae/about-mpai-cae/#Figure2) *Video Analyser* AIM, providing:
* 2 Irregularity Files;
* Irregularity Images.

## Getting started
The *Video Analyser* is written in C++23. It relies on OpenCV to elaborate Irregularity Images and on Boost C++ Libraries to create the command line interface and generate UUIDs. The configuration file is read with [nlohmann/json](https://github.com/nlohmann/json).

## Installation
[Boost C++ Libraries](https://www.boost.org) are required for creating the command line interface (with [Boost.Program_options](https://www.boost.org/doc/libs/1_81_0/doc/html/program_options.html)) and generating UUIDs (with [Uuid](https://www.boost.org/doc/libs/1_81_0/libs/uuid/doc/uuid.html)).
You can install them following [official instructions](https://www.boost.org/doc/libs/1_81_0/more/getting_started/unix-variants.html) (Boost version 1.81.0).
Boost `program_options` library shall be separately built following [these additional instructions](https://www.boost.org/doc/libs/1_81_0/more/getting_started/unix-variants.html#easy-build-and-install).

[OpenCV](https://docs.opencv.org/4.x/index.html) is required for elaborating Irregularity Images. You can install it following [official instructions](https://docs.opencv.org/3.4/d0/db2/tutorial_macos_install.html).

Finally, [nlohmann/json](https://github.com/nlohmann/json) is required for reading the configuration file.
Installation instructions are under the "Integration" section.

In the root folder there is a CMakeLists.txt file that specifies the configuration for CMake:
```
CMAKE_MINIMUM_REQUIRED(VERSION 3.2)
PROJECT(video_analyser)

SET(EXECUTABLE_OUTPUT_PATH ${PROJECT_SOURCE_DIR}/bin)
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
SET(CMAKE_CXX_STANDARD 23)

LINK_DIRECTORIES(/usr/local/lib)

FIND_PACKAGE(OpenCV REQUIRED)
FIND_PACKAGE(nlohmann_json 3.2.0 REQUIRED)
FIND_PACKAGE(Boost COMPONENTS program_options REQUIRED)

INCLUDE_DIRECTORIES(${OpenCV_INCLUDE_DIRS})
INCLUDE_DIRECTORIES(${Boost_INCLUDE_DIR})

ADD_EXECUTABLE(video_analyser ./src/script.cpp)
TARGET_LINK_LIBRARIES(video_analyser ${OpenCV_LIBRARIES} nlohmann_json::nlohmann_json ${Boost_PROGRAM_OPTIONS_LIBRARY})
```
`LINK_DIRECTORIES` specifies the path to the installed libraries.
Make sure that the flag `${Boost_PROGRAM_OPTIONS_LIBRARY}` is present under `TARGET_LINK_LIBRARIES`.

Once the libraries are installed, you can build the *Video Analyser* moving to `/build` directory and invoking CMake commands:
```
cd /path/to/video/analyser/build
cmake ..
make
```

## Usage
Once the program is built, you should customise the configuration file `config.json`.
There are four required parameters of interest:
1. `WorkingPath` that specifies the working path where all input files are stored and where all output files will be saved;
2. `FilesName` that specifies the name of the preservation files to be considered.
3. `Brands` that specifies if the tape presents brands on its surface;
4. `Speed` that specifies the speed at which the tape was read;

There are also other required parameters which deeply influence the behaviour of the *Video Analyser* and, therefore, ***should not be modified unless with great knowledge of what you are doing***. They are:
1. `TapeThresholdPercentual` that specifies the minimum percentage of different pixels for considering the current frame under the tape ROI as a potential Irregularity;
2. `CapstanThresholdPercentual` that specifies the minimum percentage of different pixels for considering the current frame under the capstan ROI as a potential Irregularity;
3. `MinDist` that specifies the minimum distance between the centers of the detected objects for the detection of the reading head;
4. `AngleThresh` that specifies the angle votes threshold for the detection of the reading head;
5. `ScaleThresh` that specifies the scale votes threshold for the detection of the reading head;
6. `PosThresh` that specifies the position votes threshold for the detection of the reading head;
7. `MinDistCapstan` that specifies the minimum distance between the centers of the detected objects for the detection of the capstan;
8. `AngleThreshCapstan` that specifies the angle votes threshold for the detection of the capstan;
9. `ScaleThreshCapstan` that specifies the scale votes threshold for the detection of the capstan;
10. `PosThreshCapstan` that specifies the position votes threshold for the detection of the capstan.

To execute the script without issues, the inner structure of the `WorkingPath` directory shall be like:
```
.
├── AccessCopyFiles
│   └── ...
├── PreservationAudioFile
│   ├── File1.wav
│   ├── File2.wav
│   └── ...
├── PreservationAudioVisualFile
│   ├── File1.mp4
│   ├── File2.mp4
│   └── ...
├── PreservationMasterFiles
│   └── ...
└── temp
    ├── File1
    │   ├── AudioAnalyser_IrregularityFileOutput1.json
    │   ├── AudioAnalyser_IrregularityFileOutput2.json
    │   ├── AudioBlocks
    │   │   ├── AudioBlock1.wav
    │   │   ├── AudioBlock2.wav
    │   │   └── ...
    │   ├── EditingList.json
    │   ├── IrregularityImages
    │   │   ├── IrregularityImage1.jpg
    │   │   ├── IrregularityImage2.jpg
    │   │   └── ...
    │   ├── RestoredAudioFiles
    │   │   ├── RestoredAudioFile1.wav
    │   │   ├── RestoredAudioFile2.wav
    │   │   └── ...
    │   ├── TapeIrregularityClassifier_IrregularityFileOutput1.json
    │   ├── TapeIrregularityClassifier_IrregularityFileOutput2.json
    │   ├── VideoAnalyser_IrregularityFileOutput1.json
    │   └── VideoAnalyser_IrregularityFileOutput2.json
    ├── File2
    │   ├── AudioAnalyser_IrregularityFileOutput1.json
    │   ├── AudioAnalyser_IrregularityFileOutput2.json
    │   ├── AudioBlocks
    │   │   ├── AudioBlock1.wav
    │   │   ├── AudioBlock2.wav
    │   │   └── ...
    │   ├── EditingList.json
    │   ├── IrregularityImages
    │   │   ├── IrregularityImage1.jpg
    │   │   ├── IrregularityImage2.jpg
    │   │   └── ...
    │   ├── RestoredAudioFiles
    │   │   ├── RestoredAudioFile1.wav
    │   │   ├── RestoredAudioFile2.wav
    │   │   └── ...
    │   ├── TapeIrregularityClassifier_IrregularityFileOutput1.json
    │   ├── TapeIrregularityClassifier_IrregularityFileOutput2.json
    │   ├── VideoAnalyser_IrregularityFileOutput1.json
    │   └── VideoAnalyser_IrregularityFileOutput2.json
    └── ...
```
`PreservationAudioFile` and `PreservationAudioVisualFile` directories contain the input of ARP Workflow, while `AccessCopyFiles` and `PreservationMasterFiles` directories contain its output. `temp` directory is used to store all files exchanged between the AIMs within the Workflow.

Please note that:
* Corresponding input files shall present the same name;
* The name of Irregularity Files given above is ***mandatory***.

With this structure, `FilesName` parameter could be equal to `File1` or `File2`.

You can now launch the *Video Analyser* moving to the `/bin` directory from the command line with:
```
cd /path/to/video/analyser/bin
./video_analyser
```
Useful log information will be displayed during execution.

To enable integration in more complex workflows, it is also possible to launch the *Video Analyser* with command line arguments:
```
./video_analyser [-h] -w WORKING_PATH -f FILES_NAME -b BRANDS -s SPEED
```
If you use the `-h` flag:
```
./video_analyser -h
```
all instructions will be displayed.

## Support
If you require additional information or have any problem, you can contact us at:
* Nadir Dalla Pozza (nadir.dallapozza@unipd.it);
* Niccolò Pretto (niccolo.pretto@unipd.it).

## Authors and acknowledgment
This project was developed by:
* Nadir Dalla Pozza (University of Padova);
* Niccolò Pretto (University of Padova);
* Sergio Canazza (University of Padova).

This project takes advantage of the following libraries:
* [Boost C++ Libraries](https://www.boost.org);
* [OpenCV](https://docs.opencv.org/4.x/index.html);
* [nlohmann/json](https://github.com/nlohmann/json).

Developed with IDE [Visual Studio Code](https://code.visualstudio.com).

## License
This project is licensed with [GNU GPL v3.0](https://www.gnu.org/licenses/gpl-3.0.html).

# TODO

This section refers to the code delivered by February 2023.

- To be able to work with the "old" neural network (by Ilenya), the output images should correspond to the old "whole tape" where, from the frame judged as interesting, an area corresponding to the height of the tape was extracted (so about the height of the current rectangle) and as wide as the original frame (so 720px). This area will then have to be resized to 224x224 as in the past. If instead you decide to use the new neural network, no changes are needed;
- A resize function of the entire video should be implemented if it does not conform to the PAL standard (currently taken for granted);
- Progressive videos, which do not require deinterlacing, should be managed (in the code there are several steps that operate considering this property);