Commit 95a3fbcb authored by Matteo Spanio's avatar Matteo Spanio
Browse files

update

parent 124dd4b2
This diff is collapsed.
Subproject commit df83fbf22cfff76b875c13d324baf584c74e96d0
# Documentation
[TOC]
> There is no worse software than undocumented software.
![Meme](../img/meme-documentation.jpg)
## Generate the documentation
Along with the source code, the documentation of the *Video Analyser* is provided in the `docs` folder. The documentation is generated with [Doxygen](https://www.doxygen.nl/index.html) and can be accessed by opening the `index.html` file in the `docs/html` folder with a browser.
To generate the documentation, run the following command from the root folder:
```
make docs
```
Note that Doxygen must be installed on your machine.
## Write the documentation
Doxygen is a tool for generating documentation from annotated C++ sources, as well as other popular programming languages like C, Objective-C, C#, PHP, Java, Python, IDL (Corba, Microsoft, and UNO/OpenOffice flavors), Fortran, VHDL, Tcl, and to some extent D.
To access Doxygen superpowers you need to add comments to your code. Doxygen supports two types of comments: single-line and multi-line comments. Single-line comments start with `///` or `//!` and multi-line comments start with `/**` and end with `*/`. The following example shows how to use both types of comments:
```
/// This is a single-line comment.
//! This is also a single-line comment.
/**
* This is a multi-line comment.
*/
```
A sample of Doxygen commented class looks like this:
```
/**
* @class SampleClass
* @brief This is a sample class.
*
* This class is used to show how to comment a class with Doxygen.
*/
class SampleClass {
private:
int field1; /**< This is a sample field. */
public:
/**
* @brief This is a sample constructor.
*
* This constructor is used to show how to comment a constructor with Doxygen.
*/
SampleClass();
/**
* @brief This is a sample destructor.
*
* This destructor is used to show how to comment a destructor with Doxygen.
*/
~SampleClass();
/**
* @brief This is a sample method.
*
* This method is used to show how to comment a method with Doxygen.
*
* @param[in] param1 This is a sample parameter.
* @param[in] param2 This is another sample parameter.
* @return This is a sample return value.
*/
int sampleMethod(int param1, int param2);
};
```
For more information about Doxygen, please refer to the [official documentation](https://www.doxygen.nl/manual/index.html).
# Style guide
The code is written following C++20 standard, to ensure the best readability and maintainability. The code is formatted using [clang-format](https://clang.llvm.org/docs/ClangFormat.html) with the [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html) as a reference.
To format the code, run the following command from the root folder:
```
make format
```
Note that clang-format must be installed on your machine.
In addition to the Google C++ Style Guide, the following rules are applied:
- when returning multiple values, use `std::tuple` or `std::pair`, instead of passing by reference;
- when dealing with nullable values, use `std::optional`;
- avoid to manipulate global variables, if you need to share data between different parts of the code, use dependency injection and pass the data as a parameter;
## Naming conventions
In C++, there are several naming conventions that are widely followed to improve code readability and maintainability. Although there is no strict standard enforced by the language, the following conventions are commonly used:
1. Variable Names:
- Use descriptive and meaningful names that reflect the purpose of the variable.
- Prefer lowercase letters for variable names.
- Use underscores (_) to separate words in multi-word variable names.
- Avoid using single uppercase letters as variable names, especially as standalone variables.
Example: `int num_items;`
2. Function Names:
- Use verbs or verb phrases to describe actions or operations performed by the function.
- Prefer lowercase letters for function names.
- Use underscores (_) to separate words in multi-word function names.
- Use parentheses () for function parameters, even if they are empty.
Example: `void calculate_average();`
3. Class/Struct Names:
- Use noun phrases or nouns to describe the purpose or nature of the class/struct.
- Use uppercase letters for each word (known as "PascalCase" or "camel case").
- Avoid abbreviations unless they are widely recognized.
Example: `class CustomerData;`
4. Constant Names:
- Use uppercase letters for constants.
- Use underscores (_) to separate words in multi-word constant names.
- Prefer meaningful and self-explanatory names for constants.
Example: `const int MAX_SIZE = 100;`
5. Global Variable Names:
- Avoid using global variables whenever possible. However, if necessary, prefix them with `g_` or use a namespace to indicate their global nature.
Example: `int g_global_variable;` or `namespace globals { int global_variable; }`
# Getting started
The *Video Analyser* is written in C++20. It relies on OpenCV to elaborate Irregularity Images and on Boost C++ Libraries to create the command line interface and generate UUIDs. The json files are read with [nlohmann/json](https://github.com/nlohmann/json).
To clone the repository, run the following command:
```
git clone https://gitlab.dei.unipd.it/mpai/video-analyzer.git
```
Since the documentation uses a git repo as a submodule, you should clone the repository with the `--recursive` option:
```
git clone --recursive https://gitlab.dei.unipd.it/mpai/video-analyzer.git
```
If you have already cloned the repository without the `--recursive` option, you can run the following command to clone the submodule:
```
git submodule update --init --recursive
```
[TOC]
## Installation
[Boost C++ Libraries](https://www.boost.org) are required for creating the command line interface (with [Boost.Program_options](https://www.boost.org/doc/libs/1_81_0/doc/html/program_options.html)) and generating UUIDs (with [Uuid](https://www.boost.org/doc/libs/1_81_0/libs/uuid/doc/uuid.html)).
You can install them following [official instructions](https://www.boost.org/doc/libs/1_81_0/more/getting_started/unix-variants.html) (Boost version 1.81.0).
Boost `program_options` library shall be separately built following [these additional instructions](https://www.boost.org/doc/libs/1_81_0/more/getting_started/unix-variants.html#easy-build-and-install).
[OpenCV](https://docs.opencv.org/4.x/index.html) is required for elaborating Irregularity Images. You can install it following [official instructions](https://docs.opencv.org/3.4/d0/db2/tutorial_macos_install.html).
To install OpenCV and Boost C++ Libraries on Ubuntu, run the following command:
```
sudo apt update && sudo apt install libboost-program-options-dev git build-essential cmake g++ wget unzip python3 python3-pip libgtk-3-dev libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libjpeg-dev libpng-dev python3-dev libavdevice-dev libdc1394-dev libgstreamer-opencv1.0-0 libavutil-dev ffmpeg
```
To compile OpenCV from source with all the optional libraries, run the following commands:
```
mkdir opencv_source && cd ./opencv_source && wget -O opencv.zip https://github.com/opencv/opencv/archive/4.5.4.zip && wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.5.4.zip && unzip opencv.zip && unzip opencv_contrib.zip && mkdir -p build && cd ./build && cmake -D OPENCV_GENERATE_PKGCONFIG=YES -D WITH_FFMPEG=ON -D WITH_V4L=ON -D WITH_PNG=ON -D WITH_GSTREAMER=ON -D BUILD_opencv_video=ON -D BUILD_opencv_videoio=ON -D OPENCV_ENABLE_NONFREE=ON -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.5.4/modules ../opencv-4.5.4 && make -j4 && make install
```
Finally, [nlohmann/json](https://github.com/nlohmann/json) is required for reading the configuration file.
Installation instructions are under the "Integration" section.
In the root folder there is a CMakeLists.txt file that specifies the configuration for CMake. Here is specified:
- the minimum required version of CMake;
- the project name;
- the C++ standard version;
- the source files;
- the include directories;
- the libraries to link.
Once the libraries are installed, you can build the *Video Analyser* moving to `/build` directory and invoking CMake commands:
```
cd /path/to/video/analyser/build
cmake ..
make
```
or just run `make build` from the root folder.
### Docker
A Dockerfile is provided to build a Docker image with the *Video Analyser*.
To build the image, run the following command from the root folder:
```
docker build -t mpai-video-analyzer .
```
To run the container, run the following command:
```
docker run -it --rm -v /path/to/video/analyser:/app -v /path/to/your/data:/data /bin/bash mpai-video-analyzer
```
where `/path/to/video/analyser` is the path to the *Video Analyser* folder.
This will mount the *Video Analyser* folder in the container and run a bash shell, where you can build the *Video Analyser* as described in the previous section. The advantage of using Docker is that you don't have to install the dependencies on your machine, but you can build the *Video Analyser* in a container.
## Usage
Once the program is built, you should customise the configuration file `config/config.json`.
There are four required parameters of interest:
1. `WorkingPath` that specifies the working path where all input files are stored and where all output files will be saved;
2. `FilesName` that specifies the name of the preservation files to be considered.
3. `Brands` that specifies if the tape presents brands on its surface;
4. `Speed` that specifies the speed at which the tape was read;
There are also other required parameters which deeply influence the behaviour of the *Video Analyser* and, therefore, ***should not be modified unless with great knowledge of what you are doing***. They are:
1. `TapeThresholdPercentual` that specifies the minimum percentage of different pixels for considering the current frame under the tape ROI as a potential Irregularity;
2. `CapstanThresholdPercentual` that specifies the minimum percentage of different pixels for considering the current frame under the capstan ROI as a potential Irregularity;
4. `AngleThresh` that specifies the angle votes threshold for the detection of the reading head;
5. `ScaleThresh` that specifies the scale votes threshold for the detection of the reading head;
6. `PosThresh` that specifies the position votes threshold for the detection of the reading head;
7. `MinDistCapstan` that specifies the minimum distance between the centers of the detected objects for the detection of the capstan;
8. `AngleThreshCapstan` that specifies the angle votes threshold for the detection of the capstan;
9. `ScaleThreshCapstan` that specifies the scale votes threshold for the detection of the capstan;
10. `PosThreshCapstan` that specifies the position votes threshold for the detection of the capstan.
To execute the script without issues, the inner structure of the `WorkingPath` directory shall be like:
```
.
├── PreservationAudioFile
│ ├── File1.wav
│ ├── File2.wav
│ └── ...
├── PreservationAudioVisualFile
│ ├── File1.mp4
│ ├── File2.mp4
│ └── ...
└── temp
├── File1
│ ├── AudioAnalyser_IrregularityFileOutput1.json
│ ├── AudioAnalyser_IrregularityFileOutput2.json
│ ├── AudioBlocks
│ │ ├── AudioBlock1.wav
│ │ ├── AudioBlock2.wav
│ │ └── ...
│ ├── EditingList.json
│ ├── IrregularityImages
│ │ ├── IrregularityImage1.jpg
│ │ ├── IrregularityImage2.jpg
│ │ └── ...
│ ├── RestoredAudioFiles
│ │ ├── RestoredAudioFile1.wav
│ │ ├── RestoredAudioFile2.wav
│ │ └── ...
│ ├── TapeIrregularityClassifier_IrregularityFileOutput1.json
│ ├── TapeIrregularityClassifier_IrregularityFileOutput2.json
│ ├── VideoAnalyser_IrregularityFileOutput1.json
│ └── VideoAnalyser_IrregularityFileOutput2.json
└── File2
├── AudioAnalyser_IrregularityFileOutput1.json
└── ...
```
`PreservationAudioFile` and `PreservationAudioVisualFile` directories contain the input of ARP Workflow. `temp` directory is used to store all files exchanged between the AIMs within the Workflow.
Please note that:
* Corresponding input files shall present the same name;
* The name of Irregularity Files given above is ***mandatory***.
With this structure, `FilesName` parameter could be equal to `File1` or `File2`.
You can now launch the *Video Analyser* moving to the `/bin` directory from the command line with:
```
cd /path/to/video/analyser/bin
./video_analyser
```
Useful log information will be displayed during execution.
To enable integration in more complex workflows, it is also possible to launch the *Video Analyser* with command line arguments:
```
./video_analyser [-h] -w WORKING_PATH -f FILES_NAME -b BRANDS -s SPEED
```
If you use the `-h` flag:
```
./video_analyser -h
```
all instructions will be displayed.
This diff is collapsed.
# General Requirements
The main goal of the MPAI-CAE ARP software is to take audio an video input of a open reel tape, analyse them and produce as output some kind of classifications and restorations (if needed).
# The input
As already said, the input consists of a video and an audio file of an open reel tape. The video file contains the video of the tape reproduced on the recorder, pointing the camera to the capstan, the pinch roller and the reading head. The audio file contains the audio of the tape reproduced on the recorder and captured from the headphone exit.
Focusing on the video analysis, the software should be able to detect tape irregularities, such as:
- Splices
- Brands on tape
- Start of tape
- End of tape
- Damaged tape
- Dirt
- Marks
- Shadows
- Wow and flutter
Most of the brands consist of the full name of the tape manufacturer, logo, or tape model codes. The brand changes in size, shape, and colour, depending on the tape used.
# Software Requirements
The software should be able to, given as input the video of an open reel tape, produce as output two irregularity files where are listed the irregularities found in the video and the irregularities found in the audio.
Irregularity files are JSON files that contain a list of irregularities. Each irregularity is a JSON object that contains structured as follow:
```js
{
"type": "object",
"properties": {
"IrregularityID": {
"type": "string",
"format": "uuid"
},
"Source": {
"enum": ["a", "v", "b"]
},
"TimeLabel": {
"type": "string",
"pattern": "[0-9]{2}:[0-5][0-9]:[0-5][0-9]\\.[0-9]{3}"
},
"IrregularityType": {
"enum": ["sp", "b", "sot", "eot", "da", "di", "m", "s", "wf", "pps", "ssv", "esv", "sb"]
},
"IrregularityProperties": {
"type": "object",
"properties": {
"ReadingSpeedStandard": {
"enum": [0.9375, 1.875, 3.75, 7.5, 15, 30]
},
"ReadingEqualisationStandard": {
"enum": ["IEC", "IEC1", "IEC2"]
},
"WritingSpeedStandard": {
"enum": [0.9375, 1.875, 3.75, 7.5, 15, 30]
},
"WritingEqualisationStandard": {
"enum": ["IEC", "IEC1", "IEC2"]
},
}
},
"ImageURI": {
"type": "string","format": "uri"
},
"AudioFileURI": {
"type": "string",
"format": "uri"
}
}
}
```
Here follows a partial list of requirements and their implementation status:
| N° | Done | Name | Priority | Description |
|:-:|:-:|---|:--:|--|
| 1 | ✔️ | Analyze single preservation file | high | The app should take in input a signle PreservationVisualFile an analyze it |
| 2 | ❌️ | Analyze a list of files | medium | The app should take in input a list of files and analyze them in batch. |
| 3 | ✔ | Output the analysis results | high | The app should create as output 2 irregularity files, one containing only irregularities from video and the other containing also audio irregularities from audio analyser |
| 4 | ❌ | Irregularity file on demand | high | The app should accept as input a number that specifies the irregularity file required: if 1 returns only the irregularities from video in the irregularity file, if 2 return an iregularity file with irregularities from both video and audio analyser |
✔️ = done | 🚧 = in progress | ❌ = todo
This diff is collapsed.
<!-- HTML footer for doxygen 1.9.1-->
<!-- start footer part -->
<!--BEGIN GENERATE_TREEVIEW-->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
$navpath
<li class="footer">$generatedby <a href="https://www.doxygen.org/index.html"><img class="footer" src="$relpath^doxygen.svg" width="104" height="31" alt="doxygen"/></a> $doxygenversion </li>
</ul>
</div>
<!--END GENERATE_TREEVIEW-->
<!--BEGIN !GENERATE_TREEVIEW-->
<hr class="footer"/><address class="footer"><small>
$generatedby&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="$relpath^doxygen.svg" width="104" height="31" alt="doxygen"/></a> $doxygenversion
</small></address>
<!--END !GENERATE_TREEVIEW-->
</body>
</html>
This diff is collapsed.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment