README.md 1.09 KB
Newer Older
1
# MPAI-MMC Text and Image Query
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
2

3
This code refers to the implementation of the MMC-TIQ, as described in the [AIM](https://mpai.community/standards/mpai-mmc/v2-2/ai-modules/text-and-image-query/)
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
4
5


6
## Guide to the TIQ code
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
7

8
Note that the Reference software implements the Basic MMC-TIQ AIM.
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
9

10
Use of this AI Module is for developers who are familiar with Python and downloading models from HuggingFace,
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
11

12
A wrapper for the BLIP NN Module:
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
13

14
15
16
    1. Manages input files and parameters: Text Object, Visual Object
    2. Executes the BLIP Module to perform the question answering on each individual pair of Text and Visual Object.
    3. Outputs Text Object as answer.
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
17

18
The OSD-TIQ Reference Software is found at the NNW gitlab site. It contains:
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
19

20
21
    1. The python code implementing the AIM.
    2. Required libraries are: pytorch and transformers (HuggingFace), PIL
Carl De Sousa Trias's avatar
Carl De Sousa Trias committed
22
23
24
25



## Installation
26
27
28
29
30
Code was designed and tested on an Ubuntu 20.04 operating system using anaconda 23.7.2 and Python 3.9.
An environment with all the necessary libraries can be created using:
```bash
conda create --name <env> --file requirements.txt
```