Skip to content
GitLab
Commit 2376d2a2 authored by Carl De Sousa Trias's avatar Carl De Sousa Trias
Browse files

Adding README 

Adding AIW multimodal question answering watermarking model 
Update of the controller.py
parent e6c23142
Hide whitespace changes
Inline Side-by-side
README.md 0 → 100644
View file @ 2376d2a2
# MPAI-NNW v1.1 implementation
This code refers to the implementation of the MPAI-NNW under MPAI-AIF, as described in https://mpai.community/wp-content/uploads/2023/10/Reference-Software-Neural-Network-Watermarking-V1.pdf.
All the code is based in Python (APIs and .
**Implemented APIs**
1. MPAI_AIFS_GetAndParseArchive, unzip and parse json and AIMs.
2. MPAI_AIFM_AIM_{Start,Pause,Resume,Stop,GetStatus}, to process on AIM/AIW.
3. MPAI_AIFM_Port_Input_{Write,Read,Reset}, to process on the port of the AIMs.
**Controller/User Agent**
1. Controller is deployed under the socket library (waiting request from _input.py_).
2. User Agent can trigger and run command by sending inputs.
3. _config.py_ shares some variables among the different files.
**Folders**
- **all_AIW** stores all the different AIW that are implemented
- NNW_NNW-QAM,NNW_NNW-QAM-Checker for the Multimodal Question Answering watermarking use case
- NNWImp,NNWRob for the controller_NNW
- **resources** external elements for some use cases (uncorrelated images for ADI, context/question of the MQA, ...)
- **Attacks** contains all the specified attacks of MPAI-NNW under the PyTorch Framework.
**Specificity to MPAI-NNW**
1. _utils.py_ contains function link to the dataset/dataloader under the PyToch framework.
2. _UCHIDA.py_ / _ADI.py_ correspond to the Neural Network Watermarking technology under evaluation.
3. AIW.zip is composed of the corresponding .json and the AIMs as Python file.
4. Folder _Attacks_ contains all the specified attacks of MPAI-NNW under the PyTorch Framework.
## Installation
Code was designed and tested on an Ubuntu 20.04 operating system using anaconda 23.7.2 and Python 3.9.
An environment with all the necessary libraries can be created using:
```bash
conda create --name <env> --file requirements.txt
```
## Run
**Initialisation**
First the Controller should be initialized (the command '-W ignore' can be added to avoid warning message during execution):
```bash
conda activate <env>
python controller.py
Controller Initialized
```
To send commands to the controller as a user agent,
a second terminal should be open, and run:
```bash
conda activate <env>
python input.py
input: <your command>
```
**Emulation of MPAI Store**
Emulate the folder of the computer as a website using the command:
```bash
python3 -m http.server
```
Then the command simulate the downloading of the AIW from a website:
```bash
conda activate <env>
python input.py
input: wget http://0.0.0.0:8000/[yourpath]/AIW.zip
```
### **List of command for controller**
Open a window for selecting the folder AIW.zip:
```bash
(env) python input.py
input: getparse
```
Run all, windows using tkinter will ask for the different files depending on the .zip parsed [AIW_MQA-NNW/AIW_MQA-NNW-Checker])
```bash
(env) python input.py
input: run all
```
**List of command for controller_NNW** (old)
for [AIWImp/AIWRob]
Open a window for selecting the folder AIW.zip:
```bash
(env) python input.py
input: getparse
```
Set the Computational Cost flag ON:
```bash
(env) python input.py
input: ComputationalCost True
```
Run the Robustness AIW with the **1** Modification and **{"P":0.5}** Parameters
```bash
(env) python input.py
input: run robustness 1 {"P":0.5}
```
Run the Imperceptibility AIW with **vgg16** as watermarked AIM and trained on the **CIFAR10** dataset
```bash
(env) python input.py
input: run imperceptibility vgg16 cifar10
```
### Some warnings
1. The AIW should be named AIW.zip and contained the .json and the needed AIMs.
2. The code does not permit misspelling.
# Licence
Licence information are detailed in the MPAI website
all_AIW/AIW_MQA-NNW-Checker/AIMs_files.py 0 → 100644
View file @ 2376d2a2
import torch
from transformers import AutoModelForQuestionAnswering
from datasets import load_dataset
from transformers import AutoTokenizer, pipeline
from tqdm.auto import tqdm
import numpy as np
import collections
import evaluate
import torch
from datasets import load_dataset
from transformers import AutoTokenizer, pipeline
from evaluate import load
from scipy.io.wavfile import write
from transformers.models.whisper.english_normalizer import BasicTextNormalizer
from tqdm import tqdm
from transformers.pipelines.pt_utils import KeyDataset
from utils import *
import wavmark
from wavmark.utils import file_reader
class NNWProof():
Answer = None
##
Proof = False
def funcNNWProof(self, input):
'''
Verify the inference
'''
payload = [0,1,1,1,1,0,0,0,0,1,1,0,1,0,1,1]
model = wavmark.load_model().to(device)
signal = file_reader.read_as_single_channel(input, aim_sr=16000)
# 5.decode watermark
payload_decoded, _ = wavmark.decode_watermark(model, signal, show_progress=True)
if isinstance(payload_decoded,type(None)): return False
BER=(payload != payload_decoded).mean() * 100
return BER==0
def run(self):
self.Proof = self.funcNNWProof(self.Answer)
\ No newline at end of file
all_AIW/AIW_MQA-NNW-Checker/NNWchecker.json 0 → 100644
View file @ 2376d2a2
{
"$schema": "",
"$id": "",
"title": "WaterChecker",
"Identifier": {
"ImplementerID": "/* String assigned by IIDRA */",
"Specification": {
"Standard": "MPAI-NNW",
"AIW": "NNW-WaterChecker",
"AIM": "NNW-WaterChecker",
"Version": "1"
}
},
"APIProfile": "basic",
"Description": "This AIF check the answer produce by an NN",
"Types": [
{
"Name":"answer_t",
"Type":"uint8[]"
},
{
"Name":"proof_t",
"Type":"boolean"
}
],
"Ports": [
{
"Name":"Answer",
"Direction":"InputOutput",
"RecordType":"answer_t"
},
{
"Name":"Proof",
"Direction":"OutputInput",
"RecordType":"proof_t"
}
],
"SubAIMs": [
{
"Name": "NNWProof",
"Identifier": {
"ImplementerID": "/* String assigned by IIDRA */",
"Specification": {
"Standard": "MPAI-NNW",
"AIW": "NNW-QAUsage",
"AIM": "NNWProof",
"Version": "1"
}
}
}
],
"Topology": [
{
"Output":{
"AIMName":"",
"PortName":"Answer"
},
"Input":{
"AIMName":"NNWProof",
"PortName":"Answer"
}
},
{
"Output":{
"AIMName":"NNWProof",
"PortName":"Proof"
},
"Input":{
"AIMName":"",
"PortName":"Proof"
}
}
]
}
all_AIW/AIW_MQA-NNW/AIMs_files.py 0 → 100644
View file @ 2376d2a2
import torch
from transformers import AutoModelForQuestionAnswering
from datasets import load_dataset
from transformers import AutoTokenizer, pipeline
from tqdm.auto import tqdm
import numpy as np
import collections
import evaluate
import torch
from datasets import load_dataset
from transformers import AutoTokenizer, pipeline
from evaluate import load
from scipy.io.wavfile import write
from transformers.models.whisper.english_normalizer import BasicTextNormalizer
from tqdm import tqdm
from transformers.pipelines.pt_utils import KeyDataset
from PIL import Image
import soundfile as sf
from lavis.models import load_model_and_preprocess
from utils import *
import wavmark
from playsound import playsound
class QuestionAnswering():
QuestionText = None
RawImage = None
##
AnswerText = None
def funcQuestionAnswering(self, raw_image_path,question):
'''
Apply an NN to answer the question
'''
raw_image=Image.open(raw_image_path).convert("RGB")
pipe = pipeline("visual-question-answering", model="Salesforce/blip-vqa-base")
output = pipe(raw_image, question, top_k=1)[0]
return output['answer']
def run(self):
self.AnswerText = self.funcQuestionAnswering(self.RawImage, self.QuestionText)
class SpeechRecognition():
QuestionAudio = None
##
QuestionText = None
def funcSpeechRecognition(self, input):
'''
Verify the inference
'''
if self.QuestionText == None:
playsound(input)
speech_reco = pipeline(
"automatic-speech-recognition", model="openai/whisper-base", device=device
)
res = speech_reco(input)
return res["text"]
def run(self):
self.QuestionText = self.funcSpeechRecognition(self.QuestionAudio)
class SpeechSynthesis():
AnswerText = None
AnswerAudio= None
def funcSpeechSynthesis(self,input):
synthesiser = pipeline("text-to-speech", "microsoft/speecht5_tts")
embeddings_dataset = load_dataset("Matthijs/cmu-arctic-xvectors", split="validation")
speaker_embedding = torch.tensor(embeddings_dataset[7306]["xvector"]).unsqueeze(0)
# You can replace this embedding with your own as well.
speech = synthesiser("The answer to your question is:"+input,
forward_params={"speaker_embeddings": speaker_embedding})
payload = [0,1,1,1,1,0,0,0,0,1,1,0,1,0,1,1]
model = wavmark.load_model().to(device)
signal, sample_rate = speech["audio"],speech["sampling_rate"]
watermarked_signal, _ = wavmark.encode_watermark(model, signal, payload, show_progress=True)
# you can save it as a new wav:
path_output = "AudioAnswer.wav"
sf.write(path_output, watermarked_signal, samplerate=16000)
playsound(path_output)
return path_output
def run(self):
self.AnswerAudio = self.funcSpeechSynthesis(self.AnswerText)
\ No newline at end of file
all_AIW/AIW_MQA-NNW/QuestionAnwseringSR.json 0 → 100644
View file @ 2376d2a2
{
"$schema": "",
"$id": "",
"title": "QASRUsage",
"Identifier": {
"ImplementerID": "/* String assigned by IIDRA */",
"Specification": {
"Standard": "MPAI-NNW",
"AIW": "NNW-QASRUsage",
"AIM": "NNW-QASRUsage",
"Version": "1"
}
},
"APIProfile": "basic",
"Description": "This AIF is an example of an integrated NNW use case",
"Types": [
{
"Name":"audio_t",
"Type":"uint8[]"
},
{
"Name":"question_t",
"Type":"uint8[]"
},
{
"Name":"image_t",
"Type":"uint8[]"
},
{
"Name":"answer_t",
"Type":"uint8[]"
},
{
"Name":"answer_audio_t",
"Type":"uint8[]"
}
],
"Ports": [
{
"Name":"QuestionAudio",
"Direction":"InputOutput",
"RecordType":"audio_t"
},
{
"Name":"QuestionText",
"Direction":"InputOutput",
"RecordType":"question_t"
},
{
"Name":"RawImage",
"Direction":"InputOutput",
"RecordType":"image_t"
},
{
"Name":"AnswerText",
"Direction":"InputOutput",
"RecordType":"answer_t"
},
{
"Name":"AnswerAudio",
"Direction":"InputOutput",
"RecordType":"answer_audio_t"
}
],
"SubAIMs": [
{
"Name": "QuestionAnswering",
"Identifier": {
"ImplementerID": "/* String assigned by IIDRA */",
"Specification": {
"Standard": "MPAI-NNW",
"AIW": "NNW-QAUsage",
"AIM": "QuestionAnswering",
"Version": "1"
}
}
},
{
"Name": "SpeechRecognition",
"Identifier": {
"ImplementerID": "/* String assigned by IIDRA */",
"Specification": {
"Standard": "MPAI-NNW",
"AIW": "NNW-QAUsage",
"AIM": "SpeechRecognition",
"Version": "1"
}
}
},
{
"Name": "SpeechSynthesis",
"Identifier": {
"ImplementerID": "/* String assigned by IIDRA */",
"Specification": {
"Standard": "MPAI-NNW",
"AIW": "NNW-QAUsage",
"AIM": "SpeechSynthesis",
"Version": "1"
}
}
}
],
"Topology": [
{
"Output":{
"AIMName":"",
"PortName":"QuestionAudio"
},
"Input":{
"AIMName":"SpeechRecognition",
"PortName":"QuestionAudio"
}
},
{
"Output":{
"AIMName":"SpeechRecognition",
"PortName":"QuestionText"
},
"Input":{
"AIMName":"QuestionAnswering",
"PortName":"QuestionText"
}
},
{
"Output":{
"AIMName":"",
"PortName":"RawImage"
},
"Input":{
"AIMName":"QuestionAnswering",
"PortName":"RawImage"
}
},
{
"Output":{
"AIMName":"QuestionAnswering",
"PortName":"AnswerText"
},
"Input":{
"AIMName":"SpeechSynthesis",
"PortName":"AnswerText"
}
},
{
"Output":{
"AIMName":"SpeechSynthesis",
"PortName":"AnswerAudio"
},
"Input":{
"AIMName":"",
"PortName":"AnswerAudio"
}
},
{
"Output":{
"AIMName":"SpeechRecognition",
"PortName":"QuestionText"
},
"Input":{
"AIMName":"",
"PortName":"QuestionText"
}
}
]
}
controller.py
View file @ 2376d2a2
# controller.py https://www.bogotobogo.com/python/python_network_programming_server_client.php # controller.py https://www.bogotobogo.com/python/python_network_programming_server_client.php
import socket import socket
import time import time
from Attacks import *
from UCHIDA import Uchi_tools
from ADI import Adi_tools
from multiprocessing import Process from multiprocessing import Process
from APIs import * from APIs import *
import psutil import psutil
import os import os
import ast import ast
import tkinter as tk import tkinter as tk
from tkinter import filedialog from tkinter import filedialog, simpledialog
import wget import wget
import config import config
from PIL import Image
# create a socket object # create a socket object
serversocket = socket.socket( serversocket = socket.socket(
...@@ -22,7 +20,7 @@ serversocket = socket.socket( ...@@ -22,7 +20,7 @@ serversocket = socket.socket(
# get local machine name # get local machine name
host = socket.gethostname() host = socket.gethostname()
port = 12345 port = 12468
# bind to the port # bind to the port
serversocket.bind((host, port)) serversocket.bind((host, port))
...@@ -58,9 +56,10 @@ while True: ...@@ -58,9 +56,10 @@ while True:
elif "getparse" in message[0].lower(): elif "getparse" in message[0].lower():
#print(type(message[1]),message[1]) #always str #print(type(message[1]),message[1]) #always str
root = tk.Tk() root = tk.Tk()
root.withdraw() root.withdraw()
filename = filedialog.askopenfilename(title='Select the parameter file', filetypes=(("Text files", filename = filedialog.askopenfilename(title='Select the zip (json and AIMs)', filetypes=(("Text files",
"*.zip"), "*.zip"),
("all files", ("all files",
"*.*"))) "*.*")))
...@@ -77,6 +76,7 @@ while True: ...@@ -77,6 +76,7 @@ while True:
# print(AIMs.keys()) # print(AIMs.keys())
# print(config.Topology) # print(config.Topology)
print(".json parsed") print(".json parsed")
elif 'write' in message[0].lower(): elif 'write' in message[0].lower():
## message[1] AIM_name, message[2] port_name, message[3] what to write ## message[1] AIM_name, message[2] port_name, message[3] what to write
MPAI_AIFM_Port_Input_Write(message[1],message[2],message[3]) MPAI_AIFM_Port_Input_Write(message[1],message[2],message[3])
...@@ -88,103 +88,28 @@ while True: ...@@ -88,103 +88,28 @@ while True:
## message[1] AIM_name, message[2] port_name ## message[1] AIM_name, message[2] port_name
MPAI_AIFM_Port_Reset(message[1],message[2]) MPAI_AIFM_Port_Reset(message[1],message[2])
elif 'ComputationalCost' in message[0].lower():
CompCostFlag=ast.literal_eval(str(message[1]))
elif 'run' in message[0].lower(): elif 'run' in message[0].lower():
### here you can want for a next message instead for elements in config.Topology:
# print("waiting for the type of run") # print(elements)
# clientsocket, addr = serversocket.accept() if elements["Output"]["AIMName"]=="": ## no outputs means it's an input
# data = clientsocket.recv(1024) ### TBD better: if conditions link to the port reading
# message = data.decode() root = tk.Tk()
# message = message.split() root.withdraw()
if "robustness" in message[1].lower(): path = filedialog.askopenfilename(title='Select '+str(elements["Input"]["PortName"]))
# run robustness 1 {"P":.1}
param = ast.literal_eval(str(message[3])) MPAI_AIFM_Port_Input_Write(elements["Input"]["AIMName"], elements["Input"]["PortName"],
M_ID = int(message[2]) path)
root = tk.Tk()
root.withdraw()
reload = filedialog.askopenfilename(title='Select the parameter file') # show an "Open" dialog box and return the path to the selected file
watermarked_parameters = torch.load(reload, map_location=torch.device('cpu'))
config.message["WatermarkedParameter"] = watermarked_parameters
tools = Uchi_tools()
root = tk.Tk()
root.withdraw()
reload_npy = filedialog.askopenfilename(title='Select the watermarking_dict')
watermarking_dict = np.load(reload_npy, allow_pickle=True).item()
MPAI_AIFM_Port_Input_Write("WatermarkDecoder", "tools", tools)
MPAI_AIFM_Port_Input_Write("WatermarkDecoder", "watermarking_dict", watermarking_dict)
MPAI_AIFM_Port_Input_Write("Comparator", "Payload", watermarking_dict["watermark"])
### Automatized
for elements in config.Topology:
# print(elements)
if elements["Output"]["AIMName"]=="":
MPAI_AIFM_Port_Input_Write(elements["Input"]["AIMName"], elements["Input"]["PortName"],
config.message[elements["Output"]["PortName"]])
else:
if CompCostFlag:
time1 = time.time()
MPAI_AIFM_AIM_Start(elements["Output"]["AIMName"])
if CompCostFlag:
time2 = time.time()
if elements["Input"]["AIMName"]=="":
print("Output of",elements["Output"]["AIMName"],"port",elements["Output"]["PortName"] )
print(MPAI_AIFM_Port_Output_Read(elements["Output"]["AIMName"],elements["Input"]["PortName"]))
else:
MPAI_AIFM_Port_Input_Write(elements["Input"]["AIMName"], elements["Input"]["PortName"],
MPAI_AIFM_Port_Output_Read(elements["Output"]["AIMName"],elements["Output"]["PortName"]))
MPAI_AIFM_AIM_Start("Comparator")
print('BER : %s' % (MPAI_AIFM_Port_Output_Read("Comparator","output_0")))
if CompCostFlag:
print("time of execution: %.5f sec" %(time2-time1))
elif "imperceptibility" in message[1].lower():
## run imperceptibility vgg16 cifar10
if "vgg16" in message[2].lower():
model = tv.models.vgg16()
model.classifier = nn.Linear(25088, 10)
else:
print(message[2],"not found - default loading vgg16")
model = tv.models.vgg16()
model.classifier = nn.Linear(25088, 10)
if "cifar10" in message[3].lower():
trainset,testset,tfm=CIFAR10_dataset()
else: else:
print(message[3],"not found - default loading CIFAR10") MPAI_AIFM_AIM_Start(elements["Output"]["AIMName"])
trainset,testset,tfm=CIFAR10_dataset() if elements["Input"]["AIMName"]=="": ## no inputs means it's an output
print("Output of",elements["Output"]["AIMName"],"- port",elements["Output"]["PortName"] )
MPAI_AIFM_Port_Input_Write("WatermarkEmbedder", "AIM", model) print()
if CompCostFlag: print(MPAI_AIFM_Port_Output_Read(elements["Output"]["AIMName"],elements["Input"]["PortName"]))
time1=time.time() print()
MPAI_AIFM_AIM_Start("WatermarkEmbedder") else:
if CompCostFlag: MPAI_AIFM_Port_Input_Write(elements["Input"]["AIMName"], elements["Input"]["PortName"],
time2=time.time() MPAI_AIFM_Port_Output_Read(elements["Output"]["AIMName"],elements["Output"]["PortName"]))
MPAI_AIFM_Port_Input_Write("AIM", "model", model)
MPAI_AIFM_Port_Input_Write("AIM", "parameters", MPAI_AIFM_Port_Output_Read("WatermarkEmbedder","output_0"))
MPAI_AIFM_Port_Input_Write("AIM", "testingDataset", testset)
MPAI_AIFM_AIM_Start("AIM")
print('AIM_watermarked_result : %s' % (MPAI_AIFM_Port_Output_Read("AIM","output_0")))
if CompCostFlag:
print("time of execution: %.5f sec" %(time2-time1))
MPAI_AIFM_Port_Input_Write("AIMtrainer", "AIM", model)
MPAI_AIFM_AIM_Start("AIMtrainer")
MPAI_AIFM_Port_Input_Write("AIM", "model", model)
MPAI_AIFM_Port_Input_Write("AIM", "parameters", MPAI_AIFM_Port_Output_Read("AIMtrainer","output_0"))
MPAI_AIFM_Port_Input_Write("AIM", "testingDataset", testset)
MPAI_AIFM_AIM_Start("AIM")
print('AIM_unwatermarked_result : %s' % (MPAI_AIFM_Port_Output_Read("AIM", "output_0")))
else:
print(message[1].lower(),"not implemented")
elif 'status' in message[0].lower(): elif 'status' in message[0].lower():
print(config.dict_process) print(config.dict_process)
......
controller_NNW.py 0 → 100644
View file @ 2376d2a2
# controller.py https://www.bogotobogo.com/python/python_network_programming_server_client.php
import socket
import time
from Attacks import *
from UCHIDA import Uchi_tools
from ADI import Adi_tools
from multiprocessing import Process
from APIs import *
import psutil
import os
import ast
import tkinter as tk
from tkinter import filedialog
import wget
import config
# create a socket object
serversocket = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)
# get local machine name
host = socket.gethostname()
port = 12345
# bind to the port
serversocket.bind((host, port))
# queue up to 5 requests
serversocket.listen(5)
print("Controller Initialized")
CompCostFlag=False
while True:
# establish a connection
clientsocket, addr = serversocket.accept()
# print("Got a connection from %s" % str(addr))
# currentTime = time.ctime(time.time()) + "\r\n"
data=clientsocket.recv(1024)
message=data.decode()
message = message.split()
if not data: break
if "help" in message[0].lower():
### to be updated
print(" ----------------------------------------")
print(" this program is the implementation of NNW in the AIF")
print(" you can run AIM/AIW by sending 'run XX' ")
print(" you can pause AIM/AIW by sending 'stop XX' ")
print(" you can resume AIM/AIW by sending 'resume XX' ")
print(" you can obtain the status of AIM/AIW by sending 'status XX' ")
print(" you can end the program by typing 'exit'")
print(" ----------------------------------------")
elif "wget" in message[0].lower():
test=wget.download(message[1])
print(type(test))
elif "getparse" in message[0].lower():
#print(type(message[1]),message[1]) #always str
root = tk.Tk()
root.withdraw()
filename = filedialog.askopenfilename(title='Select the parameter file', filetypes=(("Text files",
"*.zip"),
("all files",
"*.*")))
json_dict = MPAI_AIFS_GetAndParseArchive(filename)
time.sleep(.5)
import AIW.AIMs_files as AIMs_file
config.AIM_dict = json_dict['SubAIMs']
config.Topology = json_dict['Topology'] ### topology
for i in range(len(json_dict['SubAIMs'])):
config.AIMs[config.AIM_dict[i]["Name"]] = getattr(AIMs_file, config.AIM_dict[i]["Name"])()
### AIMs file should be in the .zip
# print(AIMs.keys())
# print(config.Topology)
print(".json parsed")
elif 'write' in message[0].lower():
## message[1] AIM_name, message[2] port_name, message[3] what to write
MPAI_AIFM_Port_Input_Write(message[1],message[2],message[3])
elif "read" in message[0].lower():
## message[1] AIM_name, message[2] port_name
result=MPAI_AIFM_Port_Output_Read(message[1],message[2])
print(message[2], "of", message[1], ":", result, type(result))
elif "reset" in message[0].lower():
## message[1] AIM_name, message[2] port_name
MPAI_AIFM_Port_Reset(message[1],message[2])
elif 'ComputationalCost' in message[0].lower():
CompCostFlag=ast.literal_eval(str(message[1]))
elif 'run' in message[0].lower():
### here you can want for a next message instead
# print("waiting for the type of run")
# clientsocket, addr = serversocket.accept()
# data = clientsocket.recv(1024)
# message = data.decode()
# message = message.split()
if "robustness" in message[1].lower():
# run robustness 1 {"P":.1}
param = ast.literal_eval(str(message[3]))
M_ID = int(message[2])
root = tk.Tk()
root.withdraw()
reload = filedialog.askopenfilename(title='Select the parameter file') # show an "Open" dialog box and return the path to the selected file
watermarked_parameters = torch.load(reload, map_location=torch.device('cpu'))
config.message["WatermarkedParameter"] = watermarked_parameters
tools = Uchi_tools()
root = tk.Tk()
root.withdraw()
reload_npy = filedialog.askopenfilename(title='Select the watermarking_dict')
watermarking_dict = np.load(reload_npy, allow_pickle=True).item()
MPAI_AIFM_Port_Input_Write("WatermarkDecoder", "tools", tools)
MPAI_AIFM_Port_Input_Write("WatermarkDecoder", "watermarking_dict", watermarking_dict)
MPAI_AIFM_Port_Input_Write("Comparator", "Payload", watermarking_dict["watermark"])
### Automatized
for elements in config.Topology:
# print(elements)
if elements["Output"]["AIMName"]=="":
MPAI_AIFM_Port_Input_Write(elements["Input"]["AIMName"], elements["Input"]["PortName"],
config.message[elements["Output"]["PortName"]])
else:
if CompCostFlag:
time1 = time.time()
MPAI_AIFM_AIM_Start(elements["Output"]["AIMName"])
if CompCostFlag:
time2 = time.time()
if elements["Input"]["AIMName"]=="":
print("Output of",elements["Output"]["AIMName"],"port",elements["Output"]["PortName"] )
print(MPAI_AIFM_Port_Output_Read(elements["Output"]["AIMName"],elements["Input"]["PortName"]))
else:
MPAI_AIFM_Port_Input_Write(elements["Input"]["AIMName"], elements["Input"]["PortName"],
MPAI_AIFM_Port_Output_Read(elements["Output"]["AIMName"],elements["Output"]["PortName"]))
MPAI_AIFM_AIM_Start("Comparator")
print('BER : %s' % (MPAI_AIFM_Port_Output_Read("Comparator","output_0")))
if CompCostFlag:
print("time of execution: %.5f sec" %(time2-time1))
elif "imperceptibility" in message[1].lower():
## run imperceptibility vgg16 cifar10
if "vgg16" in message[2].lower():
model = tv.models.vgg16()
model.classifier = nn.Linear(25088, 10)
else:
print(message[2],"not found - default loading vgg16")
model = tv.models.vgg16()
model.classifier = nn.Linear(25088, 10)
if "cifar10" in message[3].lower():
trainset,testset,tfm=CIFAR10_dataset()
else:
print(message[3],"not found - default loading CIFAR10")
trainset,testset,tfm=CIFAR10_dataset()
MPAI_AIFM_Port_Input_Write("WatermarkEmbedder", "AIM", model)
if CompCostFlag:
time1=time.time()
MPAI_AIFM_AIM_Start("WatermarkEmbedder")
if CompCostFlag:
time2=time.time()
MPAI_AIFM_Port_Input_Write("AIM", "model", model)
MPAI_AIFM_Port_Input_Write("AIM", "parameters", MPAI_AIFM_Port_Output_Read("WatermarkEmbedder","output_0"))
MPAI_AIFM_Port_Input_Write("AIM", "testingDataset", testset)
MPAI_AIFM_AIM_Start("AIM")
print('AIM_watermarked_result : %s' % (MPAI_AIFM_Port_Output_Read("AIM","output_0")))
if CompCostFlag:
print("time of execution: %.5f sec" %(time2-time1))
MPAI_AIFM_Port_Input_Write("AIMtrainer", "AIM", model)
MPAI_AIFM_AIM_Start("AIMtrainer")
MPAI_AIFM_Port_Input_Write("AIM", "model", model)
MPAI_AIFM_Port_Input_Write("AIM", "parameters", MPAI_AIFM_Port_Output_Read("AIMtrainer","output_0"))
MPAI_AIFM_Port_Input_Write("AIM", "testingDataset", testset)
MPAI_AIFM_AIM_Start("AIM")
print('AIM_unwatermarked_result : %s' % (MPAI_AIFM_Port_Output_Read("AIM", "output_0")))
else:
print(message[1].lower(),"not implemented")
elif 'status' in message[0].lower():
print(config.dict_process)
MPAI_AIFM_AIM_GetStatus(message[1])
elif 'pause' in message[0].lower():
MPAI_AIFM_AIM_Pause(message[1])
elif 'resume' in message[0].lower():
MPAI_AIFM_AIM_Resume(message[1])
elif 'stop' in message[0].lower():
if message[1].lower() in config.dict_process:
config.dict_process[message[1]].terminate()
print( message[1], "stopped")
else:
print(message[1], "isn't running")
elif "exit" in message[0].lower():
print("ending session...")
break
else:
print("input not implemented")
clientsocket.close()
print("session ended")
### TO DO https://docs.python.org/3/library/multiprocessing.html
resources/MPAIui.py 0 → 100644
View file @ 2376d2a2
import torch
import os
import wavmark
from wavmark.utils import file_reader
from PyQt5 import QtCore, QtGui, QtWidgets
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
global file_path_g
def funcNNWProof( input):
'''
Verify the inference
'''
payload = [0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1]
model = wavmark.load_model().to(device)
signal = file_reader.read_as_single_channel(input, aim_sr=16000)
# 5.decode watermark
payload_decoded, _ = wavmark.decode_watermark(model, signal, show_progress=True)
if isinstance(payload_decoded, type(None)): return False
BER = (payload != payload_decoded).mean() * 100
return BER == 0
class DragDropMainWindow(QtWidgets.QMainWindow):
fileDropped = QtCore.pyqtSignal(str)
def __init__(self, parent=None):
super(DragDropMainWindow, self).__init__(parent)
self.setAcceptDrops(True)
def dragEnterEvent(self, event):
if event.mimeData().hasUrls():
event.accept()
else:
event.ignore()
def dropEvent(self, event):
for url in event.mimeData().urls():
file_path = str(url.toLocalFile())
self.fileDropped.emit(file_path)
class Ui_MainWindow(object):
def setupUi(self, MainWindow):
MainWindow.setObjectName("MainWindow")
MainWindow.resize(800, 600)
self.centralwidget = QtWidgets.QWidget(MainWindow)
self.centralwidget.setObjectName("centralwidget")
self.pushButton = QtWidgets.QPushButton(self.centralwidget)
self.pushButton.setGeometry(QtCore.QRect(300, 470, 221, 61))
self.pushButton.setObjectName("pushButton")
self.pushButton_2 = QtWidgets.QPushButton(self.centralwidget)
self.pushButton_2.setGeometry(QtCore.QRect(590, 60, 191, 81))
self.pushButton_2.setObjectName("pushButton_2")
self.label = QtWidgets.QLabel(self.centralwidget)
self.label.setGeometry(QtCore.QRect(70, 110, 721, 391))
self.label.setObjectName("label")
MainWindow.setCentralWidget(self.centralwidget)
self.statusbar = QtWidgets.QStatusBar(MainWindow)
self.statusbar.setObjectName("statusbar")
MainWindow.setStatusBar(self.statusbar)
self.retranslateUi(MainWindow)
QtCore.QMetaObject.connectSlotsByName(MainWindow)
# Connect button clicks to functions
self.pushButton.clicked.connect(self.run_UseCase)
self.pushButton_2.clicked.connect(self.watermark_proof)
self.file_path_g=None
def retranslateUi(self, MainWindow):
_translate = QtCore.QCoreApplication.translate
MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow"))
self.pushButton.setText(_translate("MainWindow", "Running the UseCase"))
self.pushButton_2.setText(_translate("MainWindow", "Watermarking proof"))
self.label.setText(_translate("MainWindow", "<html><head/><body><p><img src=\"MPAI_NNW-MQA.png\"/></p></body></html>"))
def run_UseCase(self):
# Function to execute when the "Running the UseCase" button is clicked
print("Openning new Window")
os.system("gnome-terminal & disown")
def watermark_proof(self):
# Function to execute when the "WaterMarking proof" button is clicked
if self.file_path_g==None:
print("Please, first drag an audio file")
else:
print("Processing...")
answer=funcNNWProof(self.file_path_g)
if answer:
print("This audio is watermarked")
else:
print("This audio is not watermarked")
if __name__ == "__main__":
import sys
app = QtWidgets.QApplication(sys.argv)
MainWindow = DragDropMainWindow()
ui = Ui_MainWindow()
ui.setupUi(MainWindow)
def file_dropped(file_path):
ui.file_path_g=file_path
MainWindow.fileDropped.connect(file_dropped)
MainWindow.show()
sys.exit(app.exec_())
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment