Skip to content

Validation

Table of Contents

validation.evaluator

Abstract class for evaluators

Evaluator Objects

class Evaluator(ABC)

Abstract class for evaluators

evaluate

def evaluate(current_run: dict)

Runs the evaluation if necessary, e.g. conducting a forward pass on the validation sets

Arguments:

  • current_run: Dict containing details on the current run including dataset, no_clients

generate_report

def generate_report()

Generates a report on the evaluation, needs to be run after evaluate

validation.validation

This module contains the Validation class, which is used to evaluate the performance of a federated learning run

Validation Objects

class Validation(Evaluator)

evaluate

def evaluate(current_run: dict)

Evaluates the performance of a federated learning run

Arguments:

  • current_run: Dictionary containing the parameters of the current run, e.g. algorithm, no_clients, etc.

Returns:

None

generate_report

def generate_report()

Generates an HTML report with the results of the validation

Returns:

None

validation.training

Class for training performance evaluation

Training Objects

class Training(Evaluator)

Class for training performance evaluation

generate_report

def generate_report()

Generates a report on the training performance (e.g. loss, accuracy), diagrams and stores it as a .html file

validation.datadistribution

Data Distribution Evaluator

DataDistribution Objects

class DataDistribution(Evaluator)

Data Distribution Evaluator

evaluate

def evaluate(current_run: dict)

Evaluates the data distribution

Arguments:

  • current_run: Dict containing details on the current run including dataset, no_clients

generate_report

def generate_report()

Generates a report on the data distribution and saves it to the output directory