sc2_datasets.validators.validator_utils¶

Functions¶

`read_validation_file`(→ tuple[set[pathlib.Path], ...)	Attempts to read the validation file from a specified path.
`save_validation_file`() → None)	Attempts to save the validation file to a specified path.

Module Contents¶

read_validation_file(path: pathlib.Path) → tuple[set[pathlib.Path], set[pathlib.Path]]¶

Attempts to read the validation file from a specified path.

Parameters:: path (Path) – Specifies the path that will be used to read the validation file.
Returns:: Returns a tuple of sets containing files that were validated.
Return type:: tuple[set[Path], set[Path]]

Examples

This function is a helper that is required to have persistent validators which are able to skip the files that were previously processed. It is tasked with reading the validation file. Return of this function shoud contain information on which files were validated (all of the validated files), and which files ought to be skipped.

>>> from pathlib import Path
>>> validator_file_content = read_validation_file(path=Path("validator_file.json"))
>>> assert len(validator_file_content[0]) == 2
>>> assert len(validator_file_content[1]) == 1

save_validation_file(validated_files: set[pathlib.Path], skip_files: set[pathlib.Path], path: pathlib.Path = Path('validator_file.json')) → None¶

Attempts to save the validation file to a specified path.

Parameters:

validated_files (set[Path]) – Specifies the list of paths to replays that were verified as ones that can be used in further processing.
skip_files (set[Path]) – Specifies the list of paths to replays that were verified as ones that should be skipped in further processing.
path (Path, optional) – Specifies the path to the file that will be saved, by default Path(“validator_file.json”)

Examples

This function is a helper that is required to have persistent validators which are able to skip the files that were previously processed. It is tasked with saving the information that was processed by the validators so that future runs of the program can use this information.

>>> from pathlib import Path
>>> validated_files = {Path("validated_file_0.json"), Path("validated_file_1.json")}
>>> skip_files = {Path("validated_file_0.json")}
>>> validator_file_content = save_validation_file(
...                                         validated_files=validated_files,
...                                         skip_files=skip_files,
...                                         )