evaluate_clusterers_by_problem

tsml_eval.evaluation.evaluate_clusterers_by_problem(load_path, clusterer_names, dataset_names, save_path, resamples=None, load_test_results=True, error_on_missing=True, eval_name=None, verify_results=True, verbose=False)

Evaluate multiple clusterers on multiple datasets from file using standard paths.

Finds files using clusterer, dataset and resample names. It is expected the common tsml-eval file structure of {clusterer}/Predictions/{dataset}/{split}Resample{resample}.csv is followed.

Writes multiple csv files and figures to save_path, one for each statistic evaluated. Provides a summary csv file with the average statistic and average rank for each clusterer.

Parameters:

load_pathstr or list of str

The path to the collection of clusterer result files to evaluate. If load_path is a list, it will load results from each path in the list. It is expected that clusterer_names and dataset_names are lists of lists with the same length as load_path.

clusterer_nameslist of str, list of tuple or list of list

The names of the clusterers to evaluate. A length 2 tuple containing strings can be used to specify a clusterer name to load from in the first item and a clusterer name to use in the evaluation results in the second. If load_path is a list, clusterer_names must be a list of lists with the same length as load_path.

dataset_namesstr, list of str or list of list

The names of the datasets to evaluate. If a list of strings, each item is the name of a dataset. If a string, it is the path to a file containing the names of the datasets, one per line. If load_path is a list, dataset_names must be a list of lists with the same length as load_path.

save_pathstr

The path to save the evaluation results to.

resamplesint or list of int, default=None

The resamples to evaluate. If int, evaluates resamples 0 to resamples-1.

load_test_resultsbool, default=True

Whether to load test results as well as train results.

error_on_missingbool, default=True

Whether to raise an error if results are missing.

eval_namestr, default=None

The name of the evaluation, used in save_path.

verify_resultsbool, default=True

If the verification should be performed on the loaded results values.

verbosebool, default=False

If verbose output should be printed.