API

This page contains the auto-generated API documentation for tsml-eval package functions and classes.

Evaluation: tsml_eval.evaluation

Functions for evaluating the performance of a model.

evaluation.evaluate_classifiers(...[, ...])	Evaluate multiple classifiers on multiple datasets.
evaluation.evaluate_classifiers_from_file(...)	Evaluate multiple classifiers on multiple datasets from file.
evaluation.evaluate_classifiers_by_problem(...)	Evaluate multiple classifiers on multiple datasets from file using standard paths.
evaluation.evaluate_clusterers(...[, ...])	Evaluate multiple clusterers on multiple datasets.
evaluation.evaluate_clusterers_from_file(...)	Evaluate multiple clusterers on multiple datasets from file.
evaluation.evaluate_clusterers_by_problem(...)	Evaluate multiple clusterers on multiple datasets from file using standard paths.
evaluation.evaluate_regressors(...[, ...])	Evaluate multiple regressors on multiple datasets.
evaluation.evaluate_regressors_from_file(...)	Evaluate multiple regressors on multiple datasets from file.
evaluation.evaluate_regressors_by_problem(...)	Evaluate multiple regressors on multiple datasets from file using standard paths.
evaluation.evaluate_forecasters(...[, ...])	Evaluate multiple forecasters on multiple datasets.
evaluation.evaluate_forecasters_from_file(...)	Evaluate multiple forecasters on multiple datasets from file.
evaluation.evaluate_forecasters_by_problem(...)	Evaluate multiple forecasters on multiple datasets from file using standard paths.
evaluation.storage.ClassifierResults([...])	A class for storing and managing results from classification experiments.
evaluation.storage.ClustererResults([...])	A class for storing and managing results from clustering experiments.
evaluation.storage.ForecasterResults([...])	A class for storing and managing results from forecasting experiments.
evaluation.storage.RegressorResults([...])	A class for storing and managing results from regression experiments.
evaluation.storage.load_classifier_results(...)	Load and return classifier results from a specified file.
evaluation.storage.load_clusterer_results(...)	Load and return clusterer results from a specified file.
evaluation.storage.load_forecaster_results(...)	Load and return forecaster results from a specified file.
evaluation.storage.load_regressor_results(...)	Load and return regressor results from a specified file.
evaluation.efficiency_benchmark.compare_estimators(...)	Perform a sequence of benchmarks for two estimators.
evaluation.efficiency_benchmark.benchmark_estimator(...)	Benchmark estimator's runtime and memory usage on X and y.
evaluation.metrics.clustering_accuracy_score(...)	Calculate clustering accuracy.

Experiments: tsml_eval.experiments

Functions for running experiments.

experiments.run_classification_experiment(...)	Run a classification experiment and save the results to file.
experiments.load_and_run_classification_experiment(...)	Load a dataset and run a classification experiment.
experiments.run_clustering_experiment(...[, ...])	Run a clustering experiment and save the results to file.
experiments.load_and_run_clustering_experiment(...)	Load a dataset and run a clustering experiment.
experiments.run_forecasting_experiment(...)	Run a forecasting experiment and save the results to file.
experiments.load_and_run_forecasting_experiment(...)	Load a dataset and run a regression experiment.
experiments.run_regression_experiment(...[, ...])	Run a regression experiment and save the results to file.
experiments.load_and_run_regression_experiment(...)	Load a dataset and run a regression experiment.
experiments.set_classifier	Set classifier function.
experiments.set_clusterer	Set classifier function.
experiments.set_forecaster	Set forecaster function.
experiments.set_regressor	Set regressor function.

Utilities: tsml_eval.utils

Public utility functions used elsewhere in the package.

utils.arguments.parse_args(args)	Parse the command line arguments for tsml_eval.
utils.experiments.resample_data(X_train, ...)	Resample data without replacement using a random state.
utils.experiments.resample_data_indices(...)	Return data resample indices without replacement using a random state.
utils.experiments.stratified_resample_data(...)	Stratified resample data without replacement using a random state.
utils.experiments.stratified_resample_data_indices(...)	Return stratified data resample indices without replacement using a random state.
utils.experiments.load_experiment_data(...)	Load data for experiments.
utils.experiments.write_classification_results(...)	Write the predictions for a classification experiment in the format used by tsml.
utils.experiments.write_regression_results(...)	Write the predictions for a regression experiment in the format used by tsml.
utils.experiments.write_clustering_results(...)	Write the predictions for a clustering experiment in the format used by tsml.
utils.experiments.write_results_to_tsml_format(...)	Write the predictions for an experiment in the standard format used by tsml.
utils.experiments.fix_broken_second_line(...)	Fix a results while where the written second line has line breaks.
utils.experiments.compare_result_file_resample(...)	Validate that a two results files use the same data resample.
utils.experiments.assign_gpu([set_environ])	Assign a GPU to the current process.
utils.experiments.timing_benchmark([...])	Measures the time taken to sort a given number of numpy arrays of a specified size.
utils.experiments.estimator_attributes_to_file(...)	Write the attributes of an estimator to file(s).
utils.functions.str_in_nested_list(...)	Find an item in a nested list.
utils.functions.pair_list_to_dict(pl)	Convert a 2d list of pairs to a dict.
utils.functions.time_to_milliseconds(...)	Convert a time value from the given time unit to milliseconds.
utils.functions.rank_array(arr[, higher_better])	Assign a rank to each value in a 1D numpy array.
utils.memory_recorder.record_max_memory(function)	Record the maximum memory usage of a function.
utils.validation.is_sklearn_estimator(estimator)	Check if estimator is a scikit-learn estimator.
utils.validation.is_sklearn_classifier(...)	Check if estimator is a scikit-learn classifier.
utils.validation.is_sklearn_regressor(regressor)	Check if estimator is a scikit-learn regressor.
utils.validation.is_sklearn_clusterer(clusterer)	Check if estimator is a scikit-learn clusterer.
utils.validation.validate_results_file(file_path)	Validate that a results file is in the correct format.