Skip to content

python.md

Latest commit

 

History

History
380 lines (244 loc) · 16.5 KB

python.md

File metadata and controls

380 lines (244 loc) · 16.5 KB
Raw
description A detailed usage guide for installing and using catch22 in Python.
cover https://images.unsplash.com/photo-1614189313880-13b09111b4de?crop=entropy&cs=srgb&fm=jpg&ixid=M3wxOTcwMjR8MHwxfHNlYXJjaHw0fHxpbnN0YWxsYXRpb258ZW58MHx8fHwxNzExMzMxMzM5fDA&ixlib=rb-4.0.3&q=85
coverY 169

Python

Python Usage Guide

Select a card below to access Python-specific usage information.

Installationinstall_light.png#installation
Getting startedgetting_started_light.png#getting-started-basic-usage
Advanced usageadv_usage_light.png#advanced-usage
Frequently Asked QuestionsFAQ_light.png#faq
API ReferenceAPI_light.pngpython-api.md

Installation

Python users can install pycatch22 via pip using the following command:

pip install pycatch22

pycatch22 supports Python 3.8 and later, and has no runtime dependencies (numpy is optional). pycatch22 v1.0 uses catch22 v1.1.0.

Additionally, pycatch22 can also be installed using the following alternative installation options. Click the expandable drop-down tabs to explore these options:

Alternative Installation Option 1: Anaconda/Mamba

pycatch22 is also available as a package on anaconda which you can install via conda or mamba using:

conda install -c conda-forge pycatch22

OR

mamba install -c conda-forge pycatch22

The conda-forge package can lag behind PyPI, so check that it gives you version 1.0 or later (conda list pycatch22).

Alternative Installation Option 2: Manual Install

This should not be required, but if you find issues with the pip or conda install options, you can build pycatch22 from source (this needs a C compiler).

First, clone the pycatch22 repository to your local machine, including the catch22 C code (a git submodule), using:

git clone --recurse-submodules https://github.com/DynamicsAndNeuralSystems/pycatch22.git

Navigate to the location of the cloned repository and install manually using:

pip install .

Getting Started: Basic Usage

Here we outline how you can jump straight into computing catch22 features on your time series data with a basic usage example.

{% hint style="info" %} Upgrading from pycatch22 v0.x? pycatch22 v1.0 computes catch22 v1.0, which changed the values of most features (to match hctsa v3.0.0 exactly) and renamed several of them, so results are not comparable with v0.x. The features are also returned in a different order (e.g., the outlier-timing features are now 3rd and 4th), so select features by name rather than by position. See the feature descriptions for what changed. {% endhint %}

Expected Input

In pycatch22, a time series can be any sequence of numbers: a list, a tuple, or a numpy array:

import numpy as np
import pycatch22

rng = np.random.default_rng(42)
x = np.sin(np.linspace(0, 20*np.pi, 1000)) + 0.5*rng.standard_normal(1000)  # (or more interesting data!)
tsData_asList = list(x)

Input that is not a sequence of numbers raises a TypeError.

Computing catch22 features

All features are bundled in the method catch22_all which can simply be called on your time series data as shown below:

# for time series data as a numpy array...
results = pycatch22.catch22_all(x)
# alternatively, for time series data supplied as a list...
results_list = pycatch22.catch22_all(tsData_asList)

Expected output

The results of your calculation will be stored in a dictionary with the keys: names and values:

  • names: the feature names, as listed in the feature table, returned as a list of strings.
  • values: the feature outputs returned as a list of floats.

Note that the ordering of the feature names is the same as that in which the outputs are given i.e., the first feature in names has the output corresponding to the first (zeroth) index in values.

Continuing our example from above, we can view the features and their computed values with the following:

for feature, output in zip(results['names'], results['values']):
    print(f"{feature}: {output}")

Which gives the following output:

DN_HistogramMode_5: 0.0335441836925168
DN_HistogramMode_10: -0.7462245693351899
DN_OutlierInclude_p_001_mdrmd: 0.014000000000000012
DN_OutlierInclude_n_001_mdrmd: 0.131
CO_FirstCrossing_acf_1e_pointOfCrossing: 15.700534880150066
CO_FirstMin_ac: 2.0
...
SC_FluctAnal_2_dfa_50_1_2_logi_prop_r1: 0.5652173913043478

To look up a feature by name, you can make a dictionary of the results, e.g., dict(zip(results['names'], results['values']))['CO_FirstMin_ac'].

And that's it! You now have all the knowledge you need to apply catch22 to your time series data. To access some of the additional functionality of catch22, keep reading for advanced usage tips and examples.

Advanced Usage

Ready to explore some of the additional functionality of pycatch22?

1. Catch24

If the location and spread of the raw time series distribution may be important for your application, you can enable catch24.

{% tabs %} {% tab title="Catch24" %} Catch24 is an extension of the original catch22 feature set to include mean and standard deviation, yielding a total of 24 time series features. To access catch24, you can pass an additional parameter to the catch22_all method, telling it to compute the catch24 feature set as follows:

c24_results = pycatch22.catch22_all(x, catch24=True)
print(c24_results['names'][-2:])  # ['DN_mean', 'DN_Spread_std']

{% endtab %} {% endtabs %}

2. Computing Individual Features

If you only need one feature, you can compute it on its own (including the mean and standard deviation).

{% tabs %} {% tab title="Computing Individual Features" %} In pycatch22 each feature has its own function, named by the feature's long name (as in the feature overview table), e.g., CO_trev_1_num for computing trev. It takes the same input as catch22_all (and the same mask_nans option) and returns a float.

As example, let's compute the feature trev for the time series x from above:

trev_value = pycatch22.CO_trev_1_num(x)
print(f"trev feature output: {trev_value}")
# trev feature output: -0.08153158906199524

You can also compute a feature by its long or short name with compute_feature:

trev_value = pycatch22.compute_feature(x, 'trev')  # or 'CO_trev_1_num'
# -0.08153158906199524

Either way, only that feature is computed, and its value is exactly the one that catch22_all gives.

The long names used before v1.0 (e.g., CO_f1ecac, now CO_FirstCrossing_acf_1e_pointOfCrossing) still work as deprecated aliases: they compute the v1.0 feature and issue a DeprecationWarning. {% endtab %} {% endtabs %}

3. Short Names

For each feature, we also include a unique 'short name' for easier reference (as outlined in the feature overview table).

{% tabs %} {% tab title="Short Names" %} In pycatch22 short names can be included in the output when calling catch22_all() by setting short_names = True:

results = pycatch22.catch22_all(x, short_names=True)

The returned dictionary, results, now has three keys: names, values and short_names. As before, names is a list of the long feature names (as strings), values is a list of feature values, and short_names is a list of short names (as strings).

Continuing from the basic usage example, let's try printing the feature outputs along with their corresponding short names:

for (short_name, val) in zip(results['short_names'], results['values']):
    print(f"{short_name}: {val}")

Here is the output from our example. A lot more readable!

mode_5: 0.0335441836925168
mode_10: -0.7462245693351899
outlier_timing_pos: 0.014000000000000012
outlier_timing_neg: 0.131
acf_timescale: 15.700534880150066
...
dfa: 0.5652173913043478

pycatch22.feature_table() lists every feature's name, short name, name before v1.0 (legacy_name), and the hctsa code that reproduces it (hctsa_call), as a list of dictionaries:

pycatch22.feature_table()[4]
# {'name': 'CO_FirstCrossing_acf_1e_pointOfCrossing', 'short_name': 'acf_timescale', 'legacy_name': 'CO_f1ecac', 'hctsa_call': "CO_FirstCrossing(x_z,'ac',1/exp(1),'both').pointOfCrossing"}

{% endtab %} {% endtabs %}

4. Masking

Some features cannot be estimated for some time series, e.g., any feature of a constant time series, or dfa for a time series shorter than 79 samples.

{% tabs %} {% tab title="Masking" %} By default, such features are masked: set to a fixed value at the appropriate end of the feature's range, so that 22 real numbers are always returned. Set return_masked=True to also return masked, which says which values were masked, and mask_nans=False to get NaN instead:

y = np.random.default_rng(1).standard_normal(50)  # too short for dfa
res = pycatch22.catch22_all(y, short_names=True, return_masked=True)
print([(name, value) for name, value, m in zip(res['short_names'], res['values'], res['masked']) if m])
# [('dfa', 1.0)]

res = pycatch22.catch22_all(y, mask_nans=False)
print(res['values'][-1])
# nan

A time series that is empty or contains NaN or Inf gives NaN for every feature (it is never masked), with a RuntimeWarning. {% endtab %} {% endtabs %}

5. Multithreading

When working with large time series datasets (containing many univariate time series instances), computing features can often become a significant computational bottleneck. This is where multi-threading comes into play, offering a powerful solution to accelerate feature computation through parallelisation.

{% tabs %} {% tab title="Multithreading" %} From v1.0, pycatch22 releases Python's global interpreter lock (GIL) while it computes features, so features of different time series can be computed in parallel threads (with earlier versions, threads gave no speedup).

  1. Let's start by importing all of the libraries we will need:
import os
from concurrent.futures import ThreadPoolExecutor
import numpy as np
import pycatch22

and then create a large time series dataset consisting of 10 000 univariate time series instances, each of length 1000:

# generate list where each entry is a length 1000 time series
rng = np.random.default_rng(0)
dataset = [rng.standard_normal(1000) for _ in range(10000)]
  1. Next, we will need to define a function which takes in a single time series, applies catch22, and then returns the feature values:
def compute_features(x):
    return pycatch22.catch22_all(x)['values']
  1. Now we can compute the features of all the time series in parallel, with one thread per CPU core (os.cpu_count()):
with ThreadPoolExecutor(max_workers=os.cpu_count()) as executor:
    feature_matrix = np.array(list(executor.map(compute_features, dataset)))

print(feature_matrix.shape)  # (10000, 22): one row per time series

The speedup grows with the number of threads: for example, 4 threads computed the features of 200 time series of length 5000 about 3.8 times faster than a single thread (on Apple Silicon). {% endtab %} {% endtabs %}

FAQ

Click one of the expandable tabs below to explore commonly asked questions about pycatch22:

How can I submit a bug report for pycatch22?

If you would like to submit a bug report, you can access our Bug Tracker here. For guidelines on submitting an ideal bug report, see our page on contributing to catch22.

Which Python versions are compatible with pycatch22?

We have conducted thorough unit testing across Python versions 3.8 to 3.13 to verify and guarantee compatibility. These tests are run on Windows, macOS, and Linux, to ensure consistent performance and reliability across different platforms.

How can I install pycatch22 on my system?

We have a detailed installation guide here, including instructions for installing using pip, as well as alternative installation options including conda and manual (local) install.

How can I handle large datasets with many univariate time series?

While pycatch22 only takes a single univariate time series as input, when working with large time series datasets (containing many univariate time series instances), computing features can often become a significant computational bottleneck. While a simple for loop may suffice, such tasks are well suited to multi-threading approaches.

See above in our advanced usage guide for how you can compute catch22 features on large datasets using multi-threading.

Why do my feature values differ from an earlier version of pycatch22?

pycatch22 v1.0 computes catch22 v1.0, in which each feature exactly reproduces a feature of hctsa v3.0.0. This changed the values of most features, renamed several of them, and changed the order in which they are returned. Use pycatch22.feature_table() to match the new names to the old ones, and see the feature descriptions for how each feature changed. When reporting results, state the version you used (see Citing catch22).

How does pycatch22 integrate with sktime for time series analysis?

sktime, a comprehensive Python library for time series machine learning, leverages catch22's capabilities within a broader ecosystem of time series analysis tools for classification. For users looking to incorporate catch22's feature extraction into their sktime pipelines, they can do so by calling the Catch22Classifier which implements a random forest classifier using catch22 features calculated on time series data.

Tutorials for using catch22 within the sktime framework are also available here.