Utilities

BIDS Utilities

CPAC.utils.bids_utils.bids_decode_fname(file_path, dbg=False, raise_error=True)[source]
CPAC.utils.bids_utils.bids_entities_from_filename(filename)[source]

Function to collect a list of BIDS entities from a given filename.

Parameters:

filename (str)

Returns:

entities

Return type:

list

Examples

>>> bids_entities_from_filename(
...     's3://fake/data/sub-0001/ses-NFB3/func/'
...     'sub-0001_ses-NFB3_task-MSIT_bold.nii.gz')
['sub-0001', 'ses-NFB3', 'task-MSIT', 'bold']
CPAC.utils.bids_utils.bids_gen_cpac_sublist(bids_dir, paths_list, config_dict, creds_path, dbg=False, raise_error=True, only_one_anat=True)[source]

Generates a CPAC formatted subject list from information contained in a BIDS formatted set of data.

Parameters:
  • bids_dir (str) – base directory that contains all of the data, this could be a directory that contains data for a multiple BIDS datasets, in which case the intervening directories will be interpreted as site names

  • paths_list (str) – lists of all nifti files found in bids_dir, these paths are relative to bids_dir

  • config_dict (dict) – dictionary that contains information from the JSON sidecars found in bids_dir, keys are relative paths and values are dictionaries containing all of the parameter information. if config_dict is None, the subject list will be built without the parameters

  • creds_path (str) – if using S3 bucket, this path credentials needed to access the bucket, if accessing anonymous bucket, this can be set to None

  • dbg (bool) – indicating whether or not the debug statements should be printed

  • raise_error (bool)

  • only_one_anat (bool) – The “anat” key for a subject expects a string value, but we can temporarily store a list instead by passing True here if we will be filtering that list down to a single string later

Returns:

a list of dictionaries suitable for use by CPAC to specify data to be processed

Return type:

list

CPAC.utils.bids_utils.bids_match_entities(file_list, entities, suffix)[source]

Function to subset a list of filepaths by a passed BIDS entity.

Parameters:
  • file_list (list of str)

  • entities (str) – BIDS entities joined by underscores (e.g., ‘ses-001_task-PEER1’)

  • suffix (str) – BIDS suffix (e.g., ‘bold’, ‘T1w’)

Return type:

list of str

Examples

>>> bids_match_entities([
...     's3://fake/data/sub-001_ses-001_task-MSIT_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ], 'task-PEER1', 'bold')
['s3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz']
>>> bids_match_entities([
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ], 'PEER', 'bold')
Traceback (most recent call last):
LookupError: No match found for provided entity "PEER" in
- s3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz
- s3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz
Perhaps you meant one of these?
- task-PEER1
- task-PEER2
CPAC.utils.bids_utils.bids_parse_sidecar()[source]

Uses the BIDS principle of inheritance to build a data structure that maps parameters in side car .json files to components in the names of corresponding nifti files.

Parameters:
  • config_dict – dictionary that maps paths of sidecar json files (the key) to a dictionary containing the contents of the files (the values)

  • dbg – boolean flag that indicates whether or not debug statements should be printed

Returns:

a dictionary that maps parameters to components from BIDS filenames such as sub, sess, run, acq, and scan type

CPAC.utils.bids_utils.bids_remove_entity(name, key)[source]

Remove an entity from a BIDS string by key.

Parameters:
  • name (str) – BIDS string to remove entity from

  • key (str) – BIDS key of entity to remove

Returns:

BIDS name with entity removed

Return type:

str

Examples

>>> bids_remove_entity('atlas-Yeo_space-MNI152NLin6_res-2x2x2', 'space')
'atlas-Yeo_res-2x2x2'
>>> bids_remove_entity('atlas-Yeo_space-MNI152NLin6_res-2x2x2', 'res')
'atlas-Yeo_space-MNI152NLin6'
CPAC.utils.bids_utils.bids_retrieve_params(bids_config_dict, f_dict, dbg=False)[source]

Retrieve the BIDS parameters from bids_config_dict for BIDS file corresponding to f_dict. If an exact match for f_dict is not found the nearest match is returned, corresponding to the BIDS inheritance principle.

Parameters:
  • bids_config_dict – BIDS configuration dictionary, this is a multi-level dictionary that maps the components of a bids filename (i.e. sub, ses, acq, run) to a dictionary that contains the BIDS parameters (RepetitionTime, EchoTime, etc). This information is extracted from sidecar json files using the principle of inheritance using the bids_parse_configs function

  • f_dict – Dictionary built from the name of a file in the BIDS format. This is built using the bids_decode_fname by splitting on “-” and “_” delimeters

  • dbg – boolean flag that indicates whether or not debug statements should be printed, defaults to “False”

Returns:

returns a dictionary that contains the BIDS parameters

CPAC.utils.bids_utils.bids_shortest_entity(file_list)[source]

Function to return the single file with the shortest chain of BIDS entities from a given list, returning the first if more than one have the same minimum length.

Parameters:

file_list (list of strings)

Return type:

str or None

Examples

>>> bids_shortest_entity([
...     's3://fake/data/sub-001_ses-001_task-MSIT_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ])
's3://fake/data/sub-001_ses-001_bold.nii.gz'
CPAC.utils.bids_utils.camelCase(string: str) str[source]

Convert a hyphenated string to camelCase.

Parameters:

string (str) – string to convert to camelCase

Return type:

str

Examples

>>> camelCase('PearsonNilearn-aCompCor')
'PearsonNilearnACompCor'
>>> camelCase('mean-Pearson-Nilearn-aCompCor')
'meanPearsonNilearnACompCor'
CPAC.utils.bids_utils.cl_strip_brackets(arg_list)[source]

Removes ‘[’ from before first and ‘]’ from after final arguments in a list of commandline arguments.

Parameters:

arg_list (list)

Return type:

list

Examples

>>> cl_strip_brackets('[a b c]'.split(' '))
['a', 'b', 'c']
>>> cl_strip_brackets('a b c'.split(' '))
['a', 'b', 'c']
>>> cl_strip_brackets('[ a b c ]'.split(' '))
['a', 'b', 'c']
CPAC.utils.bids_utils.collect_bids_files_configs(bids_dir, aws_input_creds='')[source]
Parameters:
  • bids_dir

  • aws_input_creds

Returns:

CPAC.utils.bids_utils.combine_multiple_entity_instances(bids_str: str) str[source]

Combines mutliple instances of a key in a BIDS string to a single instance by camelCasing and concatenating the values.

Parameters:

bids_str (str)

Return type:

str

Examples

>>> combine_multiple_entity_instances(
...     'sub-1_ses-HBN_site-RU_task-rest_atlas-AAL_'
...     'desc-Nilearn_desc-36-param_suffix.ext')
'sub-1_ses-HBN_site-RU_task-rest_atlas-AAL_desc-Nilearn36Param_suffix.ext'
>>> combine_multiple_entity_instances(
...     'sub-1_ses-HBN_site-RU_task-rest_'
...     'run-1_framewise-displacement-power.1D')
'sub-1_ses-HBN_site-RU_task-rest_run-1_framewiseDisplacementPower.1D'
CPAC.utils.bids_utils.create_cpac_data_config(bids_dir, participant_labels=None, aws_input_creds=None, skip_bids_validator=False, only_one_anat=True)[source]

Create a C-PAC data config YAML file from a BIDS directory.

Parameters:
  • bids_dir (str)

  • participant_labels (list or None)

  • aws_input_creds

  • skip_bids_validator (bool)

  • only_one_anat (bool) – The “anat” key for a subject expects a string value, but we can temporarily store a list instead by passing True here if we will be filtering that list down to a single string later

Return type:

list

CPAC.utils.bids_utils.gen_bids_outputs_sublist(base_path, paths_list, key_list, creds_path)[source]
CPAC.utils.bids_utils.insert_entity(resource, key, value)[source]

Insert a f’{key}-{value}’ BIDS entity before desc- if present or before the suffix otherwise.

Parameters:
Return type:

str

Examples

>>> insert_entity('run-1_desc-preproc_bold', 'reg', 'default')
'run-1_reg-default_desc-preproc_bold'
>>> insert_entity('run-1_bold', 'reg', 'default')
'run-1_reg-default_bold'
>>> insert_entity('run-1_desc-preproc_bold', 'filt', 'notch4c0p31bw0p12')
'run-1_filt-notch4c0p31bw0p12_desc-preproc_bold'
>>> insert_entity('run-1_reg-default_bold', 'filt', 'notch4c0p31bw0p12')
'run-1_reg-default_filt-notch4c0p31bw0p12_bold'
CPAC.utils.bids_utils.load_cpac_data_config(data_config_file, participant_labels, aws_input_creds)[source]

Loads the file as a check to make sure it is available and readable.

Parameters:
  • data_config_file (str) – path to data config

  • participants_labels (list or None)

  • aws_input_creds

Return type:

list

CPAC.utils.bids_utils.load_yaml_config(config_filename, aws_input_creds)[source]
CPAC.utils.bids_utils.res_in_filename(cfg, label)[source]

Specify resolution in filename.

Parameters:
  • cfg (CPAC.utils.configuration.Configuration)

  • label (str)

Returns:

label

Return type:

str

Examples

>>> from CPAC.utils.configuration import Configuration
>>> res_in_filename(Configuration({
...     'registration_workflows': {
...         'anatomical_registration': {'resolution_for_anat': '2x2x2'}}}),
...     'sub-1_res-anat_bold')
'sub-1_res-2x2x2_bold'
>>> res_in_filename(Configuration({
...     'registration_workflows': {
...         'anatomical_registration': {'resolution_for_anat': '2x2x2'}}}),
...     'sub-1_res-3mm_bold')
'sub-1_res-3mm_bold'
CPAC.utils.bids_utils.sub_list_filter_by_labels(sub_list, labels)[source]

Function to filter a sub_list by provided BIDS labels for specified suffixes.

Parameters:
  • sub_list (list)

  • labels (dict)

  • labels['T1w'] (str or None) – C-PAC currently only uses a single T1w image

  • labels['bold'] (str, list, or None)

Return type:

list

CPAC.utils.bids_utils.with_key(entity: str, key: str) str[source]

Return a keyed BIDS entity.

Parameters:
Return type:

str

Examples

>>> with_key('sub-1', 'sub')
'sub-1'
>>> with_key('1', 'sub')
'sub-1'
CPAC.utils.bids_utils.without_key(entity: str, key: str) str[source]

Return a BIDS entity value.

Parameters:
Return type:

str

Examples

>>> without_key('sub-1', 'sub')
'1'
>>> without_key('1', 'sub')
'1'

BIDS Data Configuration

Configuration

C-PAC Configuration class and related functions.

class CPAC.utils.configuration.configuration.Configuration(config_map: dict | None = None, skip_env_check: bool = False)[source]

Class to set dictionary keys as map attributes.

If the given dictionary includes the key FROM, that key’s value will form the base of the Configuration object with the values in the given dictionary overriding matching keys in the base at any depth. If no FROM key is included, the base Configuration is the blank preconfiguration.

Changed in version 1.8.5: From version 1.8.0 to version 1.8.5, unspecified keys were based on the default configuration rather than the blank preconfiguration.

FROM accepts either the name of a preconfigured pipleine or a path to a YAML file.

Given a Configuration c, and a list or tuple of an attribute name and nested keys keys = ['attribute', 'key0', 'key1'] or keys = ('attribute', 'key0', 'key1'), the value ‘value’ nested in

c.attribute = {'key0': {'key1': 'value'}}

can be accessed (get and set) in any of the following ways (and more):

c.attribute['key0']['key1']
c['attribute']['key0']['key1']
c['attribute', 'key0', 'key1']
c[keys]

Examples

>>> c = Configuration({})
>>> c['pipeline_setup', 'pipeline_name']
'cpac-blank-template'
>>> c = Configuration({'pipeline_setup': {
...     'pipeline_name': 'example_pipeline'}})
>>> c['pipeline_setup', 'pipeline_name']
'example_pipeline'
>>> c['pipeline_setup', 'pipeline_name'] = 'new_pipeline2'
>>> c['pipeline_setup', 'pipeline_name']
'new_pipeline2'
>>> from CPAC.utils.tests.configs import SLACK_420349

# test “FROM: /path/to/file” >>> slack_420349_filepath = Configuration( … yaml.safe_load(SLACK_420349[‘filepath’])) >>> slack_420349_filepath[‘pipeline_setup’, ‘pipeline_name’] ‘slack_420349_filepath’

# test “FROM: preconfig” >>> slack_420349_preconfig = Configuration( … yaml.safe_load(SLACK_420349[‘preconfig’])) >>> slack_420349_preconfig[‘pipeline_setup’, ‘pipeline_name’] ‘slack_420349_preconfig’

check_pattern(orig_key, tags=None)[source]
dict()[source]

Show contents of a C-PAC configuration as a dict.

get_nested(_d, keys)[source]
key_type_error(key)[source]

Raise a KeyError if an inappropriate type of key is attempted.

keys()[source]

Show toplevel keys of a C-PAC configuration dict.

set_from_ENV(conf)[source]

Replace strings like $VAR and ${VAR} with environment variable values.

Parameters:

conf (any)

Returns:

conf

Return type:

any

Examples

>>> import os
>>> os.environ['SAMPLE_VALUE_SFE'] = '/example/path'
>>> c = Configuration()
>>> c.set_from_ENV({'key': {'nested_list': [
...     1, '1', '$SAMPLE_VALUE_SFE/extended']}})
{'key': {'nested_list': [1, '1', '/example/path/extended']}}
>>> c.set_from_ENV(['${SAMPLE_VALUE_SFE}', 'SAMPLE_VALUE_SFE'])
['/example/path', 'SAMPLE_VALUE_SFE']
>>> del os.environ['SAMPLE_VALUE_SFE']
set_nested(d, keys, value)[source]
set_without_ENV(conf)[source]

Retain strings like $VAR and ${VAR} when setting attributes.

Parameters:

conf (any)

Returns:

conf

Return type:

any

Examples

>>> import os
>>> os.environ['SAMPLE_VALUE_SFE'] = '/example/path'
>>> c = Configuration()
>>> c.set_without_ENV({'key': {'nested_list': [
...     1, '1', '$SAMPLE_VALUE_SFE/extended']}})
{'key': {'nested_list': [1, '1', '$SAMPLE_VALUE_SFE/extended']}}
>>> c.set_without_ENV(['${SAMPLE_VALUE_SFE}', 'SAMPLE_VALUE_SFE'])
['${SAMPLE_VALUE_SFE}', 'SAMPLE_VALUE_SFE']
>>> del os.environ['SAMPLE_VALUE_SFE']
sub_pattern(pattern, orig_key)[source]
switch_is_off(key: str | list[str], exclusive: bool = False) bool[source]

Return True if the key is set to ‘off’ OR ‘on’ and ‘off’ or False otherwise.

Used for tracking forking.

Parameters:
  • key (str or list of str) – key to check

  • exclusive (bool, optional, default: False) – return False if the key is set to ‘on’ and ‘off’

Returns:

True if key is set to ‘off’, False if not set to ‘off’. If exclusive is set to True, return False if the key is set to ‘on’ and ‘off’.

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'])
True
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'])
True
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'], exclusive=True)
False
switch_is_on(key: str | list[str], exclusive: bool = False) bool[source]

Return True if the key is set to ‘on’ OR ‘on’ and ‘off’ or False otherwise.

Used for tracking forking.

Parameters:
  • key (str or list of str) – key to check

  • exclusive (bool, optional, default: False) – return False if the key is set to ‘on’ and ‘off’

Returns:

True if key is set to ‘on’, False if not set to ‘on’. If exclusive is set to True, return False if the key is set to ‘on’ and ‘off’.

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'])
False
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'])
True
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'], exclusive=True)
False
switch_is_on_off(key: str | list[str]) bool[source]

Return True if the key is set to both ‘on’ and ‘off’ or False otherwise.

Used for tracking forking.

Parameters:

key (str or list of str) – key to check

Returns:

True if key is set to ‘on’ and ‘off’, False otherwise

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_on_off(['nuisance_corrections',
...                     '2-nuisance_regression', 'run'])
False
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_on_off(['nuisance_corrections',
...                     '2-nuisance_regression', 'run'])
True
update(key, val=ConfigurationDictUpdateConflation())[source]
exception CPAC.utils.configuration.configuration.ConfigurationDictUpdateConflation[source]

Custom exception to clarify similar methods.

class CPAC.utils.configuration.configuration.Preconfiguration(preconfig, skip_env_check=False)[source]

A preconfigured Configuration.

Parameters:

preconfig (str) – The canonical name of the preconfig to load

CPAC.utils.configuration.configuration.check_pname(p_name: str, pipe_config: Configuration) str[source]

Check / set p_name, the str representation of a pipeline for use in filetrees.

Parameters:
Return type:

p_name

Examples

>>> c = Configuration()
>>> check_pname(None, c)
'pipeline_cpac-blank-template'
>>> check_pname('cpac-default-pipeline', c)
'pipeline_cpac-default-pipeline'
>>> check_pname('pipeline_cpac-default-pipeline', c)
'pipeline_cpac-default-pipeline'
>>> check_pname('different-name', Configuration())
'pipeline_different-name'
>>> p_name = check_pname(None, Preconfiguration('blank'))
>>> p_name
'pipeline_cpac-blank-template'
>>> p_name = check_pname(None, Preconfiguration('default'))
>>> p_name
'pipeline_cpac-default-pipeline'
CPAC.utils.configuration.configuration.collect_key_list(config_dict)[source]

Return a list of lists of keys for a nested dictionary.

Parameters:

config_dict (dict)

Returns:

key_list

Return type:

list

Examples

>>> collect_key_list({'test': {'nested': 1, 'dict': 2}})
[['test', 'nested'], ['test', 'dict']]
CPAC.utils.configuration.configuration.configuration_from_file(config_file)[source]

Load a Configuration from a pipeline config file.

Parameters:

config_file (str) – path to configuration file

Return type:

Configuration

CPAC.utils.configuration.configuration.nipype_friendly_name(name: str) str[source]

Replace each sequence of non-alphanumeric characters…

…with a single underscore and remove any leading underscores.

Parameters:

name (str)

Return type:

str

CPAC.utils.configuration.configuration.preconfig_yaml(preconfig_name='default', load=False)[source]

Get the path to a preconfigured pipeline’s YAML file.

Raises BadParameter if an invalid preconfig name is given.

Parameters:
  • preconfig_name (str)

  • load (boolean) – return dict if True, str if False

Returns:

path to YAML file or dict loaded from YAML

Return type:

str or dict

CPAC.utils.configuration.configuration.set_subject(sub_dict: dict, pipe_config: Configuration, p_name: str | None = None) tuple[str, str, str][source]

Set pipeline name and log directory path for a given sub_dict.

Parameters:
  • sub_dict (dict)

  • pipe_config (CPAC.utils.configuration.Configuration)

  • p_name (str, optional) – pipeline name string

Returns:

  • subject_id (str)

  • p_name (str) – pipeline name string

  • log_dir (str) – path to subject log directory

Examples

>>> from tempfile import TemporaryDirectory
>>> from CPAC.utils.configuration import Configuration
>>> sub_dict = {'site_id': 'site1', 'subject_id': 'sub1',
...             'unique_id': 'uid1'}
>>> with TemporaryDirectory() as tmpdir:
...     subject_id, p_name, log_dir = set_subject(
...         sub_dict, Configuration({'pipeline_setup': {'log_directory':
...             {'path': tmpdir}}}))
>>> subject_id
'sub1_uid1'
>>> p_name
'pipeline_cpac-blank-template'
>>> log_dir.endswith(f'{p_name}/{subject_id}')
True

Tools for configuration differences.

class CPAC.utils.configuration.diff.DiffDict(*args, **kwargs)[source]

Class to semantically store a dictionary of set differences from Configuration(S) - Configuration(T).

left

dictionary of differing values from Configuration(S) (alias for s_value)

Type:

dict

minuend

dictionary of differing values from Configuration(S) (alias for s_value)

Type:

dict

right

dictionary of differing values from Configuration(T) (alias for t_value)

Type:

dict

subtrahend

dictionary of differing values from Configuration(T) (alias for t_value)

Type:

dict

s_value

dictionary of differing values from Configuration(S)

Type:

dict

t_value

dictionary of differing values from Configuration(T)

Type:

dict

class CPAC.utils.configuration.diff.DiffValue(s_value, t_value)[source]

Class to semantically store values of set difference from Configuration(S) - Configuration(T).

left

value from Configuration(S) (alias for s_value)

Type:

any

minuend

value from Configuration(S) (alias for s_value)

Type:

dict

right

value from Configuration(T) (alias for t_value)

Type:

dict

subtrahend

value from Configuration(T) (alias for t_value)

Type:

dict

s_value

value from Configuration(S)

Type:

any

t_value

value from Configuration(T)

Type:

any

CPAC.utils.configuration.diff.dct_diff(dct1, dct2)[source]

Function to compare 2 nested dicts, dropping values unspecified in the second. Adapted from https://github.com/FCP-INDI/CPAC_regtest_pack/blob/9056ef63/cpac_pipe_diff.py#L31-L78.

Parameters:
  • dct1 (dict or Configuration)

  • dct2 (dict or Configuration)

Returns:

diff – each value is a ~DiffValue of values from dct1, dct2 for each differing key between original dicts or a subdictionary thereof

Return type:

dict

Example

>>> import yaml
>>> from . import Preconfiguration, preconfig_yaml
>>> def read_yaml_file(yaml_file):
...     return yaml.safe_load(open(yaml_file, 'r'))
>>> pipeline = read_yaml_file(preconfig_yaml('default'))
>>> dct_diff(pipeline, pipeline)
{}
>>> pipeline2 = Preconfiguration('fmriprep-options')
>>> dct_diff(pipeline, pipeline2)['pipeline_setup']['pipeline_name']
('cpac-default-pipeline', 'cpac_fmriprep-options')
CPAC.utils.configuration.diff.diff_dict(diff)[source]

Method to return a dict of only changes given a nested dict of (dict1_value, dict2_value) tuples.

Parameters:

diff (dict) – output of dct_diff

Returns:

dict of only changed values

Return type:

dict

Examples

>>> diff_dict({'anatomical_preproc': {
...     'brain_extraction': {'extraction': {
...         'run': DiffValue([True], False),
...         'using': DiffValue(['3dSkullStrip'],
...                            ['niworkflows-ants'])}}}})
{'anatomical_preproc': {'brain_extraction': {'extraction': {'run': False, 'using': ['niworkflows-ants']}}}}

Functions to create YAML configuration files from templates.

class CPAC.utils.configuration.yaml_template.YamlTemplate(original_yaml, base_config=None)[source]

A class to link YAML comments to the contents of a YAML file.

comments

Flat dictionary with '.'-delimited pseudo-nested structure. E.g., comments for {'pipeline_setup': {'pipeline_name': value}} would be keyed {'pipeline_setup': comment0, 'pipeline_setup.pipeline_name: comment1} to allow comments at each level of depth.

Type:

dict

dump[source]
Type:

method

get_nested
Type:

method

original
Type:

str

dump(new_dict, parents=None)[source]

Dump YAML from a new dictionary with comments from template dictionary.

Parameters:
Return type:

str

get_nested(_d, keys)
CPAC.utils.configuration.yaml_template.create_yaml_from_template(d: Configuration | dict, template: str = 'default', import_from: str | None = None, skip_env_check: bool | None = False) str[source]

Save dictionary to a YAML file, keeping the structure from the template.

For example, first level comments and ordering.

It may not be fully robust to YAML structures, but it works for C-PAC config files!

Parameters:
  • d (dict or Configuration)

  • template (str) – path to template, name of preconfig, or YAML as a string

  • import_from (str, optional) – name of a preconfig. Full config is generated if omitted

  • skip_env_check (bool, optional) – skip environment check (for validating a config without running)

Examples

>>> import yaml
>>> from CPAC.utils.configuration import Configuration, Preconfiguration
>>> Configuration(yaml.safe_load(create_yaml_from_template({}))).dict(
...     ) == Configuration({}).dict()
True
>>> fmriprep_options = Preconfiguration('fmriprep-options')
>>> fmriprep_options - Configuration({}) != {}
True
>>> fmriprep_options - fmriprep_options
{}
>>> fmriprep_options - Preconfiguration('fmriprep-options')
{}
>>> fmriprep_options - Configuration({'FROM': 'fmriprep-options'})
{}
>>> fmriprep_options - Configuration(yaml.safe_load(
...     create_yaml_from_template(fmriprep_options, import_from=None)))
{}
>>> fmriprep_options - Configuration(yaml.safe_load(
...     create_yaml_from_template(fmriprep_options,
...                               import_from='default')))
{}
>>> fmriprep_options - Configuration(yaml.safe_load(
...     create_yaml_from_template(fmriprep_options, import_from='blank')))
{}
>>> different_sca = Configuration({'pipeline_setup': {
...     'pipeline_name': 'different_SCA'},
...     'seed_based_correlation_analysis': {'run': 'y',
...     'norm_timeseries_for_DR': 'Off'}})
>>> (Configuration(yaml.safe_load(create_yaml_from_template(
...     different_sca))) - Configuration()).get(
...     'seed_based_correlation_analysis') not in (None, {})
True
CPAC.utils.configuration.yaml_template.hash_data_config(sub_list)[source]

Generate a short SHA1 hash from a data config subject list of dicts.

Parameters:

sub_list (list of dicts)

Returns:

data_config_hash

Return type:

str, len(8)

Examples

>>> sub_list = [{'site_id': f'site{i}', 'subject_id': f'sub{i}',
...              'unique_id': f'uid{i}'} for i in range(1, 4)]
>>> sub_list[0]
{'site_id': 'site1', 'subject_id': 'sub1', 'unique_id': 'uid1'}
>>> hash_data_config(sub_list)
'6f49a278'
CPAC.utils.configuration.yaml_template.indent(line_level, plus=2)[source]

Return an indent string for a given level.

Parameters:

line_level (int) – The level of indentation to return

Returns:

The string of spaces to use for indentation

Return type:

str

CPAC.utils.configuration.yaml_template.update_a_preconfig(preconfig, import_from)[source]

Update a preconfig with comments from another config.

Parameters:
  • preconfig (str)

  • import_from (str)

CPAC.utils.configuration.yaml_template.update_all_preconfigs()[source]

Update all other preconfigs with comments from default.

CPAC.utils.configuration.yaml_template.upgrade_pipeline_to_1_8(path)[source]

Upgrade a C-PAC 1.7 pipeline config to C-PAC 1.8.

Parameters:

path (str)

Returns:

  • None

  • Outputs

  • -------

  • {path}```{now}`bak – original file

  • path – upgraded file

CPAC.utils.configuration.yaml_template.yaml_bool(value)[source]

Give On/Off value to bools.

Parameters:

value (any)

Returns:

value

Return type:

any

Examples

>>> yaml_bool(True)
'On'
>>> yaml_bool([False, 'On', True])
['Off', 'On', 'On']

Create FMRIB’s Local Analysis of Mixed Effects (FLAME) Model Files

CPAC.utils.create_flame_model_files.create_con_file(con_vecs, con_names, col_names, model_name, current_output, out_dir)[source]
CPAC.utils.create_flame_model_files.create_con_ftst_file(con_file, model_name, current_output, output_dir, column_names, coding_scheme, group_sep)[source]

Create the contrasts and fts file.

CPAC.utils.create_flame_model_files.create_dummy_string(length)[source]
CPAC.utils.create_flame_model_files.create_flame_model_files(design_matrix, col_names, contrasts_vectors, contrast_names, custom_contrasts_csv, ftest_list, group_sep, grouping_vector, coding_scheme, model_name, output_measure, output_dir)[source]
CPAC.utils.create_flame_model_files.create_fts_file(ftest_list, con_names, model_name, current_output, out_dir)[source]
CPAC.utils.create_flame_model_files.create_grp_file(design_matrix, grp_file_vector, output_dir, model_name)[source]
CPAC.utils.create_flame_model_files.write_mat_file(design_matrix, output_dir, model_name, depatsified_EV_names, current_output=None)[source]

Create FSL FLAME Preset

CPAC.utils.create_fsl_flame_preset.create_contrasts_template_df(design_df, contrasts_dct_list=None)[source]

Create the template Pandas DataFrame for the contrasts matrix CSV.

The headers in the contrasts matrix needs to match the headers of the design matrix.

CPAC.utils.create_fsl_flame_preset.create_design_matrix_df(group_list, pheno_df=None, ev_selections=None, pheno_sub_label=None, pheno_ses_label=None, pheno_site_label=None, ses_id=False)[source]

Create the design matrix intended for group-level analysis via the FSL FLAME tool.

This does NOT create the final .mat file that FSL FLAME takes in. This is an intermediary design matrix CSV meant for the user to review.

If there is a phenotype CSV provided, this function will align the participant-session ID labels in the CPAC individual-level analysis output directory with the values listed in the phenotype file.

CPAC.utils.create_fsl_flame_preset.preset_paired_two_group(group_list, conditions, condition_type='session', output_dir=None, model_name='two_sample_unpaired_T-test')[source]

Set up the design matrix and contrasts matrix for running an paired two-group difference (two-sample paired T-test).

group_list: a list of strings- sub_ses unique IDs conditions: a two-item list of strings- session or series/scan names of

the two sessions or two scans (per participant) you wish to compare

condition_type: a string, either “session” or “scan”, depending on what

is in “conditions”

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:
https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

#Paired_Two-Group_Difference_.28Two-Sample_Paired_T-Test.29

CPAC.utils.create_fsl_flame_preset.preset_single_group_avg(group_list, pheno_df=None, covariate=None, pheno_sub_label=None, output_dir=None, model_name='one_sample_T-test')[source]

Set up the design matrix CSV for running a single group average (one-sample T-test).

CPAC.utils.create_fsl_flame_preset.preset_tripled_two_group(group_list, conditions, condition_type='Sessions', output_dir=None, model_name='tripled_T-test')[source]

Set up the design matrix and contrasts matrix for running a tripled two-group difference (‘tripled’ T-test).

group_list: a list of strings- sub_ses unique IDs conditions: a three-item list of strings- session or series/scan names of

the three sessions or three scans (per participant) you wish to compare

condition_type: a string, either “session” or “scan”, depending on what

is in “conditions”

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:
https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

#Tripled_Two-Group_Difference_.28.22Tripled.22_T-Test.29

CPAC.utils.create_fsl_flame_preset.preset_unpaired_two_group(group_list, pheno_df, groups, pheno_sub_label, output_dir=None, model_name='two_sample_unpaired_T-test')[source]

Set up the design matrix and contrasts matrix for running an unpaired two-group difference (two-sample unpaired T-test).

group_list: a list of strings- sub_ses unique IDs pheno_df: a Pandas DataFrame object of the phenotypic file CSV/matrix groups: a list of either one or two strings- design matrix EV/covariate

labels to take from the phenotype DF and include in the model

pheno_sub_label: a string of the label name of the column in the phenotype

file that holds the participant/session ID for each row

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:
https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

#Unpaired_Two-Group_Difference_.28Two-Sample_Unpaired_T-Test.29

Only one “group” will be provided usually if the two groups in the phenotypic information you wish to compare are encoded in one covariate column, as categorical information. Thus, providing this one name will pull it from the phenotype file, and this function will break it out into two columns using dummy-coding.

CPAC.utils.create_fsl_flame_preset.read_group_list_text_file(group_list_text_file)[source]

Read in the group-level analysis participant-session list text file.

CPAC.utils.create_fsl_flame_preset.read_pheno_csv_into_df(pheno_csv, id_label=None)[source]

Read the phenotypic file CSV or TSV into a Pandas DataFrame.

CPAC.utils.create_fsl_flame_preset.run(pipeline_dir, derivative_list, z_thresh, p_thresh, preset=None, group_list_text_file=None, pheno_file=None, pheno_sub_label=None, output_dir=None, model_name=None, covariate=None, condition_type=None, run=False)[source]
CPAC.utils.create_fsl_flame_preset.write_config_dct_to_yaml(config_dct, out_file=None)[source]

Write out a configuration dictionary into a YAML file.

CPAC.utils.create_fsl_flame_preset.write_dataframe_to_csv(matrix_df, out_file=None)[source]

Write out a matrix Pandas DataFrame into a CSV file.

CPAC.utils.create_fsl_flame_preset.write_group_list_text_file(group_list, out_file=None)[source]

Write out the group-level analysis participant list as a text file.

Create FSL create_fsl_model

Create Group Analysis Info Files

CPAC.utils.create_group_analysis_info_files.write_blank_contrast_csv(contrasts_columns, contrast_out_path)[source]
CPAC.utils.create_group_analysis_info_files.write_custom_readme_file()[source]
CPAC.utils.create_group_analysis_info_files.write_design_matrix_csv(patsy_dmatrix, participant_column, column_names, outfile_path)[source]

Datasource

Utilities for sourcing data.

CPAC.utils.datasource.bidsier_prefix(unique_id)[source]

Return a BIDSier prefix for a given unique_id.

Parameters:

unique_id (str)

Returns:

prefix

Return type:

str

Examples

>>> bidsier_prefix('01_1')
'sub-01_ses-1'
>>> bidsier_prefix('sub-01_ses-1')
'sub-01_ses-1'
>>> bidsier_prefix('sub-01_1')
'sub-01_ses-1'
>>> bidsier_prefix('01_ses-1')
'sub-01_ses-1'
CPAC.utils.datasource.calc_delta_te_and_asym_ratio(effective_echo_spacing: float, echo_times: list) tuple[float, float][source]

Calcluate deltaTE and ees_asym_ratio from given metadata.

Parameters:
  • effective_echo_spacing (float) – EffectiveEchoSpacing from sidecar JSON

  • echo_times (list)

Returns:

CPAC.utils.datasource.check_for_s3(file_path, creds_path=None, dl_dir=None, img_type='other', verbose=False)[source]

Check if passed-in file is on S3.

CPAC.utils.datasource.check_func_scan(func_scan_dct, scan)[source]

Run some checks on the functional timeseries-related files.

For a given series/scan name or label.

CPAC.utils.datasource.create_anat_datasource(wf_name='anat_datasource')[source]

Create a dataflow for anatomical images.

CPAC.utils.datasource.create_check_for_s3_node(name, file_path, img_type='other', creds_path=None, dl_dir=None, map_node=False)[source]

Create a node to check if a file is on S3.

CPAC.utils.datasource.create_fmap_datasource(fmap_dct, wf_name='fmap_datasource')[source]

Return the field map files…

…from the dictionary of functional files described in the data configuration (sublist) YAML file.

CPAC.utils.datasource.create_func_datasource(rest_dict, rpool, wf_name='func_datasource')[source]

Return the functional timeseries-related file paths for each series/scan…

…from the dictionary of functional files described in the data configuration (sublist) YAML file.

Scan input (from inputnode) is an iterable.

CPAC.utils.datasource.create_general_datasource(wf_name)[source]

Create a general-purpose datasource node.

CPAC.utils.datasource.create_grp_analysis_dataflow(wf_name='gp_dataflow')[source]

Create a dataflow for group analysis.

CPAC.utils.datasource.create_roi_mask_dataflow(masks, wf_name='datasource_roi_mask')[source]

Create a dataflow for ROI masks.

CPAC.utils.datasource.create_spatial_map_dataflow(spatial_maps, wf_name='datasource_maps')[source]

Create a dataflow for spatial maps.

CPAC.utils.datasource.extract_scan_params_dct(scan_params_dct)[source]

Extract the scan parameters dictionary from the data configuration file.

CPAC.utils.datasource.gather_echo_times(echotime_1, echotime_2, echotime_3=None, echotime_4=None)[source]

Gather the echo times from the field map data.

CPAC.utils.datasource.gather_extraction_maps(c)[source]

Gather the timeseries and SCA analysis configurations.

CPAC.utils.datasource.get_fmap_phasediff_metadata(data_config_scan_params)[source]

Return the scan parameters for a field map phasediff scan.

CPAC.utils.datasource.get_highest_local_res(template: Path | str, tagname: str) Path[source]

Return the highest resolution of a template in the same local path.

Given a reference template path and a resolution string, get all resolutions of that template in the same local path and return the highest resolution.

Parameters:
  • template (Path or str)

  • tagname (str)

Return type:

str

Raises:

LookupError – If no matching local template is found.

Examples

>>> get_highest_local_res(
...     '/cpac_templates/MacaqueYerkes19_T1w_2mm_brain.nii.gz', '2mm')
PosixPath('/cpac_templates/MacaqueYerkes19_T1w_0.5mm_brain.nii.gz')
>>> get_highest_local_res(
...     '/cpac_templates/dne_T1w_2mm.nii.gz', '2mm')
Traceback (most recent call last):
   ...
LookupError: Could not find template /cpac_templates/dne_T1w_2mm.nii.gz
CPAC.utils.datasource.get_rest(scan, rest_dict, resource='scan')[source]

Return the path of the chosen resource in the functional file dictionary.

scan: the scan/series name or label rest_dict: the dictionary read in from the data configuration YAML file

(sublist) nested under ‘func:’

resource: the dictionary key

scan - the functional timeseries scan_parameters - path to the scan parameters JSON file, or

a dictionary containing scan parameters information (to be phased out in the future)

CPAC.utils.datasource.ingress_func_metadata(wf, cfg, rpool, sub_dict, subject_id, input_creds_path, unique_id=None, num_strat=None)[source]

Ingress metadata for functional scans.

CPAC.utils.datasource.match_epi_fmaps(bold_pedir, epi_fmap_one, epi_fmap_params_one, epi_fmap_two=None, epi_fmap_params_two=None)[source]

Match EPI field maps to the BOLD scan.

Parse the field map files in the data configuration and determine which ones have the same and opposite phase-encoding directions as the BOLD scan in the current pipeline.

Example - parse the files under the ‘fmap’ level, i.e. ‘epi_AP’:

anat: /path/to/T1w.nii.gz fmap:

epi_AP:

scan: /path/to/field-map.nii.gz scan_parameters: <config dictionary containing phase-encoding

direction>

func:
rest_1:

scan: /path/to/bold.nii.gz scan_parameters: <config dictionary of BOLD scan parameters>

  1. Check PhaseEncodingDirection field in the metadata for the BOLD.

  2. Check whether there are one or two EPI’s in the field map data.

  3. Grab the one or two EPI field maps.

CPAC.utils.datasource.res_string_to_tuple(resolution)[source]

Convert a resolution string to a tuple of floats.

Parameters:

resolution (str) – Resolution string, e.g. “3.438mmx3.438mmx3.4mm”

Returns:

resolution – Tuple of floats, e.g. (3.438, 3.438, 3.4)

Return type:

tuple

CPAC.utils.datasource.resample_func_roi(in_func, in_roi, realignment, identity_matrix)[source]

Resample functional image to ROI or ROI to functional image using flirt.

CPAC.utils.datasource.resolve_resolution(resolution, template, template_name, tag=None)[source]

Resample a template to a given resolution.

CPAC.utils.datasource.select_model_files(model, ftest, model_name)[source]

Select model files.

Datatypes

Custom datatypes for C-PAC.

class CPAC.utils.datatypes.ItemFromList(list_of_one, msg=None)[source]

Coerce single-item lists into just the only item in the list. Returns item if item is not a list, set, or tuple. Raises CoerceInvalid if impossible.

Examples

>>> ItemFromList(['seagull'])
'seagull'
>>> ItemFromList(['two', 'seagulls'])
Traceback (most recent call last):
    ...
voluptuous.error.CoerceInvalid: Cannot coerce list of length 2 to item
>>> ItemFromList('string')
'string'
class CPAC.utils.datatypes.ListFromItem(*args, **kwargs)[source]

Subclass of list to coerce non-lists into lists.

Examples

>>> list('one')
['o', 'n', 'e']
>>> ListFromItem('one')
['one']
>>> list(['one'])
['one']
>>> ListFromItem(['one'])
['one']
>>> list()
[]
>>> ListFromItem()
[]
>>> list(None)
Traceback (most recent call last):
...
TypeError: 'NoneType' object is not iterable
>>> ListFromItem(None)
[]

Documentation

Utilties for C-PAC documentation.

CPAC.utils.docs.deprecated(version: str | None = None, explanation: str | None = None) Callable[source]

Mark a function as deprecated.

Parameters:
  • version (str, optional) – The version in which the function was deprecated.

  • explanation (str, optional) – An explanation of why the function was deprecated.

Returns:

The decorated function.

Return type:

Callable

CPAC.utils.docs.docstring_parameter(*args, **kwargs) Callable[source]

Parameterize docstrings.

Use double-curly-braces ({{}}) for literal curly braces.

Examples

>>> @docstring_parameter('test', answer='Yes it does.')
... def do_nothing():
...     '''Does this {} do anything? {answer}'''
...     pass
>>> print(do_nothing.__doc__)
Does this test do anything? Yes it does.
>>> @docstring_parameter('test', answer='It should not.')
... def how_about_now():
...     '''How about {{ this }}?'''
...     pass
>>> print(how_about_now.__doc__)
How about { this }?
CPAC.utils.docs.outdent_lines(docstring: str, spaces: int = 4) str[source]

Outdent lines in a string by specified number of spaces.

Only outdents lines that are at least that indented. Useful for combining docstrings.

Examples

>>> import re
>>> re.findall(r'^    Only.*$', outdent_lines.__doc__, flags=re.MULTILINE)
['    Only outdents lines that are at least that indented.']
>>> re.findall(r'^Only.*$', outdent_lines.__doc__, flags=re.MULTILINE)
[]
>>> re.findall(r'^    Only.*$', outdent_lines(outdent_lines.__doc__),
...     flags=re.MULTILINE)
[]
>>> re.findall(r'^Only.*$', outdent_lines(outdent_lines.__doc__),
...     flags=re.MULTILINE)
['Only outdents lines that are at least that indented.']
>>> re.findall(r'^ Only.*$', outdent_lines(outdent_lines.__doc__, 3),
...     flags=re.MULTILINE)
[' Only outdents lines that are at least that indented.']
CPAC.utils.docs.version_report() str[source]

Return a formatted block of versions included in CPAC’s environment.

Extract Data

class CPAC.utils.extract_data.Configuration(config_map)[source]

Set dictionary keys as map attributes.

CPAC.utils.extract_data.extract_data(c, param_map)[source]

Generate a CPAC input subject list Python file.

The method extracts anatomical and functional data for each site (if multiple site) and/or scan parameters for each site and put it into a data structure read by Python.

Examples

subjects_list =[
{

‘subject_id’ : ‘0050386’, ‘unique_id’ : ‘session_1’, ‘anat’: ‘/Users/home/data/NYU/0050386/session_1/anat_1/anat.nii.gz’, ‘rest’:{

‘rest_1_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_1/rest.nii.gz’, ‘rest_2_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_2/rest.nii.gz’, }

‘scan_parameters’:{

‘tr’: ‘2’, ‘acquisition’: ‘alt+z2’, ‘reference’: ‘17’, ‘first_tr’: ‘’, ‘last_tr’: ‘’, }

},

]

or

subjects_list =[
{

‘subject_id’ : ‘0050386’, ‘unique_id’ : ‘session_1’, ‘anat’: ‘/Users/home/data/NYU/0050386/session_1/anat_1/anat.nii.gz’, ‘rest’:{

‘rest_1_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_1/rest.nii.gz’, ‘rest_2_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_2/rest.nii.gz’, }

},

]

CPAC.utils.extract_data.generate_supplementary_files(data_config_outdir, data_config_name)[source]

Generate phenotypic template file and subject list for group analysis.

CPAC.utils.extract_data.read_csv(csv_input)[source]

Read CSV file.

‘Acquisition’ ‘Reference’ ‘Site’ ‘TR (seconds)’

CPAC.utils.extract_data.run(data_config: Path | str) None[source]

Run a data config.

Parameters:

data_config (Path or str) – path to data_config file

Extract Data (Multiscan)

class CPAC.utils.extract_data_multiscan.Configuration(config_map)[source]
CPAC.utils.extract_data_multiscan.extract_data(c, param_map)[source]

Method to generate a CPAC input subject list python file. The method extracts anatomical functional data and scan parameters for each site( if multiple site) and for each scan and put it into a data structure read by python.

Note:

Use this tool only if the scan parameters are different for each scan as shown in the example below.

Example:

subjects_list = [
{

‘subject_id’: ‘0021001’, ‘unique_id’: ‘session2’, ‘anat’: ‘/home/data/multiband_data/NKITRT/0021001/anat/mprage.nii.gz’, ‘rest’:{

‘RfMRI_mx_1400_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_mx_1400/rest.nii.gz’, ‘RfMRI_mx_645_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_mx_645/rest.nii.gz’, ‘RfMRI_std_2500_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_std_2500/rest.nii.gz’, },

‘scan_parameters’:{
‘TR’:{

‘RfMRI_mx_1400_rest’: ‘1.4’, ‘RfMRI_mx_645_rest’: ‘1.4’, ‘RfMRI_std_2500_rest’: ‘2.5’, },

‘Acquisition’:{

‘RfMRI_mx_1400_rest’: ‘/home/data/1400.txt’, ‘RfMRI_mx_645_rest’: ‘/home/data/645.txt’, ‘RfMRI_std_2500_rest’: ‘/home/data/2500.txt’, },

‘Reference’:{

‘RfMRI_mx_1400_rest’: ‘32’, ‘RfMRI_mx_645_rest’: ‘20’, ‘RfMRI_std_2500_rest’: ‘19’, },

‘FirstTR’:{

‘RfMRI_mx_1400_rest’: ‘7’, ‘RfMRI_mx_645_rest’: ‘15’, ‘RfMRI_std_2500_rest’: ‘4’, },

‘LastTR’:{

‘RfMRI_mx_1400_rest’: ‘440’, ‘RfMRI_mx_645_rest’: ‘898’, ‘RfMRI_std_2500_rest’: ‘None’, },

}

},

]

CPAC.utils.extract_data_multiscan.generate_suplimentary_files(output_path)[source]

Method to generate phenotypic template file and subject list for group analysis.

CPAC.utils.extract_data_multiscan.read_csv(csv_input)[source]
Method to read csv file

‘Acquisition’ ‘Reference’ ‘Site’ ‘TR (seconds)’.

CPAC.utils.extract_data_multiscan.run(data_config)[source]

Run method takes data_config file as the input argument.

Extract Parameters

CPAC.utils.extract_parameters.grab(output_dir, scrubbing)[source]

Method to grab all the motion parameters and power parameters file from each subject for each pipeline and merge them.

Parameters:

output_dir (string) – Path to the datasink output directory of CPAC

CPAC.utils.extract_parameters.merge(output_dir, scan_name, threshold, motion_f, power_f, flag)[source]

Merge power parameters and motion parameters file.

CPAC.utils.extract_parameters.run(output_path, scrubbing)[source]

Google Analytics

CPAC.utils.ga.do_it(data, timeout)[source]
CPAC.utils.ga.get_or_create_config()[source]
CPAC.utils.ga.get_uid()[source]
CPAC.utils.ga.track_config(cpac_interface)[source]
CPAC.utils.ga.track_event(category, action, uid=None, label=None, value=0, software_version=None, timeout=2, thread=True)[source]

Record an event with Google Analytics.

Parameters:
  • tracking_id (str) – Google Analytics tracking ID.

  • category (str) – Event category.

  • action (str) – Event action.

  • uid (str) – User unique ID, assigned when popylar was installed.

  • label (str) – Event label.

  • value (int) – Event value.

  • software_version (str) – Records a version of the software.

  • timeout (float) – Maximal duration (in seconds) for the network connection to track the event. After this duration has elapsed with no response (e.g., on a slow network connection), the tracking is dropped.

CPAC.utils.ga.track_run(level='participant', participants=0)[source]

Interfaces

Function Interfaces

Function interface utilities for C-PAC.

class CPAC.utils.interfaces.function.Function(input_names=None, output_names='out', function=None, imports=None, as_module=False, **inputs)[source]

Runs arbitrary function as an interface

>>> func = 'def func(arg1, arg2=5): return arg1 + arg2'
>>> fi = Function(input_names=['arg1', 'arg2'], output_names=['out'])
>>> fi.inputs.function_str = func
>>> res = fi.run(arg1=1)
>>> res.outputs.out
6

Can automatically set a module name on the interface.

Automatically imports global Nipype loggers.

static sig_imports(imports: list[str]) Callable[source]

Set an ns_imports attribute on a function for Function-node functions.

This can be useful for classes needed for decorators, typehints and for avoiding redefinitions.

Parameters:

imports (list of str) – import statements to import the function in an otherwise empty namespace. If these collide with imports defined via the Function.__init__ initialization method, the imports given as a parameter here will be overridden by those from the initializer.

Returns:

func

Return type:

function

Examples

See the defintion of calculate_FD_J to see the decorator tested here being applied. >>> from CPAC.generate_motion_statistics import calculate_FD_J >>> calc_fdj = Function(input_names=[‘in_file’, ‘calc_from’, ‘center’], … output_names=[‘out_file’], … function=calculate_FD_J, … as_module=True) >>> calc_fdj.imports == [“from CPAC.utils.interfaces.function import Function”, … *calculate_FD_J.ns_imports, *_AUTOLOGGING_IMPORTS] True >>> from inspect import signature >>> from nipype.utils.functions import (getsource, … create_function_from_source) >>> f = create_function_from_source(getsource(calculate_FD_J), … calc_fdj.imports) >>> inspect.signature(calculate_FD_J) == inspect.signature(f) True

CPAC.utils.interfaces.function.pick_tissue_from_labels_file_interface(input_names=None)[source]

Create a Function interface for ~CPAC.seg_preproc.utils.pick_tissue_from_labels_file.

Parameters:

input_names (list, optional)

Return type:

nipype.interfaces.base.core.Interface

Tests

Miscellaneous

Miscellaneous utilities for Nipype translation of ANTs workflows.

This functionality is adapted from poldracklab/niworkflows: https://github.com/poldracklab/niworkflows/blob/994dd2dc/niworkflows/utils/misc.py https://fmriprep.readthedocs.io/ https://poldracklab.stanford.edu/ We are temporarily maintaining our own copy for more granular control.

CPAC.utils.misc.get_template_specs(in_template, template_spec=None, default_resolution=1)[source]

Parse template specifications.

Monitoring

Customize Nipype’s process monitoring for use in C-PAC.

See https://fcp-indi.github.io/docs/developer/nodes for C-PAC-specific documentation. See https://nipype.readthedocs.io/en/latest/api/generated/nipype.utils.profiler.html for Nipype’s documentation.

class CPAC.utils.monitoring.LoggingHTTPServer(pipeline_name, logging_dir='', host='', port=8080, request=<class 'CPAC.utils.monitoring.monitoring.LoggingRequestHandler'>)[source]
class CPAC.utils.monitoring.LoggingRequestHandler(request, client_address, server)[source]
handle()[source]
CPAC.utils.monitoring.failed_to_start(log_dir, exception)[source]

Launch a failed-to-start logger for a run that failed to start.

Must be called from within an except block.

Parameters:
  • log_dir (str) – path to logging directory

  • exception (Exception)

CPAC.utils.monitoring.getLogger(name)[source]

Get a mock logger if one exists, falling back on real loggers.

Parameters:

name (str)

Returns:

logger

Return type:

CPAC.utils.monitoring.custom_logging.MockLogger or logging.Logger

CPAC.utils.monitoring.log_nodes_cb(node, status)[source]
Function to record node run statistics to a log file as json

dictionaries

nodenipype.pipeline.engine.Node

the node being logged

statusstring

acceptable values are ‘start’, ‘end’; otherwise it is considered and error

None

this function does not return any values, it logs the node status info to the callback logger

Modified from https://github.com/nipy/nipype/blob/5ab2fa0/nipype/utils/profiler.py#L112-L156

CPAC.utils.monitoring.log_nodes_initial(workflow)[source]
CPAC.utils.monitoring.monitor_server(pipeline_name, logging_dir, host='0.0.0.0', port=8080)[source]
CPAC.utils.monitoring.recurse_nodes(workflow, prefix='')[source]
CPAC.utils.monitoring.set_up_logger(name, filename=None, level=None, log_dir=None, mock=False, overwrite_existing=False)[source]

Initialize a logger.

Parameters:
  • name (str) – logger name (for subsequent calls to logging.getLogger) to write to the same log file)

  • filename (str, optional) – filename to write log to. If not specified, filename will be the same as name with the extension log

  • level (str, optional) – one of {critical, error, warning, info, debug, notset}, case-insensitive

  • log_dir (str, optional)

  • mock (bool, optional) – if True, return a CPAC.utils.monitoring.MockLogger instead of a logging.Logger

Returns:

logger – initialized logging Handler

Return type:

logging.Handler

Examples

>>> lg = set_up_logger('test')
>>> MockLogger._get_first_file_handler(lg.handlers).baseFilename.split('/')[-1]
'test.log'
>>> lg.level
0
>>> lg = set_up_logger('second_test', 'specific_filename.custom', 'debug')
>>> MockLogger._get_first_file_handler(lg.handlers).baseFilename.split('/')[-1]
'specific_filename.custom'
>>> lg.level
10
>>> lg = set_up_logger('third_test', mock=True)
>>> getLogger('third_test') == lg
True
>>> 'third_test' in MOCK_LOGGERS
True
>>> lg.delete()
>>> 'third_test' in MOCK_LOGGERS
False

NeuroData’s MRI to Graphs (NDMG | m2g) Utilities

class CPAC.utils.ndmg_utils.graph(N, rois, attr=None, sens='dwi')[source]
as_matrix()[source]

Returns the graph as a matrix.

cor_graph(timeseries, attr=None)[source]

Takes timeseries and produces a correlation matrix Positional Arguments:

timeseries:
-the timeseries file to extract correlation for.

dimensions are [numrois]x[numtimesteps].

get_graph()[source]

Returns the graph object created.

make_graph(streamlines, attr=None)[source]

Takes streamlines and produces a graph Positional Arguments:

streamlines:
  • Fiber streamlines either file or array in a dipy EuDX or compatible format.

save_graph(graphname)[source]

Saves the graph to disk Positional Arguments:

graphname:
  • Filename for the graph.

summary()[source]

User friendly wrapping and display of graph properties.

CPAC.utils.ndmg_utils.ndmg_create_graphs(ts, labels)[source]
CPAC.utils.ndmg_utils.ndmg_roi_timeseries(func_file, label_file)[source]

Function to extract average timeseries for the voxels in each roi of the labelled atlas. Returns the roi timeseries as a numpy.ndarray. Positional Arguments

func_file:
  • the path to the 4d volume to extract timeseries

label_file:
  • the path to the labelled atlas containing labels

    for the voxels in the fmri image

roits_file:
  • the path to where the roi timeseries will be saved. If

None, don’t save and just return the roi_timeseries.

# Adapted from ndmg v0.1.1 # Copyright 2016 NeuroData (http://neurodata.io)

Neuroimaging Informatics Technology Initiative (NIfTI) Utilities

Utlities for NIfTI images.

CPAC.utils.nifti_utils.inverse_nifti_values(image)[source]

Replace zeros by ones and non-zero values by 1.

Parameters:

image (str or nibabel.nifti1.Nifti1Image) – path to the nifti file to be inverted or the image already loaded through nibabel

Returns:

output

Return type:

Nibabel Nifti1Image

CPAC.utils.nifti_utils.more_zeros_than_ones(image)[source]

Return True if there are more zeros than other values in a given nifti image.

Parameters:

image (str or nibabel.nifti1.Nifti1Image) – path to the nifti file to be inverted or the image already loaded through nibabel

Returns:

more_zeros

Return type:

boolean

CPAC.utils.nifti_utils.nifti_image_input(image: str | Nifti1Image) Nifti1Image[source]

Test if an input is a path or a nifti.image and the image loaded through nibabel.

Parameters:

image (str or nibabel.nifti1.Nifti1Image) – path to the nifti file or the image already loaded through nibabel

Returns:

img – load and return the nifti image if image is a path, otherwise simply return image

Return type:

nibabel.nifti1.Nifti1Image

Outputs

class CPAC.utils.outputs.Outputs[source]
all_native_filter = 0       True 1       True 2       True 3       True 4      False        ...   307    False 308    False 309    False 310    False 311    False Name: Space, Length: 312, dtype: bool
all_template_filter = 0      False 1      False 2      False 3      False 4       True        ...   307    False 308    False 309    False 310    False 311    False Name: Space, Length: 312, dtype: bool
anat = ['space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'space-template_desc-brain_mask', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'dseg']
any = ['alff', 'desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-T1w_sbref', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'space-template_desc-brain_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'motion', 'desc-summary_motion', 'dvars', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'space-template_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'space-fsLR_den-32k_bold-dtseries', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'vmhc', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask', 'mdmr', 'desc-zstd-mdmr', 'dseg', 'AtlasSubcortical-s2', 'space-fsLR_den-32k_bold', 'goodvoxels', 'ribbon-only', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-L_space-native_bold', 'hemi-R_space-native_bold', 'space-fsLR_den-32k_wb-spec', 'space-native_wb-spec', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion', 'space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion', 'space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-L_space-fsLR_den-32k_curv', 'hemi-R_space-fsLR_den-32k_curv', 'space-fsLR_den-32k_curv', 'hemi-L_space-fsLR_den-32k_flat', 'hemi-R_space-fsLR_den-32k_flat', 'hemi-L_space-fsLR_den-32k_inflated', 'hemi-R_space-fsLR_den-32k_inflated', 'hemi-L_space-fsLR_den-32k_veryinflated', 'hemi-R_space-fsLR_den-32k_veryinflated', 'hemi-L_space-native_inflated', 'hemi-R_space-native_inflated', 'hemi-L_space-native_veryinflated', 'hemi-R_space-native_veryinflated', 'hemi-L_space-fsLR_den-164k_midthickness', 'hemi-R_space-fsLR_den-164k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-native_midthickness', 'hemi-R_space-native_midthickness', 'hemi-L_space-fsLR_den-32k_pial', 'hemi-R_space-fsLR_den-32k_pial', 'hemi-L_space-native_den-32k_pial', 'hemi-R_space-native_den-32k_pial', 'hemi-L_space-fsLR_den-32k_sphere', 'hemi-R_space-fsLR_den-32k_sphere', 'hemi-L_space-native_desc-MSMSulc_sphere', 'hemi-R_space-native_desc-MSMSulc_sphere', 'hemi-L_space-native_sphere', 'hemi-R_space-native_sphere', 'hemi-L_space-native_desc-reg_sphere', 'hemi-R_space-native_desc-reg_sphere', 'hemi-L_space-native_desc-reg-reg_sphere', 'hemi-R_space-native_desc-reg-reg_sphere', 'hemi-L_space-native_desc-rot_sphere', 'hemi-R_space-native_desc-rot_sphere', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ', 'space-fsLR_den-32k_desc-FS_strainJ', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR', 'space-fsLR_den-32k_desc-FS_strainR', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR', 'space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-L_space-fsLR_den-32k_sulc', 'hemi-R_space-fsLR_den-32k_sulc', 'space-fsLR_den-32k_sulc', 'hemi-L_space-fsLR_den-32k_thickness', 'hemi-R_space-fsLR_den-32k_thickness', 'space-fsLR_den-32k_thickness', 'hemi-L_space-fsLR_den-164k_white', 'hemi-R_space-fsLR_den-164k_white', 'hemi-L_space-fsLR_den-32k_white', 'hemi-R_space-fsLR_den-32k_white', 'hemi-L_space-native_white', 'hemi-R_space-native_white', 'atlas-DesikanKilliany_space-fsLR_den-32k', 'atlas-Destrieux_space-fsLR_den-32k', 'atlas-DesikanKilliany_space-fsLR_den-164k', 'atlas-Destrieux_space-fsLR_den-164k', 'space-fsLR_den-32k_bold_surf_falff', 'space-fsLR_den-32k_bold_surf_alff', 'space-fsLR_den-32k_bold_surf-L_reho', 'space-fsLR_den-32k_bold_surf-R_reho', 'space-fsLR_den-32k_bold_surf-correlation_matrix']
bold_ts = ['desc-brain_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold']
ciftis = {'atlas-DesikanKilliany_space-fsLR_den-164k': 'dlabel', 'atlas-DesikanKilliany_space-fsLR_den-32k': 'dlabel', 'atlas-Destrieux_space-fsLR_den-164k': 'dlabel', 'atlas-Destrieux_space-fsLR_den-32k': 'dlabel', 'space-fsLR_den-32k_bold': 'dtseries', 'space-fsLR_den-32k_bold_surf-L_reho': 'dscalar', 'space-fsLR_den-32k_bold_surf-R_reho': 'dscalar', 'space-fsLR_den-32k_bold_surf-correlation_matrix': 'pconn', 'space-fsLR_den-32k_bold_surf_alff': 'dscalar', 'space-fsLR_den-32k_bold_surf_falff': 'dscalar', 'space-fsLR_den-32k_curv': 'dscalar', 'space-fsLR_den-32k_desc-FS_arealdistortion': 'dscalar', 'space-fsLR_den-32k_desc-FS_edgedistortion': 'dscalar', 'space-fsLR_den-32k_desc-FS_strainJ': 'dscalar', 'space-fsLR_den-32k_desc-FS_strainR': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_strainJ': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_strainR': 'dscalar', 'space-fsLR_den-32k_sulc': 'dscalar', 'space-fsLR_den-32k_thickness': 'dscalar'}
debugging = ['desc-brain_bold', 'desc-motion_bold', 'desc-sm_bold', 'space-EPItemplate_desc-brain_bold', 'space-symtemplate_desc-sm_bold', 'space-template_desc-brain_bold', 'dvars', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-reorient_T1w', 'space-template_desc-brain_T1w']
func = ['alff', 'desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-T1w_sbref', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'motion', 'desc-summary_motion', 'dvars', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'space-template_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'space-fsLR_den-32k_bold-dtseries', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'vmhc', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask', 'AtlasSubcortical-s2', 'space-fsLR_den-32k_bold', 'goodvoxels', 'ribbon-only', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-L_space-native_bold', 'hemi-R_space-native_bold', 'space-fsLR_den-32k_wb-spec', 'space-native_wb-spec', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion', 'space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion', 'space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-L_space-fsLR_den-32k_curv', 'hemi-R_space-fsLR_den-32k_curv', 'space-fsLR_den-32k_curv', 'hemi-L_space-fsLR_den-32k_flat', 'hemi-R_space-fsLR_den-32k_flat', 'hemi-L_space-fsLR_den-32k_inflated', 'hemi-R_space-fsLR_den-32k_inflated', 'hemi-L_space-fsLR_den-32k_veryinflated', 'hemi-R_space-fsLR_den-32k_veryinflated', 'hemi-L_space-native_inflated', 'hemi-R_space-native_inflated', 'hemi-L_space-native_veryinflated', 'hemi-R_space-native_veryinflated', 'hemi-L_space-fsLR_den-164k_midthickness', 'hemi-R_space-fsLR_den-164k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-native_midthickness', 'hemi-R_space-native_midthickness', 'hemi-L_space-fsLR_den-32k_pial', 'hemi-R_space-fsLR_den-32k_pial', 'hemi-L_space-native_den-32k_pial', 'hemi-R_space-native_den-32k_pial', 'hemi-L_space-fsLR_den-32k_sphere', 'hemi-R_space-fsLR_den-32k_sphere', 'hemi-L_space-native_desc-MSMSulc_sphere', 'hemi-R_space-native_desc-MSMSulc_sphere', 'hemi-L_space-native_sphere', 'hemi-R_space-native_sphere', 'hemi-L_space-native_desc-reg_sphere', 'hemi-R_space-native_desc-reg_sphere', 'hemi-L_space-native_desc-reg-reg_sphere', 'hemi-R_space-native_desc-reg-reg_sphere', 'hemi-L_space-native_desc-rot_sphere', 'hemi-R_space-native_desc-rot_sphere', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ', 'space-fsLR_den-32k_desc-FS_strainJ', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR', 'space-fsLR_den-32k_desc-FS_strainR', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR', 'space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-L_space-fsLR_den-32k_sulc', 'hemi-R_space-fsLR_den-32k_sulc', 'space-fsLR_den-32k_sulc', 'hemi-L_space-fsLR_den-32k_thickness', 'hemi-R_space-fsLR_den-32k_thickness', 'space-fsLR_den-32k_thickness', 'hemi-L_space-fsLR_den-164k_white', 'hemi-R_space-fsLR_den-164k_white', 'hemi-L_space-fsLR_den-32k_white', 'hemi-R_space-fsLR_den-32k_white', 'hemi-L_space-native_white', 'hemi-R_space-native_white', 'atlas-DesikanKilliany_space-fsLR_den-32k', 'atlas-Destrieux_space-fsLR_den-32k', 'atlas-DesikanKilliany_space-fsLR_den-164k', 'atlas-Destrieux_space-fsLR_den-164k', 'space-fsLR_den-32k_bold_surf_falff', 'space-fsLR_den-32k_bold_surf_alff', 'space-fsLR_den-32k_bold_surf-L_reho', 'space-fsLR_den-32k_bold_surf-R_reho', 'space-fsLR_den-32k_bold_surf-correlation_matrix']
functional_timeseries = ['desc-brain_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-preproc_bold']
giftis = {'hemi-L_space-fsLR_den-164k_midthickness': 'surf', 'hemi-L_space-fsLR_den-164k_white': 'surf', 'hemi-L_space-fsLR_den-32k_curv': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR': 'shape', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_bold': 'func', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask': 'shape', 'hemi-L_space-fsLR_den-32k_flat': 'surf', 'hemi-L_space-fsLR_den-32k_inflated': 'surf', 'hemi-L_space-fsLR_den-32k_midthickness': 'surf', 'hemi-L_space-fsLR_den-32k_pial': 'surf', 'hemi-L_space-fsLR_den-32k_sphere': 'surf', 'hemi-L_space-fsLR_den-32k_sulc': 'shape', 'hemi-L_space-fsLR_den-32k_thickness': 'shape', 'hemi-L_space-fsLR_den-32k_veryinflated': 'surf', 'hemi-L_space-fsLR_den-32k_white': 'surf', 'hemi-L_space-native_bold': 'func', 'hemi-L_space-native_den-32k_pial': 'surf', 'hemi-L_space-native_desc-MSMSulc_sphere': 'surf', 'hemi-L_space-native_desc-reg-reg_sphere': 'surf', 'hemi-L_space-native_desc-reg_sphere': 'surf', 'hemi-L_space-native_desc-rot_sphere': 'surf', 'hemi-L_space-native_inflated': 'surf', 'hemi-L_space-native_midthickness': 'surf', 'hemi-L_space-native_sphere': 'surf', 'hemi-L_space-native_veryinflated': 'surf', 'hemi-L_space-native_white': 'surf', 'hemi-R_space-fsLR_den-164k_midthickness': 'surf', 'hemi-R_space-fsLR_den-164k_white': 'surf', 'hemi-R_space-fsLR_den-32k_curv': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR': 'shape', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold': 'func', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask': 'shape', 'hemi-R_space-fsLR_den-32k_flat': 'surf', 'hemi-R_space-fsLR_den-32k_inflated': 'surf', 'hemi-R_space-fsLR_den-32k_pial': 'surf', 'hemi-R_space-fsLR_den-32k_sphere': 'surf', 'hemi-R_space-fsLR_den-32k_sulc': 'shape', 'hemi-R_space-fsLR_den-32k_thickness': 'shape', 'hemi-R_space-fsLR_den-32k_veryinflated': 'surf', 'hemi-R_space-fsLR_den-32k_white': 'surf', 'hemi-R_space-native_bold': 'func', 'hemi-R_space-native_den-32k_pial': 'surf', 'hemi-R_space-native_desc-MSMSulc_sphere': 'surf', 'hemi-R_space-native_desc-reg-reg_sphere': 'surf', 'hemi-R_space-native_desc-reg_sphere': 'surf', 'hemi-R_space-native_desc-rot_sphere': 'surf', 'hemi-R_space-native_inflated': 'surf', 'hemi-R_space-native_midthickness': 'surf', 'hemi-R_space-native_sphere': 'surf', 'hemi-R_space-native_veryinflated': 'surf', 'hemi-R_space-native_white': 'surf', 'space-fsLR_den-32k_wb-spec': 'spec', 'space-native_wb-spec': 'spec'}
native_nonsmooth = ['alff', 'falff', 'reho']
native_raw = ['alff', 'desc-sm_alff', 'desc-sm_falff', 'falff', 'desc-sm_reho', 'reho', 'desc-zstd-mdmr']
native_smooth = ['desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-T1w_sbref', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'motion', 'desc-summary_motion', 'dvars', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'space-fsLR_den-32k_bold-dtseries', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'mdmr', 'desc-zstd-mdmr', 'dseg', 'AtlasSubcortical-s2', 'space-fsLR_den-32k_bold', 'goodvoxels', 'ribbon-only', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-L_space-native_bold', 'hemi-R_space-native_bold', 'space-fsLR_den-32k_wb-spec', 'space-native_wb-spec', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion', 'space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion', 'space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-L_space-fsLR_den-32k_curv', 'hemi-R_space-fsLR_den-32k_curv', 'space-fsLR_den-32k_curv', 'hemi-L_space-fsLR_den-32k_flat', 'hemi-R_space-fsLR_den-32k_flat', 'hemi-L_space-fsLR_den-32k_inflated', 'hemi-R_space-fsLR_den-32k_inflated', 'hemi-L_space-fsLR_den-32k_veryinflated', 'hemi-R_space-fsLR_den-32k_veryinflated', 'hemi-L_space-native_inflated', 'hemi-R_space-native_inflated', 'hemi-L_space-native_veryinflated', 'hemi-R_space-native_veryinflated', 'hemi-L_space-fsLR_den-164k_midthickness', 'hemi-R_space-fsLR_den-164k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-native_midthickness', 'hemi-R_space-native_midthickness', 'hemi-L_space-fsLR_den-32k_pial', 'hemi-R_space-fsLR_den-32k_pial', 'hemi-L_space-native_den-32k_pial', 'hemi-R_space-native_den-32k_pial', 'hemi-L_space-fsLR_den-32k_sphere', 'hemi-R_space-fsLR_den-32k_sphere', 'hemi-L_space-native_desc-MSMSulc_sphere', 'hemi-R_space-native_desc-MSMSulc_sphere', 'hemi-L_space-native_sphere', 'hemi-R_space-native_sphere', 'hemi-L_space-native_desc-reg_sphere', 'hemi-R_space-native_desc-reg_sphere', 'hemi-L_space-native_desc-reg-reg_sphere', 'hemi-R_space-native_desc-reg-reg_sphere', 'hemi-L_space-native_desc-rot_sphere', 'hemi-R_space-native_desc-rot_sphere', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ', 'space-fsLR_den-32k_desc-FS_strainJ', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR', 'space-fsLR_den-32k_desc-FS_strainR', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR', 'space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-L_space-fsLR_den-32k_sulc', 'hemi-R_space-fsLR_den-32k_sulc', 'space-fsLR_den-32k_sulc', 'hemi-L_space-fsLR_den-32k_thickness', 'hemi-R_space-fsLR_den-32k_thickness', 'space-fsLR_den-32k_thickness', 'hemi-L_space-fsLR_den-164k_white', 'hemi-R_space-fsLR_den-164k_white', 'hemi-L_space-fsLR_den-32k_white', 'hemi-R_space-fsLR_den-32k_white', 'hemi-L_space-native_white', 'hemi-R_space-native_white', 'atlas-DesikanKilliany_space-fsLR_den-32k', 'atlas-Destrieux_space-fsLR_den-32k', 'atlas-DesikanKilliany_space-fsLR_den-164k', 'atlas-Destrieux_space-fsLR_den-164k', 'space-fsLR_den-32k_bold_surf_falff', 'space-fsLR_den-32k_bold_surf_alff', 'space-fsLR_den-32k_bold_surf-L_reho', 'space-fsLR_den-32k_bold_surf-R_reho', 'space-fsLR_den-32k_bold_surf-correlation_matrix']
reference = Resource  ... Multi-File 0                                               alff  ...            1                                       desc-sm_alff  ...            2                                  desc-sm-zstd_alff  ...            3                                     desc-zstd_alff  ...            4                                space-template_alff  ...            ..                                               ...  ...        ... 307               space-fsLR_den-32k_bold_surf_falff  ...            308                space-fsLR_den-32k_bold_surf_alff  ...            309              space-fsLR_den-32k_bold_surf-L_reho  ...            310              space-fsLR_den-32k_bold_surf-R_reho  ...            311  space-fsLR_den-32k_bold_surf-correlation_matrix  ...             [312 rows x 10 columns]
reference_csv = '/home/circleci/docs_env/lib/python3.10/site-packages/CPAC/resources/cpac_outputs.tsv'
template_nonsmooth = ['space-template_alff', 'space-template_dcb', 'space-template_dcw', 'space-template_ecb', 'space-template_ecw', 'space-template_falff', 'space-template_lfcdb', 'space-template_lfcdw', 'space-template_reho']
template_raw = ['space-template_alff', 'space-template_desc-sm_alff', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm_reho', 'space-template_reho']
template_smooth = ['space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-template_desc-brain_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'space-symtemplate_desc-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'vmhc', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask']
to_fisherz = []
to_smooth = ['alff', 'space-template_alff', 'space-template_dcb', 'space-template_dcw', 'space-template_ecb', 'space-template_ecw', 'falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_lfcdw', 'reho', 'space-template_reho']
to_zstd = ['alff', 'desc-sm_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'desc-sm_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'desc-sm_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_reho', 'desc-zstd-mdmr']

pytest

Utilities for Pytest integration.

CPAC.utils.pytest.skipif(condition, reason)[source]

Skip test if we have Pytest, ignore test entirely if not.

Serialization

Strategy

class CPAC.utils.strategy.Strategy[source]
append_name(name)[source]
fork()[source]
get(resource_key)[source]
static get_forking_labels(strategies)[source]
static get_forking_points(strategies)[source]
get_leaf_properties()[source]
get_name()[source]
get_node_from_resource_pool(resource_key)[source]
get_nodes_names()[source]
get_resource_pool()[source]
property resource_pool

Strategy’s ResourcePool dict.

property rpool

Strategy’s ResourcePool.

set_leaf_properties(node, out_file)[source]
update_resource_pool(resources, override=False)[source]

Test Initialization

Assist in initializing CPAC tests resources.

CPAC.utils.test_init.concordance(x, y)[source]

Return the concordance correlation coefficient as defined by Lin (1989).

Parameters:
  • x (list or array) – a list of array of length N of numbers

  • y (list or array) – a list of array of length N of numbers

Returns:

rho_c – the concordance value as a float

Return type:

numpy.float32

CPAC.utils.test_init.create_dummy_node(name: str, fields: list[str] | None = None)[source]

Create a dummy IdentityInterface source for upstream resources.

Parameters:
  • name (str) – a name for the dummy Node

  • fields (list of str, optional) – a list of resources to be present in the created Node. If not provided, the only resource will be called ‘resource’

Return type:

Node

CPAC.utils.test_init.default_bucket_name()[source]

Return the default S3 bucket name used in test suite.

Parameters:

None

Returns:

bucket_name – default S3 bucket name for testing

Return type:

string

CPAC.utils.test_init.download_cpac_resources_from_s3(local_base)[source]

Download the CPAC testing resources directory from S3.

Parameters:

local_base (string) – the local directory to save the ‘cpac_resources’ contents

CPAC.utils.test_init.download_resource_from_s3(s3_url_path)[source]

Download test resource from S3 bucket.

CPAC.utils.test_init.pearson_correlation(nii_1, nii_2)[source]
CPAC.utils.test_init.populate_all_templates()[source]

Populate all of the template files.

Parameters:

None

Return type:

None

CPAC.utils.test_init.populate_template_config(config_type: str) str[source]

Populate a template config file from CPAC_RESOURCE_DIR with actual filepaths.

Parameters:

config_type (string) – config file to populate; accepts ‘data_config’ and ‘pipeline_config’

Returns:

config_test – filepath to the newly written config file for testing

Return type:

string

CPAC.utils.test_init.return_all_niis(base_dir)[source]

Walk through a base directory and all subsequent files.

Return the filepaths of all nifti files found.

Parameters:

base_dir (string) – filepath to the base directory to search through

Returns:

nii_list – a list of filepath strings of the nifti files found in base_dir

Return type:

list

CPAC.utils.test_init.return_aws_creds()[source]

Return the AWS credentials file given by the CPAC_AWS_CREDS environment variable.

Parameters:

None

Returns:

aws_creds – filepath to the AWS credentials with access key id and secret access key

Return type:

string

CPAC.utils.test_init.return_resource_dir()[source]

Return the filepath of the CPAC_RESOURCE_DIR.

Note the CPAC_RESOURCE_DIR environment variable must be set.

Parameters:

None

Returns:

resource_dir – the file path on disk where the cpac resources folder is

Return type:

string

CPAC.utils.test_init.return_resource_subfolder(subfolder)[source]

Funnction to return subfolders of the CPAC_RESOURCE_DIR.

Parameters:

subfolder (string) – subfolder name to return path of

Returns:

resource_subfolder – filepath to the resource subfolder

Return type:

string

CPAC.utils.test_init.return_strats_obj()[source]

Return the file path of the strategies obj file from the CPAC_RESOURCE_DIR.

Parameters:

None

Returns:

strats_obj – filepath to the strategies obj file

Return type:

string

CPAC.utils.test_init.return_subj_measure_dirs(measure)[source]

Grab the base directories of subject’s output files for a given measure or workflow.

Parameters:

measure (string) – the measure or workflow or derivative of interest to parse for; this must be the folder name where all of the subject’s test outputs are located (e.g. ‘network_centrality’)

Returns:

subj_measure_dirs – a list of strings of the base directories for each instance of the desired measure folder within the test subjects outputs

Return type:

list

CPAC.utils.test_init.return_subject_list()[source]

Return the file path of the subject list file from the CPAC_RESOURCE_DIR.

Parameters:

None

Returns:

subject_list – filepath to the subject list yaml file

Return type:

string

CPAC.utils.test_init.return_test_subj()[source]

Return the subject id.

Note the CPAC_RESOURCE_DIR environment variable must be set.

Parameters:

None

Returns:

resource_dir – the file path on disk where the cpac resources folder is

Return type:

string

CPAC.utils.test_init.setup_test_logger(logger_name, log_file, level, to_screen=False)[source]

Initialize and configure a logger that can write to file and (optionally) the screen.

Parameters:
  • logger_name (string) – name of the logger

  • log_file (string) – file path to the log file on disk

  • level (integer) – indicates the level at which the logger should log; this is controlled by integers that come with the python logging package. (e.g. logging.INFO=20, logging.DEBUG=10)

  • to_screen (boolean (optional)) – flag to indicate whether to enable logging to the screen

Returns:

logger – Python logging.Logger object which is capable of logging run- time information about the program to file and/or screen

Return type:

logging.Logger object

CPAC.utils.test_init.smooth_nii_file(self, nii_file, fwhm, mask_file=None)[source]

Gaussian smooth nifti files and optionally use a mask on the smoothed data.

Parameters:
  • nii_file (string) – filepath to the nifti file to smooth

  • fwhm (float) – FWHM for Gaussian smoothing kernel, in mm

  • mask_file (string (optional); default=None) – filepath to the mask file to use

Returns:

smooth_arr – smoothed nifti image as a numpy array

Return type:

numpy.ndarray

Test Mocks

CPAC.utils.test_mocks.configuration_strategy_mock(method='FSL')[source]
CPAC.utils.test_mocks.file_node(path, file_node_num=0)[source]

Test Resources

CPAC.utils.test_resources.setup_test_wf(s3_prefix, paths_list, test_name, workdirs_to_keep=None)[source]

Set up a basic template Nipype workflow for testing single nodes or small sub-workflows.

Tests

The Trimmer

CPAC.utils.trimmer.compute_datasink_dirs(graph, datasink, output_dir=None, container=None)[source]
CPAC.utils.trimmer.expand_workflow(wf)[source]
CPAC.utils.trimmer.is_datasink(n)[source]
CPAC.utils.trimmer.list_files(path, s3_creds_path=None)[source]
CPAC.utils.trimmer.the_trimmer(wf, output_dir=None, container=None, s3_creds_path=None)[source]

The trimmer: trimming your workflow based on its datasinks.

The assumption here is that all your desired outputs will be placed in an output directory by using a DataSink node.

By analysing a specific output directory, and comparing what is in it with the DataSinks of a workflow, it is possible to audit which Datasinks have already outputted to the output directory. If the DataSink has already fulfilled its function, we infer that previous nodes also had, since they are prerequisites for the DataSink to run. This is the simplest case in which we can prune nodes (i.e. do not execute them), optimizing the execution time.

A brief syntax note: [node] is a node → is a connection, disconsidering which field/attribute →(field)→ indicates a connection considering the field ✓ is a datasink with an existing file ❌ is a datasing witn an missing file

E.g.

[node1] → [node2] → [node3] → [datasink to file.txt ✓]

since file.txt already exist, there is no need to execute the [node3]. Since [node2] only outputs to [node3], and [node3] will not be executed, we can prune [node2]. Same for [node1]. In this case, our workflow will not have any more nodes to run.

There are more complex cases:

1) A node outputs for several nodes, and some of their results are not in the output directory.

[node1] → [node2] → [node3] → [datasink to file1.txt ✔]

↳ [node4] → [datasink to file2.txt ❌]

for this case, we cannot prune [node1], since its output is used in another branch, for [node4], that is not cached. After trimming, the remaining workflow is:

[node1]

↳ [node4] → [datasink to file2.txt ❌]

2) The node has several outputs, and an uncached branch down the graph requires one of its outputs.

[registration] →(warped image)→ [datasink to warped.nii.gz ✔]

↳(transforms)→ [apply transforms] → [datasink to func_warped.nii.gz ❌] [functional] ↗

given func_warped.nii.gz is not cached, we need to perform “apply transforms”, that requires the transforms from the [registration] node. In this case, even that warped.nii.gz is cached, we will reexecute the [registration] again to get the transforms. After trimming, the remaining workflow is:

[registration]

↳(transforms)→ [apply transforms] → [datasink to func_warped.nii.gz ❌] [functional] ↗

For this implementation, we disregard MapNodes, as their outputs is harder to check.

Iterables are considered in the implementation by expanding the original workflow into what is called an execution graph, creating a node for each iterable value.

Parameters:
  • wf (Workflow) – A Nipype workflow to be pruned.

  • output_dir (Path) – The directory in which the outputs are stored. If not provided, value is inferred from the DataSink nodes.

  • container (Path) – The subdirectory from the output_dir in which the output are stored. If not provided, value is inferred from the DataSink nodes.

  • s3_creds_path (Path) – Path to S3 credentials, in case output_dir is in a S3 bucket.

Returns:

  • wf_new (Workflow) – Prunned workflow

  • (replacement_mapping, deletions) ((Dict, List)) – replacement_mapping contains the nodes replaces with input nodes, pointing to files from the output_dir

    deletions contains the nodes removed from the workflow, as they do not need to be executed

Typing

Utilities

General-purpose utilities for C-PAC.

class CPAC.utils.utils.ScanParameters(scan_parameters: str | dict, subject_id: str, scan: str)[source]

A dictionary of scan parameters and access methods.

check(val_to_check: str, throw_exception: bool)[source]

Check that a value is populated for a given key in a parameters dictionary.

fetch(keys: list[str] | None = None, *, match_case: Literal[False], throw_exception: bool) Any[source]
fetch(keys: list[str] | None = None, *, match_case: Literal[True], throw_exception: bool) tuple[Any, tuple[str, str]]

Fetch the first found parameter from a scan params dictionary.

Returns:

  • value – The value of the parameter.

  • keys, optional – The matched keys (only if match_case is True)

  • throw_exception – Raise an exception if value is "" or None?

fetch_and_convert(keys: list[str], convert_to: type | None = None, fallback: Any | None = None, warn_typeerror: bool = True, throw_exception: bool = False) Any[source]

Fetch a parameter from a scan params dictionary and convert it to a given type.

Catch TypeError exceptions and return a fallback value in those cases.

Parameters:
  • keys – if multiple keys provided, the value corresponding to the first found will be returned

  • convert_to – the type to return if possible

  • fallback – a value to return if the keys are not found in scan_parameters

  • warn_typeerror – log a warning if value cannot be converted to convert_to type?

  • throw_exception – raise an error for empty string or NoneTypes?

Returns:

The gathered parameter coerced to the specified type, if possible. fallback otherwise.

Return type:

value

CPAC.utils.utils.add_afni_prefix(tpattern)[source]

Add ‘@’ prefix to tpattern.txt filename.

CPAC.utils.utils.check_command_path(path)[source]

Chek if command path exists.

CPAC.utils.utils.check_config_resources(c: Configuration | dict) tuple[float | int, int, int, int][source]

Check pipeline config againts computer resources.

CPAC.utils.utils.check_prov_for_motion_tool(prov)[source]

Check provenance for motion correction tool.

CPAC.utils.utils.check_prov_for_regtool(prov)[source]

Check provenance for registration tool.

CPAC.utils.utils.check_random_state(seed)[source]

Turn seed into a np.random.RandomState instance.

Code from scikit-learn (https://github.com/scikit-learn/scikit-learn)

Parameters:

seed (None | int | instance of RandomState) – If seed is None, return the RandomState singleton used by np.random. If seed is an int, return a new RandomState instance seeded with seed. If seed is already a RandomState instance, return it. Otherwise raise ValueError.

CPAC.utils.utils.check_system_deps(check_ants=False, check_ica_aroma=False, check_centrality_degree=False, check_centrality_lfcd=False)[source]

Check system for neuroimaging tools AFNI, C3D, FSL and ANTs.

CPAC.utils.utils.compute_fisher_z_score(correlation_file, timeseries_one_d, input_name)[source]

Compute the fisher z transform of the input correlation map.

If the correlation map contains data for multiple ROIs then return z score for each ROI as a seperate NIfTI file.

Parameters:

correlation_file (string) – Input correlations file

Returns:

out_file – list of z_scores for mask or ROI

Return type:

list (nifti files)

CPAC.utils.utils.concat_list(in_list1=None, in_list2=None)[source]

Concatenate a pair of lists.

Parameters:
  • in_list1 (list or str) – file path or a list of file paths

  • in_list2 (list or str) – file path or a list of file paths

Returns:

out_list – a list of file paths

Return type:

list

CPAC.utils.utils.correlation(matrix1, matrix2, match_rows=False, z_scored=False, symmetric=False)[source]

Calcluate the correlation between two matrices.

CPAC.utils.utils.create_id_string(cfg, unique_id, resource, scan_id=None, template_desc=None, atlas_id=None, fwhm=None, subdir=None, extension=None)[source]

Create the unique key-value identifier string for BIDS-Derivatives file names.

This is used in the file renaming performed during the Datasink connections.

Example

>>> from CPAC.utils.configuration import Configuration
>>> create_id_string(Configuration(), 'sub-1_ses-1',
...                  'res-derivative_desc-Mean-1_timeseries',
...                  scan_id='rest', atlas_id='Yeo_desc-7')
'sub-1_ses-1_task-rest_atlas-Yeo7_res-3mm_desc-Mean1_timeseries'
CPAC.utils.utils.create_log(wf_name='log', scan_id=None)[source]

Workflow to create log.

CPAC.utils.utils.delete_nested_value(d, keys)[source]

Delete nested values.

Parameters:
Returns:

updated

Return type:

dict

Examples

>>> delete_nested_value(
...     {'nested': {'key1': 'value', 'key2': 'value'}},
...     ['nested', 'key1'])
{'nested': {'key2': 'value'}}
CPAC.utils.utils.extract_output_mean(in_file, output_name)[source]

Copy from a 1D file to a text file.

function takes ‘in_file’, which should be an intermediary 1D file from individual-level analysis, containing the mean of the output across all voxels.

it then parses this value and writes it to a .csv file named output_means.csv located in the subject’s output directory

CPAC.utils.utils.find_files(directory, pattern)[source]

Find files in directory.

CPAC.utils.utils.get_fisher_zscore(input_name, map_node=False, wf_name='fisher_z_score')[source]

Run the compute_fisher_z_score function as part of a one-node workflow.

CPAC.utils.utils.get_flag_wf(wf_name='get_flag')[source]

Create a workflow to get a flag.

CPAC.utils.utils.get_last_prov_entry(prov)[source]

Get the last provenance entry.

CPAC.utils.utils.get_operand_string(mean, std_dev)[source]

Get operand string for fslmaths.

Parameters:
  • mean (string) – path to img containing mean

  • std_dev (string) – path to img containing standard deviation

Returns:

op_string – operand string

Return type:

string

CPAC.utils.utils.get_scan_params(subject_id: str, scan: str, pipeconfig_start_indx: int | str | None, pipeconfig_stop_indx: int | str | None, data_config_scan_params: dict | str | None = None) tuple[str | None, str | None, int | None, int | None, int | None, int | None, Literal['i', 'i-', 'j', 'j-', 'k', 'k-', ''], float | None][source]

Extract slice timing correction parameters and scan parameters.

Parameters:
  • subject_id – subject id

  • scan – scan id

  • pipeconfig_start_indx – starting volume index as provided in the pipeline config yaml file

  • pipeconfig_stop_indx – ending volume index as provided in the pipeline config yaml file

  • data_config_scan_params – file path to scan parameter JSON file listed in data config yaml file or loaded paramater dictionary

Returns:

CPAC.utils.utils.get_zscore(map_node=False, wf_name='z_score')[source]

Workflow to calculate z-scores.

Parameters:

wf_name (string) – name of the workflow

Returns:

wf

Return type:

workflow object

Notes

Source

Workflow Inputs:

inputspec.input_file : string
    path to input functional derivative file for which z score has to be calculated
inputspec.mask_file : string
    path to whole brain functional mask file required to calculate zscore

Workflow Outputs:

outputspec.z_score_img : string
     path to image containing Normalized Input Image Z scores across full brain.

High Level Workflow Graph:

../images/generated/zscore.png

Detailed Workflow Graph:

../images/generated/zscore_detailed.png

Example

>>> wf = get_zscore('example_input')
>>> wf.inputs.inputspec.input_file = '/home/data/graph_working_dir/calculate_centrality/degree_centrality_binarize.nii.gz'
>>> wf.inputs.inputspec.mask_file = '/home/data/graphs/GraphGeneration/new_mask_3m.nii.gz'
>>> wf.run()  
CPAC.utils.utils.list_item_replace(l, old, new)[source]

Replace an item in a list.

Parameters:
  • l (list or string)

  • old (any) – item to replace

  • new (any) – new item

Returns:

l – updated

Return type:

list or string

Examples

>>> list_item_replace(['AFNI', 'FSL'], 'AFNI', '3dSkullStrip')
['3dSkullStrip', 'FSL']
>>> list_item_replace(['AFNI', 'FSL'], 'FSL', 'BET')
['AFNI', 'BET']
CPAC.utils.utils.lookup_nested_value(d, keys)[source]

Look up nested values.

Parameters:
Returns:

yaml

Return type:

str or dict

Examples

>>> lookup_nested_value({'nested': {'True': True}}, ['nested', 'True'])
True
>>> lookup_nested_value({'nested': {'None': None}}, ['nested', 'None'])
''
CPAC.utils.utils.read_json(json_file)[source]

Read a JSON file and return the contents as a dictionary.

CPAC.utils.utils.remove_False(d, k)[source]

Remove “Off” and False from a list at a given nested key.

Parameters:
Returns:

d – updated

Return type:

dict

Examples

>>> remove_False({'a': {'b': [1, False, 2, "Off", 3]}}, ['a', 'b'])
{'a': {'b': [1, 2, 3]}}
CPAC.utils.utils.remove_None(d, k)[source]

Remove “None” and None from a list at a given nested key.

Parameters:
Returns:

d – updated

Return type:

dict

Examples

>>> remove_None({'a': {'b': [1, None, 2, "None", 3]}}, ['a', 'b'])
{'a': {'b': [1, 2, 3]}}
CPAC.utils.utils.repickle(directory)[source]

Deprecated since version 1.8.8.

Python 2’s end of life was over 4 years prior to this release. A user jumping from a C-PAC version that used Python 2 can use this function in any C-PAC version from 1.6.2 up until its removal in an upcoming version.

Recursively check a directory; convert Python 2 pickles to Python 3 pickles.

directory : str

None

CPAC.utils.utils.replace_in_strings(d, replacements=None)[source]

Recursively replace substrings.

Parameters:
  • d (any)

  • replacements (list of 2-tuples) –

    0str

    substring to replace

    1str

    replacement substring

Returns:

d – same as input, but updated

Return type:

any

Examples

>>> replace_in_strings({'key': 'test${resolution_for_func_preproc}'})
{'key': 'test${func_resolution}'}
CPAC.utils.utils.safe_shape(*vol_data)[source]

Check if the volume of multiple ndarrays are the same shape.

The volume is encoded in the first three dimensions of the ndarray.

Parameters:
  • vol_data0 (ndarray) – Volumes to check

  • vol_data1 (ndarray) – Volumes to check

  • ... (ndarray) – Volumes to check

  • vol_datan (ndarray) – Volumes to check

Returns:

same_volume – True only if all volumes have the same shape.

Return type:

bool

CPAC.utils.utils.set_nested_value(d, keys, value)[source]

Set nested values.

Parameters:
Returns:

updated

Return type:

dict

Examples

>>> set_nested_value({}, ['nested', 'keys'], 'value')
{'nested': {'keys': 'value'}}
CPAC.utils.utils.update_config_dict(old_dict)[source]

Convert an old config dict to a new config dict.

Parameters:

old_dict (dict)

Returns:

  • new_dict (dict) – 1.8 nested config dictionary

  • old_dict (dict) – remaining undefined mappings

  • combined_dict (dict) – 1.8 nested config dictionary plus remaining undefined mappings

Examples

>>> a, b, c = update_config_dict({
...     'pipelineName': 'example-pipeline', '2': None})
>>> a
{'pipeline_setup': {'pipeline_name': 'example-pipeline'}}
>>> b
{'2': None}
>>> c
{'pipeline_setup': {'pipeline_name': 'example-pipeline'}, '2': None}
CPAC.utils.utils.update_nested_dict(d_base, d_update, fully_specified=False)[source]

Update dictionary of varying depth.

Parameters:
  • d_base (dict) – original dictionary

  • d_update (dict) – dictionary with updates

  • fully_specified (bool) – if True, overwrite instead of update

Returns:

d_base – original dictionary with updates

Return type:

dict

Examples

>>> d_base = {'pipeline_name': 'cpac-default-pipeline',
...     'output_directory': {'path': '/output',
...     'write_func_outputs': False,
...     'write_debugging_outputs': False,
...     'output_tree': 'default',
...     'quality_control': {
...         'generate_quality_control_images': True,
...         'generate_xcpqc_files': True}},
...     'working_directory': {'path': '/tmp', 'remove_working_dir': True},
...     'log_directory': {'run_logging': True, 'path': '/logs'},
...     'system_config': {'maximum_memory_per_participant': 1,
...     'max_cores_per_participant': 1,
...     'num_ants_threads': 4,
...     'num_participants_at_once': 1},
...     'Amazon-AWS': {'aws_output_bucket_credentials': None,
...                    's3_encryption': False}}
>>> d_update = {'pipeline_name': 'cpac_fmriprep-options',
...     'system_config': {'num_ants_threads': 1},
...     'Amazon-AWS': {'s3_encryption': True}}
>>> str(update_nested_dict(d_base, d_update)) == str({
...     'pipeline_name': 'cpac_fmriprep-options', 'output_directory': {
...         'path': '/output', 'write_func_outputs': False,
...         'write_debugging_outputs': False, 'output_tree': 'default',
...         'quality_control': {
...             'generate_quality_control_images': True,
...             'generate_xcpqc_files': True}
...     }, 'working_directory': {
...        'path': '/tmp', 'remove_working_dir': True
...     }, 'log_directory': {'run_logging': True, 'path': '/logs'},
...     'system_config': {
...     'maximum_memory_per_participant': 1,
...     'max_cores_per_participant': 1,
...     'num_ants_threads': 1, 'num_participants_at_once': 1
... }, 'Amazon-AWS': {
...     'aws_output_bucket_credentials': None, 's3_encryption': True}})
True
>>> tse_base = {'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/CC400.nii.gz': 'Avg',
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Avg'
... }, 'realignment': 'ROI_to_func'}}
>>> str(update_nested_dict(tse_base, {})) == str({
...     'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...         '/cpac_templates/CC400.nii.gz': 'Avg',
...         '/cpac_templates/aal_mask_pad.nii.gz': 'Avg'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'tse_roi_paths': {'/cpac_templates/rois_3mm.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/rois_3mm.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'roi_paths_fully_specified': False,
...     'tse_roi_paths': {'/cpac_templates/rois_3mm.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/CC400.nii.gz': 'Avg',
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Avg',
...     '/cpac_templates/rois_3mm.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'roi_paths_fully_specified': False,
...     'tse_roi_paths': {'/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True,
...     'tse_roi_paths': {
...         '/cpac_templates/CC400.nii.gz': 'Avg',
...         '/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'tse_roi_paths': {'/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
CPAC.utils.utils.update_pipeline_values_1_8(d_old)[source]

Update pipeline config values that changed from C-PAC 1.7 to 1.8.

Parameters:

d_old (dict)

Returns:

d – updated

Return type:

dict

Examples

>>> update_pipeline_values_1_8({'segmentation': {'tissue_segmentation': {
...     'using': ['FSL-FAST Thresholding', 'Customized Thresholding']}}})
{'segmentation': {'tissue_segmentation': {'using': ['FSL-FAST'], 'FSL-FAST': {'thresholding': {'use': 'Custom'}}}}}
>>> update_pipeline_values_1_8({'segmentation': {'tissue_segmentation': {
...     'using': ['FSL-FAST Thresholding']}}})
{'segmentation': {'tissue_segmentation': {'using': ['FSL-FAST'], 'FSL-FAST': {'thresholding': {'use': 'Auto'}}}}}
CPAC.utils.utils.update_values_from_list(d_old, last_exception=None)[source]

Convert 1-length lists of an expected type to single items of that type…

…or to convert singletons of an expected list of a type into lists thereof. Also handles some type conversions against the schema.

Parameters:
  • d_old (dict)

  • last_exception (Exception or None) – if the same exception recurs, raise it.

Returns:

d – updated

Return type:

dict

Examples

>>> update_values_from_list({'pipeline_setup': {
...     'pipeline_name': ['one_string']}})
{'pipeline_setup': {'pipeline_name': 'one_string'}}
>>> update_values_from_list({'nuisance_corrections': {
...     '1-ICA-AROMA': {'run': [False]}}})
{'nuisance_corrections': {'1-ICA-AROMA': {'run': [False]}}}
CPAC.utils.utils.write_output_json(json_data, filename, indent=3, basedir=None)[source]

Write a dictionary to a JSON file.

CPAC.utils.utils.write_to_log(workflow, log_dir, index, inputs, scan_id)[source]

Write into log file the status of the workflow run.

CPAC.utils.utils.zscore(data, axis)[source]

Calculate the z-score of a dataset along a given axis.

Versioning

Gather and report versions.