API Reference

immlib is a library of tools for manipulating immutable scientific data.

The immlib library is designed to enable immutable data structures and lazy computation in a scientific context, and it works primarily via a collection of utility functions and through the use of decorators, which are generally applied to classes and their members to declare how an immutable data-structure’s members are related. Taken together, these utilities form a DSL-like system for declaring workflows and immutable data-structures with full inheritance support.

units

The registry for units tracked by immlib. The immlib.units object is a global pint-module unit registry that can be used as a single global place for tracking units. Immlib functions that interact with units generally take an argument ureg that can be used to modify this registry. Additionally, the default registry (this object, immlib.units) can be temporarily changed in a local block using with immlib.default_ureg(ureg): ....

Type:

pint.UnitRegistry

version

A representation of the immlib version. The version string may be obtained via immlib.version.string; major, minor, and micro numbers (when present) may be obtained via immlib.version.major, immlib.version.minor, and immlib.version.micro (when not provided the are set to None), and a stage tag (a string), if given, can be obtained via immlib.version.stage.

Type:

immlib.Version

submodules

A tuple of strings, each of which is the name of one of the submodules in immlib. The modules are listed in load-order and all immlib submodules, including private submodules, are included.

Type:

tuple of str

docproc

This object is used to process all of the doc-strings in the immlib library; it should be used only with the immlib.docwrap decorator, which can safely be applied anywhere in a sequence of decorators and which correctly applies the wraps decorator to its argument. Function documentation is always processed using the sections=('Parameters', 'Returns', 'Raises', 'Examples', 'Inputs', 'Outputs') parameter and the with_indent(4) decorator. The base-name for the function f is f.__module__ + '.' + f.__name__.

Type:

docrep.DocstringProcessor object

class ArrayIndex(array, freeze=True)

A type that indexes the elements of an array for easy searching.

The ArrayIndex class is a class that stores a (typically read-only) numpy array whose elements must all be unique and that creates an index of that array’s elements. ArrayIndex objects primarily support a a find method that can be used to look up object indices.

ArrayIndex objects require that the arrays they are given contain unique objects that are sortable and hashable.

Examples

>>> from immlib import ArrayIndex
>>> labels = [['r1c1', 'r1c2'], ['r2c1', 'r2c2']]
>>> index = ArrayIndex(labels)
>>> index.find('r1c1')
    (0, 0)
>>> index.find(['r2c1', 'r1c2'])
    (array([1, 0]), array([0, 1]))
>>> index.find(['r2c2', 'r2c1', 'r1c2'], ravel=True)
    array([3, 1, 2])

find(ids, *, ravel=False, **kw)

Finds and returns the indices of the given identities.

index.find(id) returns the index, in the original array on which index is based, of the identity id. If id is not in the original array, then a KeyError is raised.

Parameters:

• ids (array-like) – The identity or identities to look up in the index.

• ravel (boolean, optional) – Whether the return value should be an array representing a raveled index into the flattened version of the original indexed array (True) or a tuple representing an unraveled multi-index into the original indexed array (False). The default is False.

• default (object, optional) – If default is not given, then an error is raised when an identity is not found. If default is given, however, the default value is inserted into the place of any missing indices and no error is raised.

Returns:

The indices, into the original indexed array, of the given identities, ids.

Return type:

indices

property flatdata

Returns a named tuple containing the flattened data used by the ArrayIndex type to lookup identities.

index.flatdata returns a named 2-tuple with keys ident and index. The ident element is a read-only numpy array containing the sorted and flattened identities represented in the original array. The index element is a read-only numpy array containing the argsort of the flattened original array object.

class Immutable(*args, **kw)

A type that becomes immutable immediately after initialization.

Any class that inherits from Immutable should implement an __init__ method, within which it is allowed to change the attributes of the self object normally. After the __init__ method terminates, the object becomes read-only and can no longer be updated.

class ImmutableType(name, bases, attrs, **kwargs)

A meta-class for types that are immutable.

When this metaclass is used in a class, objects of the type become immutable immediately after the __init__ method is run. Such types should not overload the __new__ classmethod and instead should see to their initialization in __init__ as usual. Once the __init__ method has finished, the __setattr__, __delattr__, __setitem__, and __delitem__ methods will raise a TypeError.

class ImmutableBase(*args, **kw)

The base class of all immlib immutable classes.

class Load(template=None)

Loads a Python object from a stream or path then returns the object.

immlib.load(path, format) loads a Python object from the given path using the named file format and returns the loaded object or rasies an error on failure. If the format can be deduced from the path suffix, then it may be omitted.

immlib.load(stream, format) reads a Python object from the given stream object using the named format and returns the object. The format argument cannot be omitted when the first argument is a stream.

In fact, immlib.load is an object of the immlib.pathlib.Load type that primarily behaves like a function. The immlib.load.register and immlib.load.unregister methods can be used to add understood formats.

Parameters:

• source (path-like or stream) – The input source from which the object is to be loaded. This may be either a path-like object or a readable IOBase stream.

• format (str or None, optional) – If provided, format must be a string that names a format that has been previously registered with load using the load.register method. If format is not provided or is None (the default), then an attempt is made to deduce the format using the suffix of the source argument, assuming that source is a path and not a stream. If source is a stream or if the suffix does not indicate a specific format, then a ValueError is raised.

• **kwargs – Any additional parameters are passed to the registered export function.

Returns:

The object that was loaded from the given stream or path.

Return type:

object

Raises:

• TypeError – If the input source is not a stream or path-like object or if the format is not a string.

• ValueError – If the format is not recognized or if it cannot be deduced from the source.

static from_dir(src, filter=None)

Loads a nested dictionary structure of a directory.

Load.from_dir(path) returns path if path refers to a file. Alternatively, if path refers to a directory, this function returns a lazy dictionary whose keys are the names of the entries in the directory and whose values are the result of calling Load.from_dir on their paths.

class MetaObject(*args, **kwargs)

Base planobject type for objects that keep track of metadata.

Parameters:

metadata (None or Mapping, optional) – The dictionary of metadata that is to be attached to the object. If the argument None is provided (the default), then the empty dictionary is used.

metadata

A lazy dictionary of the metadata tracked by the object.

Type:

ldict

clear_metadata()

Returns a duplicate object with its metadata dictionary cleared.

dropmeta(*args)

Returns a duplicate object with given metadata keys cleared.

The arguments must be keys, which are dropped from the metadata of the duplicate object.

plan = plan(<1 calcs>, <1 params>)

set_metadata(md)

Returns a duplicate object with the given metadata dictionary.

The argument must be a dict-like object.

withmeta(*args, **kwargs)

Return a duplicate object with updated metadata.

The arguments and keyword arguments to withmeta are merged, left-to-right, into the current metadata; this new dictionary is used as the metadata parameter of the new object.

class OSFClient(project='xxxxx', storage='osfstorage', file_cache_mode=None, local_cache_dir=None, content_type_method=<function guess_type>, pagesize=100, mkdir_mode=509)

Client class for the OSF.

class OSFPath(cloud_path: CloudPathT, *args: Any, **kwargs: Any)

class OSFPath(cloud_path: str | CloudPath, *args: Any, **kwargs: Any)

Class for representing and operating on OSF repositories.

OSFPath(path) returns a path object representing an OSF path; OSF paths use the format osf://<project-id>/<path>. The project ID is derived from the OSF tag; i.e., the website https://osf.io/<project-id> is the primary website of the OSF project. The project-ID may be followed by a colon and an OSF storage name (the project ID by itself is equivalent to osf://<project-ID>:osfstorage/).

For example, the project found at the OSF website https://osf.io/bw9ec/ has the URL osf://bw9ec/.

Parameters:

• cloud_path (str or path-like) – The OSF path that the created OSFPath object is to represent.

• client (OSFClient or None, optional) – The OSFClient object to use. The OSFClient is responsible primarily for the caching of data locally. If OSFClient is None, then an OSFClient object is created for the project using a temporary cache directory.

• local_cache_dir (str or path-like, optional) – The local directory in which cache files should be stored. This option is ignored if client is not None; otherwise it is passed to the created client object. The cache directory is the root cache directory for the entire OSF project.

• file_cache_mode (cloudpathlib.enums.FileCacheMode, optional) – How often to clear the file cache; see [cloudpathlib’s caching docs](https://cloudpathlib.drivendata.org/stable/caching/) for more information about the options in cloudpathlib.enums.FileCacheMode.

• mkdir_mode (int, optional) – The mode to use when making directories in the cache. By default this is 0o775. This option is ignored if the client option is not None.

• pagesize (int, optional) – The number of items to include in a single page when paging directory contents from the OSF server. The default is 100. This option is ignored if the client option is not None.

client

alias of OSFClient

property drive

The drive prefix (letter or UNC path), if any. _(Docstring copied from pathlib.Path)_

is_dir()

Whether this path is a directory.

_(Docstring copied from pathlib.Path)_

is_file()

Whether this path is a regular file (also True for symlinks pointing to regular files).

_(Docstring copied from pathlib.Path)_

mkdir(parents=False, exist_ok=False)

Create a new directory at this given path.

_(Docstring copied from pathlib.Path)_

stat()

Return the result of the stat() system call on this path, like os.stat() does.

_(Docstring copied from pathlib.Path)_

touch(exist_ok: bool = True)

Create this file with the given access mode, if it doesn’t exist.

_(Docstring copied from pathlib.Path)_

class Save(template=None)

Saves a Python object to a stream or path then returns the stream/path.

immlib.save(path, object, format) saves the given object to the given path using the named file format and returns the path on success or rasies an error on failure. If the format can be deduced from the path suffix, then it may be omitted.

immlib.save(stream, object, format) writes the given object to the given stream object using the named format and returns the stream.

In fact, immlib.save is an object of the immlib.pathlib.Save type that primarily behaves like a function. The immlib.save.register and immlib.save.unregister methods can be used to add understood formats.

Parameters:

• dest (path-like or stream) – The output destination to which the object is to be saved. This may be either a path-like object or a writeable IOBase stream.

• obj (object) – Any object that can be saved in the given format.

• format (str or None, optional) – If provided, format must be a string that names a format that has been previously registered with save using the save.register method. If format is not provided or is None (the default), then an attempt is made to deduce the format using the suffix of the dest argument, assuming that dest is a path and not a stream. If dest is a stream or if the suffix does not indicate a specific format, then a ValueError is raised.

• **kwargs – Any additional parameters are passed to the registered export function.

Returns:

The destination stream or path. If the dest argument is a path name, then a path object is returned instead of the path name.

Return type:

path-like or stream

Raises:

• TypeError – If the destination is not a stream or path-like object or if the format is not a string.

• ValueError – If the format is not recognized or if it cannot be deduced from the destination object.

class Version(string=None, /, *, package_name=None, pyproject_path=None, on_error='warn', tag_prefixes=('rc', 'a', 'b'))

A type that represents a Python package version.

Python packages are represented simultaneously as version strings, version tuples, and by the version components major, minor, micro, and stage.

Parameters:

• string (str or None, optional) – The version string to be represented. If this argument is not provided, then one or both of the package_name and pyproject_path options must be provided so that the version string can be obtained via the package version or the pyproject.toml file.

• package_name (str or None, optional) – If the first argument (string) is provided, then this argument is ignored; otherwise, the version string is first searched for by this package name using the importlib or importlib_metadata packages. If found, then this version string is represented in the Version object.

• pyproject_path (path-like or None, optional) – If the first argument (string) is not given and the package_name is not given, then the version is searched for in the pyproject.toml file given by this path. In order for such a file to be valid, it must contain a line that, when stripped of whitespace, begins with the string ‘version=’ followed by a string representation (e.g., ‘version=”1.12.5”’). If such a line is found in the [project] section of the TOM: file pointed to by this argument, then it is represented as the version string in the Version object.

• on_error ({'warn' | 'ignore' | 'raise'}, optional) – How to handle failures to deduce or parse the version number. If ‘raise’ is given, then the errors are allowed to be raised. If ‘warn’, then a warning is raised and a null version is returned. If ‘ignore’, then errors are ignored and a null version is returned. The default is ‘raise’.

• tag_prefixes (tuple of str, optional) – An optional tuple of strings that can appear as the prefixes of stage tagss at the end of the version string. By default, this is (‘rc’, ‘a’, ‘b’), so version strings like ‘1.1.12a6’ and ‘1.1.12rc6’ are valid but ‘1.1.12c6’ is not.

string

The string representing the package version. For example “1.2.15” or “0.2.2.dev1”.

Type:

str

tuple

The components of the version string, for example, (1, 2, 15) or (0, 2, 2, ‘dev1’). Any missing component is excluded.

Type:

tuple of int and str

major

The major version number, typically indicates major API version.

Type:

int

minor

The minor version number, typically indicates minor API version.

Type:

int

micro

The micro version number, typically indicates patch increment number.

Type:

int

stage

The development stage of the version. For example ‘dev1’ or ‘rc2’.

Type:

str

getstring(pyproject_path=None)

Returns the current version string for the given package name.

Version.getstring(package_name) returns the version string of the package with the given package name.

Version.getstring(pyproject_path=path) returns the version string found in the pyproject.toml file found at the given path.

Version.getstring(package_name, path) returns Version.getstring(package_name) if the given package_name is found, otherwise returns Version.getstring(pyproject_path=path).

alike_units(a, b, /, *, ureg=None)

Returns True if the arguments are alike units, otherwise False.

alike_units(a, b) returns True if a and b can be cast to each other in terms of units and False otherwise. Both a and b can either be units, unit names, or quantities with units. If either a or b is neither a unit nor a quantity, then it is considered equivalent to having units of None, i.e., no units.

Parameters:

• a (unit-like) – A unit object or the name of a unit or a quantity.

• b (unit-like) – A unit object or the name of a unit or a quantity.

• ureg (pint.UnitRegistry, None, Ellipsis, optional) – The pint.UnitRegistry object to use. If Ellipsis, then the immlib.units registry is used. If None, then the registry of object a is used if available or that of object b if not. If neither a nor b has an available registry, then immlib.units is used.

Returns:

True if the units a and b are alike and False otherwise.

Return type:

bool

argfilter(fn=None, /, **kwargs)

A decorator that creates decorators that filter function arguments.

A function decorated with @argfilter is turned into a an argument filter function, which itself can be used to decorate functions whose arguments need to be filtered.

In the definition of the filter function, the names of arguments must match those of the arguments they will be filtering on other functions. The filter function must return a tuple of the filtered values in the order they are defined in the function’s argument list. The arguments may be given in any order, but a * in the arguments list indicates that any of the arguments following the * are not themselves being filtered and thus will not be returned from the filter function.

When a filter function is used to decorate another function, the decorator can optionally be given named arguments where the name corresponds to one of the arguments to the origional filter definition and the value corresponds to the name that is used for this parameter in the decorated functions. In this way, the parameter names don’t have to match exactly those of the filter function and can instead be specified in the decoration.

Examples

>>> @argfilter
... def fix_angle(angle, *, unit):
...     angle = np.asarray(angle)
...     if unit == 'degrees':
...         angle = np.pi / 180 * angle
...     elif unit != 'radians':
...         raise ValueError(f'unrecognized unit: {unit}')
...     return (angle,)
... @fix_angle
... def cos_halfangle(angle, unit='radians'):
...     return np.cos(angle / 2)

>>> cos_halfangle([0, 360], 'degrees')
    array([1., -1.])

>>> @fix_angle(angle='theta')
... def sin_halfangle(theta, unit='radians'):
...     return np.sin(theta / 2)

>>> sin_halfangle([0, 360], 'degrees')
    array([0., 0.])

class args(*args, **kwargs)

An object type that represents a set of function arguments.

args(x1, x2 ... k1=v1, k2=v2 ...) yields an args object that represents the positional arguments x1, x2 ... and the named arguments k1=v1, k2=v2, etc.

If a is an instance of args and f is a function, then the arguments in a can be applied to f using either of the following .. method:: - f @ a

- ``a.passto(f)``

Note that if f is an object that defines the __matmul__ method, then the former syntax will call that method instead of the __rmatmul__ method of the args object a and thus won’t work.

copy(args=None, kwargs=None)

Returns a copy of the current args, potentially with updates.

array_args(fn=None, /, *args)

Converts arguments of the decorated function into NumPy arrays.

The decorator @array_args, when applied to a function, will convert all of that function’s arguments into NumPy arrays prior to invoking the function. array_args considers pint.Quantity objects whose magnitudes are arrays to be arrays and will convert arguments that are quantitites whose magnitudes are not arrays into new quantities with array magnitudes.

If a function is decorated with @array_args('arg1', 'arg2' ...) then only the arguments whose names are given (arg1, arg2, …) are converted into arrays.

assoc(d, /, *args, **kwargs)

Returns a copy of the given dictionary with additional key-value pairs.

assoc(d, key, val) returns a copy of the dictionary d with the given key-value pair associated in the new copy. The return value is always the same type as the argument d but is always an updated copy. The argument d is never mutated.

assoc(d, key1, val1, key2, val2 ...) associates all the given keys to the given values in the returned copy.

assoc(d, key1=val1, key2=val2 ...) uses the keyword arguments as the arguments that are to be associated. These may be mixed with positional key-value pairs.

assoc(d) returns a copy of d.

Parameters:

• d (dict-like) – A dictionary that is to be copied and updated with the following arguments.

• args – Sequential pairs of keys and values (i.e., len(args) must be even) that should be updated in the returned dictionary.

• kwargs – Additional key-value pairs to be updated in the returned dictionary.

Returns:

A copy of d with updated keys and values.

Return type:

dict-like

azpath(obj, *args, **kwargs)

Creates and returns an AzureBlobPath representing an Azure repository.

azpath(p) creates and returns an AzureBlobPath object, which is a type of cloudpathlib.CloudPath object, from the path or path-string p. If p is an AzureBlobPath, then it is returned as-is. Otherwise pathstr(p) is converted into an AzureBlobPath; pathstr(p) may start with 'az://' (not case-sensitive) or, if it does not have a scheme specifier, 'az://' will be prepended to it.

azpath(p, a1, a2...) converts p into an AzureBlobPath then joins the a1, a2, etc. values to the end of the path and returns the joined path.

The azpath function accepts all the optional arguments of the AzureBlobClient type from cloudpathlib as well as the client option. If the client option is given along with additional optional arguments, then the optional arguments are ignored.

Additionally, azpath parses the option cache_path, which is not normally accepted by AzureBlobPath, which instead requires the option local_cache_dir. Any time that a local_cache_dir is given, it overrides the cache_path; however, if local_cache_dir is not given and cache_path is, then the directory os.path.join(cache_path,"az") is given as the local_cache_dir option.

class calc(*args, name=None, lazy=True, lrucache=0, pathcache=None, indent=None)

Decorator type that represents a single calculation in a calc-plan.

The calc class encapsulates data regarding the calculation of a single set of output values from a separate set of input values: a calculation component that can be fit together with other such components to make a calculation plan.

@calc by itself can be used as a decorator to indicate that the function that follows is a calculation component; calculation components can be combined to form plan objects, which can encapsulate a flexible workflow of Python computations. When @calc is used as a decorator by itself, then the calc is considered to have a single output value whose name is the same as that of the function it decorates.

@calc(names...) accepts a string or strings that name the output values of the calc function. In this case, the decorated function must return either a tuple of thes values in the order they are given or a dictionary in which the keys are the same as the given names.

@calc(None) is a special instance which indicates that the lazy argument is to be ignored (it is forced to be False), no output values are to be produced by the function, and the calculation must always run when the input parameters are updated.

The calc class parses its inputs and outputs through the immlib.docwrap function in order to collect documentation (see the input_docs and output_docs attributes, below). The 'Inputs' and 'Outputs' sections are tracked as the documentation of the parameters, and are required to be formatted using [NumPy’s documentation style](https://numpydoc.readthedocs.io/en/latest/format.html) in order for the parameter documentation to be properly extracted. Users of calculation objects should decorate their functions using docwrap manually themselves, however (if desired), because decorating a function with calc alone does not cause the function’s documentation to be available to other functions that use @docwrap to format their docstrings.

Caching for calculations requires some care. First, the calc- and plan-based workflow system in immlib is designed to work best with calc objects that are pure functions. A function f(*args, **kw) is pure if it has no side-effects and if f(*args1, **kw1) == f(*args2, **kw2) is true whenever args1 == args2 and kw1 == kw2. That is, f always produces the same outputs when given the same inputs. Plans that contain unpure functions can work fine in many contexts, but unpure calc objects will break caching because the return value of a cached unpure calculation will always be the same value. (In other words, the value that is calculated and cached by the function the first time it is called.)

Second, the calc type has an option, pathcache, which can be set to an explicit path to which all calculations run by the created calc object will be cached and later uncached if re-requested. This is occasionally appropriate for a particular compute environment, but a better approach is typically to grant control of caching and cache paths to the user who creates the plandict object downstream of the creation of the calc objects. To enable this behavior, one should instead use the option pathcache=True, which enables caching of calculations to a specific cache path when provided by the user during the creation of the plandict (the default is False, which disables path caching for the calculation).

Parameters:

• outputs (strings) – The positional arguments to @calc() provide the names of the output variables. The names must all be valid variable names (see immlib.strisvar).

• name (None or str, optional) – The name of the function. The default, None, uses fn.__name__.

• lazy (bool, optional) – Whether the calculation unit should be calculated lazily (True) or eagerly (False) when a plandict is created. The default is True.

• lrucache (int, optional) – The number of recently calculated results to cache. If this value is 0, then no memoization is done (the default). If lrucache is an integer greater than 0, then an LRU cache is used with a maximum size of lrucache. If lrucache is inf, then all values are cached indefinitely. Note that this cache is performed at the level of the calculation using Python’s functools caching decorators.

• pathcache (None, bool, or path-like, optional) – If pathcache is a path-like object (typically a pathlib.Path orstring) that references a directory, then the results are cached in files in the given directory whenever possible. The pathcache option may also a 2-tuple containing a path followed by options to the joblib.Memory constructor; see immlib.util.to_pathcache for more information.

• indent (int or None, optional) – The indentation level of the function’s docstring. The default is None, which indicates that the indentation level should be deduced from the docstring itself.

name

The name of the calculation function.

Type:

str

base_function

The original function, prior to decoration for caching.

Type:

callable

lrucache

The in-memory cache being used. If this value is None``then no in-memory cache is being used. If it is an integer, this indicates the number of least recently used objects being stored in the cache. Otherwise, ``lrucache will be a function used to wrap the base_function of the calculation for caching. The lrucache parameter is filtered by the immlib.util.to_lrucache function in order to convert it into a valid functools.lru_cache object.

Type:

None or lrucache-like

pathcache

The file-system-based cache being used. If this value is None or False, then no filesystem cache is being used by the calculation directly. If this value is a path object, then that path is the directory in which cache files are saved/loaded. If pathcache is a joblib.Memory object, then this object handles the caching for the calculation. Otherwise, the value will be True, indicating that caching should be performed automatically using the cache_path input to the calc. If cache_path was not already one of the inputs, it is added as an input with the default value None. When automatic caching is performed, the cache_path is automatically converted into a joblib.Memory object using the immlib.util.to_pathcache function.

Type:

None or pathcache-like

function

The function itself.

Type:

callable

signature

The signature of fn, as returned from inspect.signature(fn).

Type:

inspect.Signature

inputs

The names of the input parameters for the calculation.

Type:

pcollections.pset of str

outputs

The names of the output values of the calculation.

Type:

tuple of str

defaults

A persistent dictionary whose keys are input parameter names and whose values are the default values for the associated parameters.

Type:

pcollections.pdict

lazy

Whether the calculation is intended as a lazy (True) or eager (False) calculation.

Type:

bool

input_docs

A pdict object whose keys are input names and whose values are the documentation for the associated input parameters.

Type:

pcollections.pdict

output_docs

A pdict object whose keys are output names and whose values are the documentation for the associated output values.

Type:

pcollections.pdict

call(*args, **kwargs)

Calls the calculation and returns the results dictionary.

c.call(...) is an alias for c(...).

See also calc.mapcall, calc.eager_call, and calc.lazy_call.

eager_call(*args, **kwargs)

Eagerly calls the given calculation using the arguments.

c.eager_call(...) returns the result of calling the calculation c(...) directly. Using the eager_call method is different from calling the __call__ method only in that the eager_call method ignores the lazy member and always returns the direct results of calling the calculation; using the __call__ method will result in eager_call being run if the calculation is not lazy and in lazy_call being run if the calculation is lazy.

See also

calc.eager_mapcall, calc.lazy_call, calc.lazy_mapcall

eager_mapcall(*args, **kwargs)

Calls the given calculation using the parameters in mappings.

c.eager_mapcall(map1, map2..., key1=val1, key2=val2...) returns the result of calling the calculation c(...) using the parameters found in the provided mappings and key-value pairs. All arguments of mapcall are merged left-to-right using immlib.merge then passed to c.function as required by it.

lazy_call(*args, **kwargs)

Returns a lazy-dict of the results of calling the calculation.

calc.lazy_call(...) is equivalent to calc(...) except that the lazydict that it returns encapsulates the running of the calculation itself, so that calc(...) is not run until one of the lazy values is requested.

See also

calc.mapcall, calc.lazy_mapcall

lazy_mapcall(*args, **kwargs)

Calls the given calculation lazily using the parameters in mappings.

c.lazy_mapcall(map1, map2..., key1=val1, key2=val2...) returns the result of calling the calculation c(...) using the parameters found in the provided mappings and key-value pairs. All arguments of mapcall are merged left-to-right using immlib.merge then passed to c.function as required by it.

The only difference between calc.mapcall and calc.lazy_mapcall is that the lazydict returned by the latter method encapsulates the calling of the calculation itself, so no call to the calculation is made until one of the values of the lazydict is requested.

See also

calc.eager_mapcall, calc.lazy_call, calc.eager_call

mapcall(*args, **kwargs)

Calls the calculation and returns the results dictionary.

c.mapcall(map1, map2..., key1=val1, key2=val2...) returns the result of calling the calculation c(...) using the parameters found in the provided mappings and key-value pairs. All arguments of mapcall are merged left-to-right using immlib.merge then passed to c.function as required by it.

See also

calc.lazy_mapcall, calc.eager_mapcall, calc.call

rename_keys(*args, **kwargs)

Returns a copy of the calculation with inputs and outputs renamed.

calc.rename_keys(...) returns a copy of calc in which the input and output values of the function have been translated. The translation is found from merging the list of 0 or more dict-like arguments given left-to-right followed by the keyword arguments into a single dictionary. The keys of this dictionary are translated into their associated values in the returned dictionary.

If any of the values of the merged dictionary are 2-tuples, then they are interpreted as (input_tr, output_tr). In this case, then the key must be associated with a name that appears in both the calculation’s input list and its output list, and the two names are translated differently.

update_function(fn)

Updates the function and its calc object.

On occasion, a function decorated with @calc is later decorated with another feature, such as a decorator that causes its inputs to be promoted. Such a decorator, when it comes after the @calc decorator (i.e., on a line prior to the @calc), will not update the calculation object and thus the calculation object, when invoked, will not call the fully decorated version of its function. To fix this, any calc object whose base_function member variable is identical to the function given to a plan object (i.e., f is not to_calc(f).base_function), then this method is called to return a calc object whose base_function has been updated. If possible, it also updates the fn argument to use the new calc object.

In general, this function should not be called directly by the user; rather, it gets run automatically when a calc is added to a new plan or planobject.

with_lrucache(new_cache)

Returns a copy of a calc with a different in-memory cache strategy.

with_pathcache(new_path)

Returns a copy of a calc with a different cache directory.

can_hash(obj)

Returns True if obj is safe to hash and False otherwise.

can_hash(obj) is equivalent to hashsafe(obj) is not None. This differs from is_ahashable(obj) in that is_ahashable only checks whether obj is an instance of Hashable while hashsafe(obj) attempts to hash obj and returns None when a TypeError is raised.

Note

A fairly reliable test of whether an object is immutable or not in Python is whether it can be hashed.

See also

hashsafe, is_ahashable

can_iter(obj)

Returns True if obj is safe to iterate and False otherwise.

can_iter(obj) is equivalent to itersafe(obj) is not None. This differs from is_aiterable(obj) in that is_aiterable only checks whether obj is an instance of Iterable; itersafe tries to run iter(obj) and returns None when a TypeError is raised.

See also

itersafe, is_aiterable

class default_docproc(docproc)

Context manager for setting the default immlib.docproc document processing object.

The following code-block can be used to evaluate the code represented by ... using the docrep.DocstringProcessor object docproc as the default immlib.docproc processor:

with immlib.default_docproc(docproc):
    ...

If the immlib.docproc value has accidentally been corrupted, then it can be reset using the following:

immlib.default_docproc.reset()

Parameters:

docproc (docrep.DocstringProcessor object) – The docstring processing object to use as the default in immlib in the contextualized code.

See also

docwrap

decorator that simplifies the use of the docrep library.

static reset()

Resets the value of immlib.docproc to its value when the immlib library was originally loaded.

class default_ureg(ureg)

Context manager for setting the default immlib unit registry.

The following code-block can be used to evaluate the code represented by ... using the unit-registry ureg as the default immlib.units registry:

with immlib.default_ureg(ureg):
    ...

dictmap(f, keys, /, *args, **kw)

Returns a dict with the given keys and the values map(f, keys).

dictmap(f, keys) returns a dict object whose keys are the elements of iter(keys) and whose values are the elements of map(f, keys).

dictmap(f, keys, *args, **kw) returns a dict object whose keys are the elements of iter(keys) and whose values are the elements of [f(k, *args, **kw) for k in iter(keys)].

Parameters:

• f (function) – The function used to create the values in the new dictionary; it must accept one arguments (f(k)) plus any additional arguments provided in *args and **kwargs.

• keys (iterable) – An iterable object whose values are to become the keys of the new dictionary.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

A dictionary of the given keys with each key k mapped to f(k).

Return type:

dict

dissoc(d, /, *args)

Returns a copy of the given dictionary with certain keys removed.

dissoc(d, key) returns a copy of the dictionary d with the given key disssociated in the new copy. The return value is always the same type as the argument d.

dissoc(d, key1, key2 ...) dissociates all the given keys from their values in the returned copy.

dissoc(d) returns a copy of d.

Parameters:

• d (dict-like) – A dictionary that is to be copied and updated according to the following arguments.

• args – Keys that should be removed from the copy of d that is returned.

Returns:

A copy of d with the given keys removed.

Return type:

dict-like

docwrap(f=None, /, *, indent=None, proc=Ellipsis)

Applies standard doc-string processing to the decorated function.

The immlib.docwrap decorator applies a standard set of pre-processing to the docstring of the function that follows it. This processing amounts to using the docrep module’s DocstringProcessor as a filter on the documentation of the function. The function’s documentation is always placed in the base-name equal to its fully-qualified namespace name.

When called as @docwrap(name) for a string name, the documentation for the decorated function is instead placed under the base-name name.

Parameters:

• f (function or str or None, optional) –

  The function to be decorated, when @docwrap is used alone as a decorator, or when used as a decorator with only the other options given, such as @docwrap(indent=8). If a string is given, as in @docwrap('immlib.dictmap') then the given string is used as the function’s name instead of its __module__ plus its __name__. This is mostly useful when using @docwrap with a function defined in a private submodule; for example immlib.dictmap is defined in immlib.util._core but is imported into a reclaimed by the immlib core namespace, so it is typically considered to belong to that namespace.

  Typically this argument does not need to be provided as it is given after the decorator line; the exception to this is when a string is given.

• indent (None or int, optional) – The number of spaces that are used as indentation before lines in the docstring. This is mostly useful when decorated functions appear in indented contexts and thus the default indentation of 4 is inappropriate. If None is given, then the decorator finds the non-empty line, not including the first line, with the smallest indentation and uses that.

• proc (docrep.DocstringProcessor or Ellipsis, optional) – The proc option provides the document processor object from the docrep library that should be used to process the decorated object. Because these objects can be specifically configured to enable different docstring formats, this option is provided to the user. The default value is Ellipsis, in which case the immlib.docproc object is used. The docproc object has been configured to work with the Input and Output sections that are used with calculations and plans. The immlib.with_docproc function can be used to change the immlib.docproc object that is used in a local code-block.

Returns:

The decorated function or object, after its docstring has been parsed.

Return type:

object

See also

default_docproc

Run a code-block with a specific default docstring processor.

filepath(p, *args)

Returns a local Path object for the given path if possible.

The filepath function is intended to coerce remote paths (such as the S3 or OSF paths managed through the cloudpathlib.CloudPath class) into paths representing their local caches. If a local file is requested, then it is always downloaded before the Path is returned. For directories, the cache directory itself will always exist, but no such guarantee is made about its contents.

If the argument to filepath is a string and not a path object, then it is converted into a path via the immlib.path function.

Parameters:

p (path-like) – The object whose local cache path is to be returned after it has been downloaded.

Returns:

If the input path p is already a local path, then it is returned. If p is a remote path, then it is downloaded and its cache path is returned. If there is no local cache or if the file does not exist, an error is raised.

Return type:

Path

freezearray(arr)

Freezes a NumPy array or SciPy sparse array in-place.

freezearray(x) sets the 'WRITEABLE' bit on the numpy array x or on x.data if x is a SciPy sparse array. If x is neither a NumPy array nor a SciPy sparse array, then a TypeError is raised. No value is returned.

freezearray(q) is equivalent to freezearray(q.m) if q is a pint.Quantity object.

Warning

This function mutates its argument in-place.

See also

frozenarray

frozenarray(obj, /, dtype=None, *, copy=False, **kwargs)

Roughly equivalent to numpy.array but returns read-only arrays.

frozenarray(obj) is equivalent to numpy.array(obj) with a small number of exceptions:

• Primarily, the returned object is always a frozen array (i.e., an array with the 'WRITEABLE' flag set to False).

• The default value of the copy option is False, meaning that a copy of the array will only be made if required by the other parameters or if the array is not already read-only. If you wish to make an array read-only rather than obtaining a read-only copy of it, use the freezearray() function.

• SciPy sparse arrays are also handled by setting the write flag on the obj.data member.

• If obj is a pint.Quantity object, then an equivalent quantity with the magnitude made read-only is returned.

If a PyTorch tensor is passed to frozenarray, it will be converted into a frozen NumPy array; PyTorch tensors themselves cannot be frozen, however.

See also

numpy.array

Create an array that is not frozen.

freezearray

Convert an argument to a frozen array in-place.

get(d, k, /, *args, **kwargs)

Returns a value from either a mapping or a sequence.

The get function is essentially a function version of the get method that works for both Mapping and Sequence types (e.g., dict, list, tuple, and related types that implement their abstract bases).

get(d, k) extracts element k from object d and returns it. If k is not a valid index of d (i.e., k is not a key of d, if d is a mapping, or is not an integer index of d if d is a sequence), then the optional value default is returned. If detault is not explicitly provided, then an error is raised. Note that if a non-integer key is provided for a sequence, this is treated as a missing index.

Note

The default value may be expressed as either a third positional argument or a named argument (default).

Parameters:

• d (object) – The dict-like or list-like object from which an element is being extracted.

• k (object) – The key or index into d whose value is to be extracted.

• args – The default value to be returned if an item is not found. The default value may be specified as a third positional argument.

• kwargs – The default value to be returned if an item is not found. The default value may be specified as a named argument with the name "default".

Returns:

The object d[k], if the key k is found in the collection d. Otherwise, default is returned.

Return type:

object

Raises:

KeyError – If the key or index k is not found in the collection d and no default option is given.

gspath(obj, *args, **kwargs)

Creates and returns an GSPath representing a Google Storage repository.

gspath(p) creates and returns a GSPath object, which is a type of cloudpathlib.CloudPath object, from the path or path-string p. If p is a GSPath, then it is returned as-is. Otherwise pathstr(p) is converted into an GSPath; pathstr(p) may start with 'gs://' (not case-sensitive) or, if it does not have a scheme specifier, 'gs://' will be prepended to it.

gspath(p, a1, a2...) converts p into an GSPath then joins the a1, a2, etc. values to the end of the path and returns the joined path.

The gspath function accepts all the optional arguments of the GSClient type from cloudpathlib as well as the client option. If the client option is given along with additional optional arguments, then the optional arguments are ignored.

Additionally, gspath parses the option cache_path, which is not normally accepted by GSPath, which instead requires the option local_cache_dir. Any time that a local_cache_dir is given, it overrides the cache_path; however, if local_cache_dir is not given and cache_path is, then the directory os.path.join(cache_path, "gs") is given as the local_cache_dir option.

hashsafe(obj)

Returns hash(obj) if obj is hashable, otherwise returns None.

This function attempts to hash an object and returns None when doing so raises a TypeError.

Note

A fairly reliable test of whether an object is immutable or not in Python is whether it can be hashed.

Parameters:

obj (object) – The object to be hashed.

Returns:

If the object is hashable, returns the hashcode; otherwise, returns None.

Return type:

int or None

See also

can_hash, is_ahashable

is_abytes(obj)

Returns True if an object is a byte-string, otherwise False.

is_abytes(obj) returns True if the given object obj is an instance of the abstract collections.abc.ByteString type.

Parameters:

obj (object) – The object whose quality as an ByteString object is to be assessed.

Returns:

True if obj is an instance of ByteString, otherwise False.

Return type:

bool

is_acoll(obj)

Returns True if an object is a collection (a sized iterable container).

is_acoll(obj) returns True if the given object obj is an instance of the abstract collections.abc.Collection type.

Parameters:

obj (object) – The object whose quality as an Collection object is to be assessed.

Returns:

True if obj is an instance of Collection, otherwise False.

Return type:

bool

is_acontainer(obj)

Returns True if an object implements __contains__, otherwise False.

is_acontainer(obj) returns True if the given object obj is an instance of the abstract collections.abc.Container type.

Parameters:

obj (object) – The object whose quality as a Container object is to be assessed.

Returns:

True if obj is an instance of Container, otherwise False.

Return type:

bool

is_ahashable(obj)

Returns True if an object is a hashable object, otherwise False.

is_ahashable(obj) returns True if the given object obj is an instance of the abstract collections.abc.Hashable type. This differs from the can_hash function, which checks whehter calling hash on an object raises an exception.

Parameters:

obj (object) – The object whose quality as an Hashable object is to be assessed.

Returns:

True if obj is an instance of Hashable, otherwise False.

Return type:

boolean

See also

can_hash

is_aiterable(obj)

Returns True if an object implements __iter__, otherwise False.

is_aiterable(obj) returns True if the given object obj is an instance of the abstract collections.abc.Iterable type.

Parameters:

obj (object) – The object whose quality as an Iterable object is to be assessed.

Returns:

True if obj is an instance of Iterable, otherwise False.

Return type:

bool

is_aiterator(obj)

Returns True if an object is an instance of collections.abc.Iterator.

is_aiterable(obj) returns True if the given object obj is an instance of the abstract collections.abc.Iterator type.

Parameters:

obj (object) – The object whose quality as an Iterator object is to be assessed.

Returns:

True if obj is an instance of Iterator, otherwise False.

Return type:

bool

is_amap(obj)

Returns True if an object is an abstract mapping, otherwise False.

is_amap(obj) returns True if the given object obj is an instance of the abstract collections.abc.Mapping type.

Parameters:

obj (object) – The object whose quality as an Mapping object is to be assessed.

Returns:

True if obj is an instance of Mapping, otherwise False.

Return type:

bool

is_ammap(obj)

Returns True if an object is a mutable mapping, otherwise False.

is_ammap(obj) returns True if the given object obj is an instance of the abstract collections.abc.MutableMapping type.

Parameters:

obj (object) – The object whose quality as an MutableMapping object is to be assessed.

Returns:

True if obj is an instance of MutableMapping, otherwise False.

Return type:

bool

is_amseq(obj)

Returns True if an object is a mutable sequence, otherwise False.

is_amseq(obj) returns True if the given object obj is an instance of the abstract collections.abc.MutableSequence type.

Parameters:

obj (object) – The object whose quality as an MutableSequence object is to be assessed.

Returns:

True if obj is an instance of MutableSequence, otherwise False.

Return type:

bool

is_amset(obj)

Returns True if an object is a mutable set, otherwise False.

is_amset(obj) returns True if the given object obj is an instance of the abstract collections.abc.MutableSet type.

Parameters:

obj (object) – The object whose quality as an MutableSet object is to be assessed.

Returns:

True if obj is an instance of MutableSet, otherwise False.

Return type:

bool

is_apmap(obj)

Returns True if an object is a persistent mapping, otherwise False.

is_apmap(obj) returns True if the given object obj is an instance of the abstract pcollections.abc.PersistentMapping type.

Parameters:

obj (object) – The object whose quality as an PersistentMapping object is to be assessed.

Returns:

True if obj is an instance of PersistentMapping, otherwise False.

Return type:

bool

is_apseq(obj)

Returns True if an object is a persistent sequence, otherwise False.

is_apseq(obj) returns True if the given object obj is an instance of the abstract pcollections.abc.PersistentSequence type.

Parameters:

obj (object) – The object whose quality as an PersistentSequence object is to be assessed.

Returns:

True if obj is an instance of PersistentSequence, otherwise False.

Return type:

bool

is_apset(obj)

Returns True if an object is a persistent set, otherwise False.

is_apset(obj) returns True if the given object obj is an instance of the abstract pcollections.abc.PersistentSet type.

Parameters:

obj (object) – The object whose quality as an PersistentSet object is to be assessed.

Returns:

True if obj is an instance of PersistentSet, otherwise False.

Return type:

bool

is_areversible(obj)

Returns True if an object is an instance of Reversible.

is_areversible(obj) returns True if the given object obj is an instance of the abstract collections.abc.Reversible type.

Parameters:

obj (object) – The object whose quality as an Reversible object is to be assessed.

Returns:

True if obj is an instance of Reversible, otherwise False.

Return type:

bool

is_array(obj, /, *, dtype=None, shape=None, ndim=None, numel=None, frozen=None, sparse=None, quant=None, unit=Ellipsis, ureg=None)

Returns True if an object is a numpy.ndarray object, otherwise returns False.

is_array(obj) returns True if the given object obj is an instance of the numpy.ndarray class or is a scipy.sparse array, or if obj is a pint.Quantity object whose magnitude is one of these. Additional constraints may be placed on the object via the optional argments.

Note that to immlib, both numpy.ndarray arrays and scipy.sparse arrays are considered “arrays”. This behavior can be changed with the sparse parameter.

Parameters:

• obj (object) – The object whose quality as a NumPy array object is to be assessed.

• dtype (dtype-like or None, optional) – The NumPy dtype that is required of the obj in order to be considered a valid ndarray. The obj.dtype matches the given dtype parameter if either dtype is None (the default) or if obj.dtype is a sub-dtype of dtype according to numpy.issubdtype. Alternately, dtype can be a tuple, in which case, obj is considered valid if its dtype is any of the dtypes in dtype. Note that in the case of a tuple, the dtype of obj must appear exactly in the tuple rather than be a subtype of one of the objects in the tuple.

• ndim (int, tuple or ints, or None, optional) – The number of dimensions that the object must have in order to be considered a valid numpy array. If None, then any number of dimensions is acceptable (this is the default). If this is an integer, then the number of dimensions must be exactly that integer. If this is a list or tuple of integers, then the dimensionality must be one of these numbers.

• shape (int, tuple of ints, or None, optional) – If the shape parameter is not None, then the given obj must have a shape that matches the parameter value. The value shape must be a tuple that is equal to the obj’s shape tuple with the following additional rules: a -1 value in the shape tuple will match any value in the obj’s shape tuple, and a single Ellipsis may appear in shape, which matches any number of values in the obj’s shape tuple. The default value of None indicates that no restriction should be applied to the obj’s shape.

• numel (int, tuple of ints, or None, optional) – If the numel parameter is not None, then the given obj must have the same number of elements as given by numel. If numel is a tuple, then the number of elements in obj must be in the numel tuple. The number of elements is the product of its shape.

• frozen (bool or None, optional) – If None, then no restrictions are placed on the 'WRITEABLE' flag of obj. If True, then the data in obj must be read-only in order for obj to be considered a valid array. If False, then the data in obj must not be read-only.

• sparse (boolean or False, optional) – If the sparse` parameter is None, then no requirements are placed on the sparsity of obj for it to be considered a valid array. If sparse is True or False, then obj must either be sparse or not be sparse, respectively, for obj to be considered valid. If sparse is a string, then it must be either 'coo', 'lil', 'csr', or 'csr', indicating the required sparse array type. Only scipy.sparse matrices are considered valid sparse arrays.

• quant (bool, optional) – Whether Quantity objects should be considered valid arrays or not. If quant=True then obj is considered a valid array only when obj is a quantity object with a numpy array as the magnitude. If False, then obj must be a numpy array itself and not a Quantity to be considered valid. If None (the default), then either quantities or numpy arrays are considered valid arrays.

• unit (unit-like, Ellipsis, or None, optional) – A unit with which the object obj’s unit must be compatible in order for obj to be considered a valid array. An obj that is not a quantity is considered to have a unit of None, which is not the same as being a quantity with a dimensionless unit. In other words, is_array(array, quant=None) will return True for a numpy array while is_array(arary, quant='dimensionless') will return False. If unit=Ellipsis (the default), then the object’s unit is ignored.

• ureg (pint.UnitRegistry, None, or Ellipsis, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then the registry of obj is used if obj is a quantity, and immlib.units is used if not.

Returns:

True if obj is a valid numpy array, otherwise False.

Return type:

bool

See also

is_tensor, is_numeric

is_aseq(obj)

Returns True if an object is a sequence, otherwise False.

is_aseq(obj) returns True if the given object obj is an instance of the abstract collections.abc.Sequence type.

Parameters:

obj (object) – The object whose quality as an Sequence object is to be assessed.

Returns:

True if obj is an instance of Sequence, otherwise False.

Return type:

bool

is_aset(obj)

Returns True if an object is a set type, otherwise False.

is_aset(obj) returns True if the given object obj is an instance of the abstract collections.abc.Set type.

Parameters:

obj (object) – The object whose quality as an Set object is to be assessed.

Returns:

True if obj is an instance of Set, otherwise False.

Return type:

bool

is_asized(obj)

Returns True if an object implements len(), otherwise False.

is_asized(obj) returns True if the given object obj is an instance of the abstract collections.abc.Sized type.

Parameters:

obj (object) – The object whose quality as a Sized object is to be assessed.

Returns:

True if obj is an instance of Sized, otherwise False.

Return type:

bool

is_azpath(obj)

Detects whether the input is an AzureBlobPath object.

is_azpath(obj) returns True if obj is an instance of the AzureBlobPath class and False otherwise.

See also: like_azpath

Parameters:

obj (object) – The object whose membership in the AzureBlobPath class is to be determined.

Returns:

True if obj is an instance of AzureBlobPath and False otherwise.

Return type:

boolean

is_bool(obj, /)

Determines whether the argument is a scalar boolean or not.

is_bool(obj) returns True if obj is a scalar boolean and False otherwise.

See also

is_scalar, is_booldata

is_booldata(obj, /)

Returns True if an object is a boolean, otherwise False.

is_booldata(obj) returns True if the given object obj is an instance of the bool type or if it is an instance of a boolean NumPy array or PyTorch tensor.

Parameters:

obj (object) – The object whose quality as a bool object or boolean array or tensor is to be assessed.

Returns:

True if obj is an instance of bool or is a boolean array or tensor, otherwise False.

Return type:

boolean

See also

is_intdata, is_realdata, is_complexdata, is_numberdata

is_bytes(obj)

Returns True if an object is a bytes object, otherwise False.

is_bytes(obj) returns True if the given object obj is an instance of the bytes type and returns False otherwise.

Parameters:

obj (object) – The object whose quality as an bytes object is to be assessed.

Returns:

True if obj is an instance of bytes, otherwise False.

Return type:

bool

is_calcfn(obj, /)

Determines if an object is function that was decorated by @calc.

is_calcfn(obj) returns True if obj is a function that was decorated with an @calc decorator or if obj is a calc object, and it returns False otherwise.

Functions decorated with @calc are not changed but rather are given some metadata, which is stored in the member field calc. For such functions, this field contains an object of type calc.

See also

calc, to_calc, is_calc

is_complex(obj, /)

Determines whether the argument is a scalar complex number or not.

is_complex(obj) returns True if obj is a scalar complex number and False otherwise. Note that booleans, integers, and real numbers are all considered valid complex numbers.

See also

is_number, is_complexdata

is_complexdata(obj)

Returns True if an object is a complex number, otherwise False.

is_complexdata(obj) returns True if the given object obj is an instance of the numbers.Complex type or an instance of a complex-valued NumPy array or PyTorch tensor.

Parameters:

obj (object) – The object whose quality as a Complex object is to be assessed.

Returns:

True if obj is an instance of Complex, otherwise False.

Return type:

boolean

See also

is_booldata, is_intdata, is_realdata, is_numberdata

is_ddict(obj)

Returns True if an object is a defaultdict object.

is_ddict(obj) returns True if the given object obj is an instance of the collections.defaultdict type.

Parameters:

obj (object) – The object whose quality as a defaultdict object is to be assessed.

Returns:

True if obj is an instance of defaultdict, otherwise False.

Return type:

bool

is_dense(obj, /, dtype=None, *, shape=None, ndim=None, numel=None, quant=None, ureg=None, unit=Ellipsis)

Returns True if an object is a dense NumPy array or PyTorch tensor.

is_dense(obj) returns True if the given object obj is an instance of one of the NumPy ndarray classes, is a dense PyTorch tensor, or is a quantity whose magnintude is one of theese. Additional constraints may be placed on the object via the optional argments.

Parameters:

• obj (object) – The object whose quality as a dense numerical object is to be assessed.

• dtype (dtype-like or None, optional) – The NumPy or PyTorch dtype or dtype-like object that is required to match that of the obj in order to be considered valid. The obj.dtype matches the given dtype parameter if either dtype is None (the default) or if obj.dtype is equivalent to dtype. Alternately, dtype can be a tuple, in which case, obj is considered valid if its dtype is any of the dtypes in dtype.

• ndim (int, tuple or ints, or None, optional) – The number of dimensions that the object must have in order to be considered valid. If ndim is None, then any number of dimensions is acceptable (this is the default). If it is an integer, then the number of dimensions must be exactly that integer. If this is a list or tuple of integers, then the dimensionality must be one of the listedn numbers.

• shape (int, tuple of ints, or None, optional) – If the shape parameter is not None, then the given obj must have a shape that matches the parameter value. The value of shape must be a tuple that is equal to the shape of obj with the following additional rules: a -1 value in the shape tuple will match any value in the shape of obj, and a single Ellipsis may appear in shape, which matches any number of values in the shape tuple of obj. The default value of None indicates that no restriction should be applied to the shape of obj.

• numel (int, tuple of ints, or None, optional) – If the numel parameter is not None, then the given obj must have the same number of elements as given by numel. If numel is a tuple, then the number of elements in obj must be in the numel tuple. The number of elements is the product of its shape.

• quant (bool, optional) – Whether pint.Quantity objects should be considered valid or not. If quant=True then obj is considered a valid numerical object only when obj is a quantity object with a valid numerical object as the magnitude. If quant=False, then obj must be a numerical object itself and not a pint.Quantity to be considered valid. If quant=None (the default), then either quantities or numerical objects are considered valid.

• ureg (UnitRegistry, None, or Ellipsis, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then the registry of obj is used if obj is a quantity, and immlib.units is used if not.

• unit (unit-like or Ellipsis, optional) – A unit with which the unit of obj must be compatible in order for obj to be considered a valid numerical object. An obj that is not a quantity is considered to have dimensionless units. If unit=Ellipsis (the default), then the object’s unit is ignored.

Returns:

True if obj is a valid dense numerical object, otherwise False.

Return type:

bool

is_dict(obj)

Returns True if an object is a dict object.

is_dict(obj) returns True if the given object obj is an instance of the dict type.

Parameters:

obj (object) – The object whose quality as an dict object is to be assessed.

Returns:

True if obj is an instance of dict, otherwise False.

Return type:

bool

is_filepath(p)

Detects whether an object is a filesystem Path object.

Any object that inherits from the Path type is considered a filesystem path. File-paths correspond to URLs that begin with 'file://'.

Parameters:

p (path-like) – The object whose quality as a path is to be assessed.

Returns:

True if p is an instance of the Path type and False otherwise.

Return type:

boolean

is_frozenset(obj)

Returns True if an object is a frozenset object.

is_frozenset(obj) returns True if the given object obj is an instance of the frozenset type.

Parameters:

obj (object) – The object whose quality as an frozenset object is to be assessed.

Returns:

True if obj is an instance of frozenset, otherwise False.

Return type:

bool

See also

is_set, is_aset, is_amset, is_apset, is_pset, is_tset

is_gspath(obj)

Detects whether the input is an GSPath object.

is_gspath(obj) returns True if obj is an instance of the GSPath class and False otherwise.

See also: like_gspath

Parameters:

obj (object) – The object whose membership in the GSPath class is to be determined.

Returns:

True if obj is an instance of GSPath and False otherwise.

Return type:

boolean

is_intdata(obj, /)

Returns True if an object is a Python integer, otherwise False.

is_intdata(obj) returns True if the given object obj is an instance of the numbers.Integral type or if it is an instance of a numeric NumPy array or PyTorch tensor whose dtype is an integer type.

Parameters:

obj (object) – The object whose quality as a Integral object or integer-valued array or tensor is to be assessed.

Returns:

True if obj is an instance of Integral or is an integer numpy array, otherwise False.

Return type:

boolean

See also

is_booldata, is_realdata, is_complexdata, is_numberdata

is_integer(obj, /)

Determines whether the argument is a scalar integer or not.

is_integer(obj) returns True if obj is a scalar integer and False otherwise. Note that booleans are considered integers.

See also

is_scalar, is_intdata

is_lambda(obj)

Returns True if an object is a lambda function, otherwise False.

is_lambda(obj) returns True if the given object obj is an instance of the types.LambdaType type.

Parameters:

obj (object) – The object whose quality as a LambdaType object is to be assessed.

Returns:

True if obj is an instance of LambdaType, otherwise False.

Return type:

bool

is_ldict(obj)

Returns True if an object is a persistent lazy dictionary object.

is_ldict(obj) returns True if the given object obj is an instance of the pcollections.ldict type and False otherwise.

Parameters:

obj (object) – The object whose quality as a ldict object is to be assessed.

Returns:

True if obj is an instance of ldict, otherwise False.

Return type:

bool

is_list(obj)

Returns True if an object is a list object.

is_list(obj) returns True if the given object obj is an instance of the list type.

Parameters:

obj (object) – The object whose quality as an list object is to be assessed.

Returns:

True if obj is an instance of list, otherwise False.

Return type:

bool

is_llist(obj)

Returns True if an object is a persistent lazy list object.

is_llist(obj) returns True if the given object obj is an instance of the pcollections.llist type and False otherwise.

Parameters:

obj (object) – The object whose quality as a llist object is to be assessed.

Returns:

True if obj is an instance of llist, otherwise False.

Return type:

bool

is_mcoll(obj)

Returns True if an object is a mutable list, set, or dict.

is_mcoll(obj) returns True if the given object obj is an instance of the dict, set, or list types, all of which are mutable collections. Otherwise, False is returned.

Parameters:

obj (object) – The object whose quality as a mutable collection is to be assessed.

Returns:

True if obj is a list, set, or dict and False otherwise.

Return type:

bool

is_number(obj, /, dtype=None)

Determines whether the argument is a scalar number or not.

is_number(obj) returns True if obj is a scalar number and False otherwise. The following are considered scalar numbers:

• Any instances of numbers.Number,

• Any numpy array x whose shape is () such that x.item() is a scalar.

Parameters:

• obj (object) – The object whose quality as a scalar number is to be tested.

• dtype (bool, int, float, complex, or None, optional) – The type of the scalar. If this is None (the default)``, then the type of the scalar must be a number but it needn’t be any particular number. Otherwise, it must match the given type.

Returns:

True if obj is a scalar number value and False otherwise.

Return type:

bool

See also

like_number, is_numberdata, is_numeric, is_bool, is_integer, is_real, is_complex

is_numberdata(obj, /)

Returns True if an object is a Python number, otherwise False.

is_numberdata(obj) returns True if the given object obj is an instance of the numbers.Number type or if it is an instance of a numeric NumPy array or PyTorch tensor.

Except in special cases, is_numberdata(x) is equivalent to is_complexdata(x).

is_numberdata is related to the function is_numeric: if is_numeric(x) is True then is_numberdata(x) is also True. However, is_numberdata(10) is True while is_numeric(10) is not. is_numberdata is designed for determining whether an object represents numbers, whereas is_numeric is designed for querying the properties of NumPy arrays and PyTorch tensors such as their shapes and data types.

Parameters:

obj (object) – The object whose quality as a Number object or numerical array or tensor is to be assessed.

Returns:

True if obj is an instance of Number or is a numerical array or tensor, otherwise False.

Return type:

boolean

See also

is_booldata, is_intdata, is_realdata, is_complexdata

is_numeric(obj, /, dtype=None, *, shape=None, ndim=None, numel=None, sparse=None, quant=None, unit=Ellipsis, ureg=None)

Returns True if an object is a numerical collection type and False otherwise.

is_numeric(obj) returns True if the given object obj is an instance of the torch.Tensor class, the numpy.ndarray class, one one of the scipy.sparse array classes, or is a pint.Quantity object whose magnitude is an instance of one of these types. Additional constraints may be placed on the object via the optional argments.

Note

The is_numeric function is similar to the is_array and is_tensor functions butis agnostic about whether its argument is a PyTorch tensor, a NumPy array, or an object that can be converted into one of these types.

Parameters:

• obj (object) – The object whose quality as a numeric object is to be assessed.

• dtype (dtype-like or None, optional) – The NumPy or PyTorch dtype or dtype-like object that is required to match that of the obj in order to be considered valid. The obj.dtype matches the given dtype parameter if either dtype is None (the default) or if obj.dtype is equivalent to dtype. Alternately, dtype can be a tuple, in which case, obj is considered valid if its dtype is any of the dtypes in dtype.

• ndim (int, tuple or ints, or None, optional) – The number of dimensions that the object must have in order to be considered valid. If ndim is None, then any number of dimensions is acceptable (this is the default). If it is an integer, then the number of dimensions must be exactly that integer. If this is a list or tuple of integers, then the dimensionality must be one of the listedn numbers.

• shape (int, tuple of ints, or None, optional) – If the shape parameter is not None, then the given obj must have a shape that matches the parameter value. The value of shape must be a tuple that is equal to the shape of obj with the following additional rules: a -1 value in the shape tuple will match any value in the shape of obj, and a single Ellipsis may appear in shape, which matches any number of values in the shape tuple of obj. The default value of None indicates that no restriction should be applied to the shape of obj.

• sparse (bool or False, optional) – If the sparse parameter is None, then no requirements are placed on the sparsity of obj for it to be considered valid. If sparse is True or False, then obj must either be sparse or not be sparse, respectively, for obj to be considered valid. If sparse is a string, then it must be a valid sparse array type that matches the type of obj for obj to be considered valid.

• numel (int, tuple of ints, or None, optional) – If the numel parameter is not None, then the given obj must have the same number of elements as given by numel. If numel is a tuple, then the number of elements in obj must be in the numel tuple. The number of elements is the product of its shape.

• quant (bool, optional) – Whether pint.Quantity objects should be considered valid or not. If quant=True then obj is considered a valid numerical object only when obj is a quantity object with a valid numerical object as the magnitude. If quant=False, then obj must be a numerical object itself and not a pint.Quantity to be considered valid. If quant=None (the default), then either quantities or numerical objects are considered valid.

• unit (unit-like or Ellipsis, optional) – A unit with which the unit of obj must be compatible in order for obj to be considered a valid numerical object. An obj that is not a quantity is considered to have dimensionless units. If unit=Ellipsis (the default), then the object’s unit is ignored.

• ureg (UnitRegistry, None, or Ellipsis, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then the registry of obj is used if obj is a quantity, and immlib.units is used if not.

Returns:

True if obj is a valid numerical object, otherwise False.

Return type:

bool

See also

is_array, is_tensor

is_odict(obj)

Returns True if an object is an OrderedDict object.

is_odict(obj) returns True if the given object obj is an instance of the collections.OrderedDict type.

Parameters:

obj (object) – The object whose quality as an OrderedDict object is to be assessed.

Returns:

True if obj is an instance of OrderedDict, otherwise False.

Return type:

bool

is_osfpath(obj)

Detects whether the input is an OSFPath object.

is_osfpath(obj) returns True if obj is an instance of the OSFPath class and False otherwise.

See also: like_osfpath

Parameters:

obj (object) – The object whose membership in the OSFPath class is to be determined.

Returns:

True if obj is an instance of OSFPath and False otherwise.

Return type:

boolean

is_path(p)

Detects whether an object is either a Path or a CloudPath object.

Both Path and CloudPath objects abstractly represent paths, but they do not share a subclass. is_path tests whether an object’s type is a subclass of any of path types recognized by immlib. Additional path types can be registered by adding immlib.paths.PathTypeRecord instances to the immlib.pathtypes dictionary. The key for such a record should be the string prefix for the path type (such as 's3' for an S3Path type).

Parameters:

p (path-like) – The object whose quality as a path is to be assessed.

Returns:

True if p has a type that is recognized by immlib as a path type and False otherwise.

Return type:

boolean

is_pcoll(obj)

Detects if an object is a plist, pset, pdict, llist or ldict.

is_pcoll(obj) returns True if the given object obj is an instance of the persistent collection types plist, pset, pdict, llist, or ldict. Otherwise, False is returned.

Note that this function tests against a specific set of concrete types; instances of objects whose types are subclasses of these types will be treated as instances of the base types; however other immutable types not defined in immlib or pcollections will not be recognized by this function.

Parameters:

obj (object) – The object whose quality as a persistent collection is to be assessed.

Returns:

True if obj is a persistent collection and False otherwise.

Return type:

bool

is_pdict(obj)

Returns True if an object is a persistent dictionary object.

is_pdict(obj) returns True if the given object obj is an instance of the pcollections.pdict type and False otherwise.

Note

The ldict type is a subtype of pdict, so for is_pdict(ldict()) returns True.

Parameters:

obj (object) – The object whose quality as a pdict object is to be assessed.

Returns:

True if obj is an instance of pdict, otherwise False.

Return type:

bool

is_plan(arg)

Determines if an object is a plan instance.

is_plan(x) returns True if x is a calculation plan and False otherwise.

is_plandict(arg)

Determines if an object is a plandict instance.

is_plandict(x) returns True if x is a plandict object and False otherwise.

is_planobject(obj)

Determines if an object is an instance of a immlib.plantype object.

is_planobject(obj) returns True if obj is an instance of a immlib.plantype class and False otherwise.

See also: plantype, is_plantype

is_plantype(obj)

Determines if an object is a immlib.plantype.

is_plantype(obj) returns True if obj is a immlib plantype class and False otherwise. Note that this works for the type but not instances of the type, for which you should use is_planobject.

See also: is_planobject, plantype

is_plist(obj)

Returns True if an object is a persistent list object.

is_plist(obj) returns True if the given object obj is an instance of the pcollections.plist type and False otherwise.

Parameters:

obj (object) – The object whose quality as a plist object is to be assessed.

Returns:

True if obj is an instance of plist, otherwise False.

Return type:

bool

is_pset(obj)

Returns True if an object is a persistent set object.

is_pset(obj) returns True if the given object obj is an instance of the pcollections.pset type and False otherwise.

Parameters:

obj (object) – The object whose quality as a pset object is to be assessed.

Returns:

True if obj is an instance of pset, otherwise False.

Return type:

bool

See also

is_set, is_aset, is_amset, is_apset, is_tset, is_frozenset

is_quant(obj, /, unit=Ellipsis, *, ureg=None)

Returns True if given a pint.Quantity object and False otherwise.

is_quant(obj) returns True if obj is a pint.Quantity object and False otherwise. The optional parameter unit may additionally specify a unit that obj must be compatible with.

Note

The parameter value unit=None type indicates a scalar without a unit (i.e., an object that is not a quantity), and so, while None is a valid value, this function will always return False when it is passed.

Parameters:

• obj (object) – The object whose quality as a pint.Quantity object is to be assessed.

• unit (unit-like or None, optional) – The unit that the object must have in order to be considered valid. This may be a pint.Unit or unit-name (see also immlib.unit), a list or tuple of such units/unit-names, or None. If Ellipsis is given (the default), then the object must be a pint.Quantity object, but it doesn’t matter what the unit of the object is. Otherwise, the object must have a unit equivalent to the unit or to one of the units given (unit may be a tuple of possible units). The pint.UnitRegistry objects for the units given via this parameter are ignored; only the ureg parameter influences the pint.UnitRegistry requirements.

• ureg (pint.UnitRegistry, Ellipsis, None, optional) – The pint.UnitRegistry object to use for units. If Ellipsis, then value immlib.units is used. If ureg is None (the default), then a specific unit registry is not checked.

Returns:

True if obj is a pint.Quantity whose unit is compatible with the requested unit and False otherwise.

Return type:

bool

Raises:

TypeError – If the ureg parameter is not a pint.UnitRegistry, Ellipsis, or None.

is_real(obj, /)

Determines whether the argument is a scalar real number or not.

is_real(obj) returns True if obj is a scalar real number and False otherwise. Note that booleans and integers are considered real numbers.

See also

is_scalar, is_realdata

is_realdata(obj, /)

Returns True if an object is a Python number, otherwise False.

is_realdata(obj) returns True if the given object obj is an instance of the numbers.Real type or of a real-valued NumPy array or PyTorch tensor.

Parameters:

obj (object) – The object whose quality as a Real object or real-values NumPy array ot PyTorch tensor is to be assessed.

Returns:

True if obj is an instance of Real or is a real-valued array or tensor, otherwise False.

Return type:

bool

See also

is_booldata, is_intdata, is_complexdata, is_numberdata

is_s3path(obj)

Detects whether the input is an S3Path object.

is_s3path(obj) returns True if obj is an instance of the S3Path class and False otherwise.

See also: like_s3path

Parameters:

obj (object) – The object whose membership in the S3Path class is to be determined.

Returns:

True if obj is an instance of S3Path and False otherwise.

Return type:

boolean

is_set(obj)

Returns True if an object is a set object.

is_set(obj) returns True if the given object obj is an instance of the set type. Note that this is not the same as is_aset which determines whether the object is of the collections.abc.Set abstract type.

Parameters:

obj (object) – The object whose quality as an set object is to be assessed.

Returns:

True if obj is an instance of set, otherwise False.

Return type:

bool

See also

is_aset, is_amset, is_apset, is_pset, is_tset, is_frozenset

is_sparse(obj, /, dtype=None, *, shape=None, ndim=None, numel=None, quant=None, ureg=None, unit=Ellipsis)

Returns True if an object is a sparse SciPy array or a sparse PyTorch tensor.

is_sparse(obj) returns True if the given object obj is an instance of one of the SciPy sparse array classes, is a sparse PyTorch tensor, or is a pint.Quantity whose magnintude is one of theese. Additional constraints may be placed on the object via the optional argments.

Parameters:

• obj (object) – The object whose quality as a sparse numerical object is to be assessed.

• dtype (dtype-like or None, optional) – The NumPy or PyTorch dtype or dtype-like object that is required to match that of the obj in order to be considered valid. The obj.dtype matches the given dtype parameter if either dtype is None (the default) or if obj.dtype is equivalent to dtype. Alternately, dtype can be a tuple, in which case, obj is considered valid if its dtype is any of the dtypes in dtype.

• ndim (int, tuple or ints, or None, optional) – The number of dimensions that the object must have in order to be considered valid. If ndim is None, then any number of dimensions is acceptable (this is the default). If it is an integer, then the number of dimensions must be exactly that integer. If this is a list or tuple of integers, then the dimensionality must be one of the listedn numbers.

• shape (int, tuple of ints, or None, optional) – If the shape parameter is not None, then the given obj must have a shape that matches the parameter value. The value of shape must be a tuple that is equal to the shape of obj with the following additional rules: a -1 value in the shape tuple will match any value in the shape of obj, and a single Ellipsis may appear in shape, which matches any number of values in the shape tuple of obj. The default value of None indicates that no restriction should be applied to the shape of obj.

• numel (int, tuple of ints, or None, optional) – If the numel parameter is not None, then the given obj must have the same number of elements as given by numel. If numel is a tuple, then the number of elements in obj must be in the numel tuple. The number of elements is the product of its shape.

• quant (bool, optional) – Whether pint.Quantity objects should be considered valid or not. If quant=True then obj is considered a valid numerical object only when obj is a quantity object with a valid numerical object as the magnitude. If quant=False, then obj must be a numerical object itself and not a pint.Quantity to be considered valid. If quant=None (the default), then either quantities or numerical objects are considered valid.

• ureg (UnitRegistry, None, or Ellipsis, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then the registry of obj is used if obj is a quantity, and immlib.units is used if not.

• unit (unit-like or Ellipsis, optional) – A unit with which the unit of obj must be compatible in order for obj to be considered a valid numerical object. An obj that is not a quantity is considered to have dimensionless units. If unit=Ellipsis (the default), then the object’s unit is ignored.

• sparsetype ('matrix', 'array', or None, optional) – The kind of sparse array to accept: either 'matrix' for the scipy sparse matrix types (e.g., scipy.sparse.csr_matrix) or 'array' for the sparse array types (e.g., scipy.sparse.csr_array). If the value is None (the default) then either is accepted.

Returns:

True if obj is a valid sparse numerical object, otherwise False.

Return type:

bool

is_str(obj)

Returns True if an object is a string and False otherwise.

is_str(obj) returns True if the given object obj is an instance of the str type and False otherwise.

Parameters:

obj (object) – The object whose quality as a string object is to be assessed.

Returns:

True if obj is a string, otherwise False.

Return type:

bool

is_tcoll(obj)

Returns True if an object is a transient tlist, tset, or tdict.

is_tcoll(obj) returns True if the given object obj is an instance of the tdict, tset, or tlist types, all of which are transient collections. Otherwise, False is returned.

Parameters:

obj (object) – The object whose quality as a transient collection is to be assessed.

Returns:

True if obj is a tlist, tset, or tdict and False otherwise.

Return type:

bool

is_tensor(obj, /, dtype=None, *, shape=None, ndim=None, numel=None, device=None, requires_grad=None, sparse=None, quant=None, unit=Ellipsis, ureg=None)

Returns True if the argument is a torch.tensor object, otherwise returns False.

is_tensor(obj) returns True if the given object obj is an instance of the torch.Tensor class or is a pint.Quantity object whose magnitude is an instance of torch.Tensor. Additional constraints may be placed on the object via the optional argments.

Parameters:

• obj (object) – The object whose quality as a PyTorch tensor object is to be assessed.

• dtype (dtype-like or None, optional) – The PyTorch dtype or a dtype-like object that is required to match that of the obj in order to be considered a valid tensor. The obj.dtype matches the given dtype parameter if either dtype is None (the default) or if obj.dtype is equal to the PyTorch equivalent ot dtype. Alternately, dtype can be a tuple, in which case, obj is considered valid if its dtype is any of the dtypes in dtype. ndim : int or tuple or ints or None, optional The number of dimensions that the object must have in order to be considered a valid tensor. If None, then any number of dimensions is acceptable (this is the default). If this is an integer, then the number of dimensions must be exactly that integer. If this is a list or tuple of integers, then the dimensionality must be one of these numbers.

• shape (int, tuple of ints, None, optional) – If the shape parameter is not None, then the given obj must have a shape shape that matches the parameter value. The value of shape must be a tuple that is equal to obj’s shape tuple with the following additional rules: a -1 value in the shape tuple will match any value in the obj’s shape tuple, and a single Ellipsis may appear in shape, which matches any number of values in the obj’s shape tuple. The default value of None indicates that no restriction should be applied to the obj’s shape.

• numel (int, tuple of ints, or None, optional) – If the numel parameter is not None, then the given obj must have the same number of elements as given by numel. If numel is a tuple, then the number of elements in obj must be in the numel tuple. The number of elements is the product of its shape.

• device (device-name or None, optional) – If device is None, then a tensor with any device field is considered valid; otherwise, the device parameter must equal obj.device for obj to be considered a valid tensor. The default value is None.

• requires_grad (bool or None, optional) – If None, then a tensor with any requires_grad field is considered valid; otherwise, the requires_grad parameter must equal obj.requires_grad for obj to be considered a valid tensor. The default value is None.

• sparse (bool or None, optional) – If the sparse parameter is None, then no requirements are placed on the sparsity of obj for it to be considered a valid tensor. If sparse is True or False, then obj must either be sparse or not be sparse, respectively, for obj to be considered valid. If sparse is a string, then it must be either 'coo' or 'csr', indicating the required sparse array type.

• quant (bool, optional) – Whether pint.Quantity objects should be considered valid tensors or not. If quant is True then obj is considered a valid array only when obj is a quantity object with a torch tensor as the magnitude. If quant is False, then obj must be a torch tensor itself and not a Quantity to be considered valid. If quant is None (the default), then either quantities or torch tensors are considered valid.

• unit (unit-like, None, Ellipsis, optional) – A unit with which the object obj’s unit must be compatible in order for obj to be considered a valid tensor. An obj that is not a quantity is considered to have a unit of None. If unit=Ellipsis (the default), then the object’s unit is ignored.

• ureg (UnitRegistry, None, or Ellipsis, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then the registry of obj is used if obj is a quantity, and immlib.units is used if not.

Returns:

True if obj is a valid PyTorch tensor whose properties match the requirements spelled out by the optional parameters, otherwise False.

Return type:

boolean

See also

is_array, is_numeric

is_tplandict(arg)

Determines if an object is a tplandict instance.

is_tplandict(x) returns True if x is a tplandict object and False otherwise.

is_tuple(obj)

Returns True if an object is a tuple object.

is_tuple(obj) returns True if the given object obj is an instance of the tuple type.

Parameters:

obj (object) – The object whose quality as an tuple object is to be assessed.

Returns:

True if obj is an instance of tuple, otherwise False.

Return type:

bool

is_unit(q, /, *, ureg=None)

Returns True if q is a pint.Unit object and False otherwise.

is_unit(q) returns True if q is a unit object (of type pint.Unit) and False otherwise.

Parameters:

• q (object) – The object whose quality as a pint.Unit is to be assessed.

• ureg (UnitRegistry, Ellipsis, or None, optional) – The pint.UnitRegistry object that the given unit object must belong to. If None (the default), then any unit registry is allowed. If Ellipsis, then the immlib.units registry is used. Otherwise, this must be a specific pint.UnitRegistry object.

Returns:

True if q is a pint.Unit object and False otherwise.

Return type:

bool

Raises:

TypeError – If the ureg parameter is not a pint.UnitRegistry, Ellipsis, or None.

is_ureg(obj)

Returns True if an object is a ping.UnitRegistry object.

is_ureg(obj) returns True if the given object obj is an instance of the pint.UnitRegistry type.

Parameters:

obj (object) – The object whose quality as an UnitRegistry object is to be assessed.

Returns:

True if obj is an instance of UnitRegistry, otherwise False.

Return type:

bool

is_url(url, /)

Returns True if given a valid URL string and False otherwise.

is_url(url) returns True if and only if the given URL is a valid URL string that includes the URL scheme and the netloc unless the scheme is 'file', in which case the netloc is optional. Whether the URL can be requested or not does not make a difference; is_url operates on the given URL string alone.

See also

can_download_url

itemmap(f, d, /, *args, **kwargs)

Returns a dictionary object whose values are a function of a given dictionary’s items.

itemmap(f, d) returns a dict whose keys are the same as those of the given mapping object d and whose values, for each key k are f(k, d[k]).

itemmap(f, d, *args, **kw) additionally passes the given arguments to the function f, such that in the resulting map, each key k is mapped to f(k, d[k], *args, **kw).

Unlike lazyitemmap, this function returns either a dict, a pdict, or an ldict depending on the input argument d. If d is an ldict, then an ldict is returned; if d is a pdict, a pdict is returned, and otherwise, a dict is returnd.

Parameters:

• f (function) – The function used to create the values in the new dictionary; it must accept two arguments (f(k, d[k])) plus any additional arguments provided in *args and **kwargs.

• d (collections.abc.Mapping) – A mapping whose keys are to be preserved and remapped to a function of their values.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

This function always returns a dictionary object whose type is either pcollections.pdict, pcollections.ldict, or dict, depending on the type of d.

Return type:

pcollections.pdict or pcollections.ldict or dict

itersafe(obj)

Returns an iterator of the given object or None if it is not iterable.

itersafe(obj) is equivalent to iter(obj) with the exception that, if obj is not iterable, it returns None instead of raising an exception.

Parameters:

obj (object) – The object to be iterated.

Returns:

If obj is iterable, returns iter(obj); otherwise, returns None.

Return type:

iterator or None

See also

can_iter, is_aiterable

keymap(f, d, /, *args, **kwargs)

Returns a dict object whose values are a function of a dict’s keys.

keymap(f, d) returns a dict whose keys are the same as those of the given dict object and whose values, for each key k are f(k).

keymap(f, d, *args, **kw) additionally passes the given arguments to the function f, such that in the resulting map, each key k is mapped to f(k, *args, **kw).

This function returns either a dict or a pdict. If d is a pdict, a pdict is returned, and otherwise, a dict is returnd. Unlike the valmap function, an ldict is never returned because the lazy values of such a dictionary are not accessed by keymap; if a lazy dictionary is required, then the function lazykeymap should be used instead.

Parameters:

• f (function) – The function used to create the values in the new dictionary.

• d (collections.abc.Mapping or iterable) – A mapping whose keys are to be preserved and remapped to a function of themselves. Alternatively, this may be an iterable of the keys instead of a dict with matching keys.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

This function always returns a dictionary object whose type is either pcollections.pdict or dict, depending on the type of d.

Return type:

collections.abc.Mapping object

lambdadict(*args, **kwargs)

Builds and returns a ldict with lambda functions calculated lazily.

lambdadict(args...) is equivalent to merge(args...) except that always returns an object of type pcollections.ldict and that any lambda function in the values provided by the merged arguments is made into a lazy partial function whose inputs come from the lambda-function variable names in the same resulting ldict.

Warning

This function will gladly return an ldict that encapsulates an infinite loop if you are not careful. For example, the following lambdadict will infinitely loop when either key is requested: ld = lambdadict(a=lambda b:b, b=lambda a:a).

Examples

>>> d = lambdadict(a=1, b=2, c=lambda a,b: a + b)
>>> d.is_lazy('c')
True

>>> d.is_ready('c')
False

>>> d['c']
3

>>> d
{|'a': 1, 'b': 2, 'c': 3|}

lazyitemmap(f, d, /, *args, **kwargs)

Returns an ldict object whose values are a function of a dict’s items.

lazyitemmap(f, d) yields an ldict whose keys are the same as those of the given dict object and whose values, for each key k, are lazily computed as f(k, d[k]).

itemmap(f, d, *args, **kw) additionally passes the given arguments to the function f, such that in the resulting map, each key k is mapped to f(k, d[k], *args, **kw).

Parameters:

• f (function) – The function used to create the values in the new dictionary; it must accept two arguments (f(k, d[k])) plus any additional arguments provided in *args and **kwargs.

• d (collections.abc.Mapping) – A mapping whose items are to be preserved and remapped to a function of their keys and values.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

This function always returns a lazy dictionary object of type pcollections.ldict.

Return type:

pcollections.ldict

lazykeymap(f, d, /, *args, **kwargs)

Returns a object of type pcollections.ldict whose values are a function of the keys of the mapping d.

keymap(f, d) returns a dict whose keys are the same as those of the given dict object and whose values, for each key k are f(k). If d is a sequence or iterable, then it is treated as a sequence of keys.

keymap(f, d, *args, **kw) additionally passes the given arguments to the function f, such that in the resulting map, each key k is mapped to f(k, *args, **kw).

Parameters:

• f (function) – The function used to create the values in the new dictionary.

• d (collections.abc.Mapping or iterable) – A mapping whose keys are to be preserved and remapped to a function of themselves. Alternatively, this may be an iterable of the keys instead of a dict with matching keys.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

This function always returns an object of type ldict whose values are lazy.

Return type:

pcollections.ldict

lazyvalmap(f, d, /, *args, **kwargs)

Returns a dict object whose values are transformed by a function.

lazyvalmap(f, d) returns a dict whose keys are the same as those of the given dict object and whose values, for each key k are f(d[k]). All values are created lazily.

lazyvalmap(f, d, *args, **kw) additionally passes the given arguments to the function f, such that in the resulting map, each key k is mapped to f(d[k], *args, **kw).

Parameters:

• f (function) – The function used to create the values in the new dictionary.

• d (collections.abc.Mapping) – A mapping whose keys are to be preserved and remapped to a function of their values.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

This function always returns a lazy dictionary object of type pcollections.ldict.

Return type:

pcollections.ldict

ldictmap(f, keys, *args, **kw)

Returns a lazy dictionary with the given keys and the values map(f, keys).

lazydictmap(f, keys) returns a pcollections.ldict object whose keys are the elements of iter(keys) and whose values are the elements of map(f, keys). All values are lazy.

lazydictmap(f, keys, *args, **kw) returns a pcollections.ldict object whose keys are the elements of iter(keys) and whose values are the elements of [f(k, *args, **kw) for k in iter(keys)], lazily calculated.

Parameters:

• f (function) – The function used to create the values in the new dictionary; it must accept one arguments (f(k)) plus any additional arguments provided in *args and **kwargs.

• keys (iterable) – An iterable object whose values are to become the keys of the new dictionary.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

A persistent lazy dictionary of the given keys with each key k mapped to f(k).

Return type:

pcollections.ldict

like_azpath(obj)

Detects whether an input can be converted into an AzureBlobPath object.

like_azpath(obj) returns True if obj is an instance of the AzureBlobPath class or is a string that forms a valid Azure path, and False otherwise.

See also: is_azpath

Parameters:

obj (object) – The object whose ability to be converted into an AzureBlobPath instance is to be determined.

Returns:

True if obj is an instance of AzureBlobPath or is a string that could be converted into an AzureBlobPath and False otherwise.

Return type:

boolean

like_filepath(obj)

Detects whether an input can be converted into a Path object.

like_filepath(obj) returns True if obj is an instance of the Path class or is a string that forms a valid path, and False otherwise. Most strings are at least theoretically valid paths, but any string that starts with a sheme followed by '://' must have the 'file' scheme.

See also: is_filepath

Parameters:

obj (object) – The object whose ability to be converted into a Path instance is to be determined.

Returns:

True if obj is an instance of AzureBlobPath or is a string that could be converted into an AzureBlobPath and False otherwise.

Return type:

boolean

like_gspath(obj)

Detects whether the input can be converted into an GSPath object.

like_gspath(obj) returns True if obj is an instance of the GSPath class or is a string that forms a valid GS path, and False otherwise.

See also: is_gspath

Parameters:

obj (object) – The object whose ability to be converted into an GSPath instance is to be determined.

Returns:

True if obj is an instance of GSPath or is a string that could be converted into an GSPath and False otherwise.

Return type:

boolean

like_number(obj, /)

Determines whether the argument holds a scalar number value or not.

like_number(x) returns True if x is already a scalar number, if x is a single-element numpy array or tensor, or if x is a sequence or set that has only one numerical element; otherwise, it returns False.

If like_number(x) returns True, then to_number(x) will always return a valid Python number (i.e., an object of type numbers.Number).

See also

is_number, to_number

like_osfpath(obj)

Detects whether the input can be converted into an OSFPath object.

like_osfpath(obj) returns True if obj is an instance of the OSFPath class or is a string that forms a valid OSF path, and False otherwise.

See also: is_osfpath

Parameters:

obj (object) – The object whose ability to be converted into an OSFPath instance is to be determined.

Returns:

True if obj is an instance of OSFPath or is a string that could be converted into an OSFPath and False otherwise.

Return type:

boolean

like_path(p)

Detects whether an object is either like a Path or CloudPath object.

Both Path and CloudPath object abstractly represent paths, but they do not share a subclass. like_path tests whether an object’s type is a subclass of any of path types recognized by immlib or is a string or bytes object that could be converted into a path. Additional path types can be registered by adding immlib.pathlib.PathTypeRecord instances to the immlib.pathlib.pathtypes dictionary. The key for such a record should be the string prefix for the path type (such as 's3' for an S3Path type).

Parameters:

p (object) – The object whose quality as a path-like object is to be assessed.

Returns:

True if p has a type that is recognized by immlib as a path type or is an object that can be converted into a path type and False otherwise.

Return type:

boolean

like_s3path(obj)

Detects whether the input can be converted into an S3Path object.

like_s3path(obj) returns True if obj is an instance of the S3Path class or is a string that forms a valid S3 path, and False otherwise.

See also: is_s3path

Parameters:

obj (object) – The object whose ability to be converted into an S3Path instance is to be determined.

Returns:

True if obj is an instance of S3Path or is a string that could be converted into an S3Path and False otherwise.

Return type:

boolean

like_unit(obj, /, *, ureg=Ellipsis)

Returns True if obj is or names a pint.Unit and False otherwise.

like_unit(obj) returns True if obj is a pint.Unit object or a string that names a pint.Unit and False otherwise.

Parameters:

• obj (object) – The object whose quality as a pint.Unit is to be assessed.

• ureg (pint.UnitRegistry, Ellipsis, or None, optional) – The pint.UnitRegistry object to use. If None, then any registry is allowed but an exception is raised if obj is a string because there is no registry in which to look it up. If Ellipsis (the default), then the immlib.units registry is required.

Returns:

True if obj is a pint.Unit or a string naming such a unit and False otherwise.

Return type:

bool

mag(obj, /, unit=Ellipsis, *, strict=False)

Returns the magnitude of the given object.

mag(quantity) returns the magnitude of the given quantity, regardless of the quantity’s unit.

mag(obj), for a non-quantity object obj, simply returns obj.

mag(arg, unit) returns arg.m_as(unit) if arg is a quantity and returns arg itself if arg is not a quantity.

mag(arg, Ellipsis) is equivalent to mag(arg).

mag(obj, None) returns obj if it is not a pint.Quantity and raises an exception if obj is a pint.Quantity.

If mag(quantity, unit) is given a quantity not compatible with the given unit, then an error is raised.

Note that if the first argument to mag() is not a quantity, then the unit argument is always ignored, and the first argument is returned as-is. This behavior can be changed using the strict option.

Parameters:

• obj (object) – The object that is to be converted into a magnitude.

• unit (unit-like, None, or Ellipsis, optional) – The unit in which the magnitude of the argument obj should be returned. The default argument of Ellipsis indicates that the value’s native unit, if any, should be used. A value of None indicates that the obj must have no units (i.e., not be a quantity), otherwise an exception is raised.

• strict (bool, optional) – Whether strict matching of the unit is performed. If False (the default), then a non-quantity (such as a plain NumPy array) is treated as a quantity whose unit is the type passed in the unit parameter; if True, then obj must be compatible with the unit parameter or an error is raised.

Returns:

The magnitude of obj in the requested unit, if obj is a quantity, or obj itself, if it is not a quantity.

Return type:

object

Raises:

• DimensionalityError – If the given obj is a quantity whose unit is not compatible with the unit parameter.

• ValueError – If unit is None but obj is a quantity or if a unit is requested of a non-quantity with the strict option enabled.

merge(*args, **kwargs)

Merges dict-like objects left-to-right. See also rmerge.

merge(...) collapses all arguments, which must be Mapping objects of some kind (dict, pdict, ldict, or a similar type), into a single mapping from left-to-right (i.e., with values in dictionaries to the right in the argument list overwriting values to the left in the argument list). The mapping that is returned depends on the inputs: if any of the input mappings are ldict objects, then an ldict is returned (and the laziness of arguments is respected); otherwise, a pdict object is retuend.

Named arguments may be passed after the dictionaries; these are collectively considered equivalent to one additional dictionary argument to the right of the positional mapping arguments.

Parameters:

• args – A sequence of collections.abc.Mapping objects such as dict objects.

• kwargs – Additional key-value pairs that are merged into the result last.

Returns:

A dictionary that represents the merger of all given dictionaries and key-value pairs. If any of the arguments are lazy dictionaries (pcollections.ldict) then the return value is also lazy in order to respect the laziness of the arguments.

Return type:

pcollections.pdict or pcollections.ldict

See also

rmerge

Merges dictionaries from right to left.

nestget(d, /, *args, **kwargs)

Returns a value from a data structure of nested mappings and sequences.

The nestget function is essentially a nested version of the get function that works for both Mapping and Sequence types (e.g., dict, list, tuple, and related types that implement their abstract bases).

nestget(data, k1, k2, k3...) extracts element k1 from data then element k2 from that value, then element k3 from that value, etc., until there are no more keys; the final value is returned. If any of the values are missing, then the optional value default is returned if it is provided and an error is raised if it is not. Note that the provided keys may be integer indices for list-like objects that may be included in the nesting. If a string key is given for a list-like container, then this is treated as a missing key, not an error.

This function raises a KeyError` when a key or index is not found in the relevant container, but this behavior can be changed by passing the optional named parameter ``default. If default is provided, then this value is returned if any keys are missing.

Parameters:

• d (object) – The dict-like or list-like object from which an element is being extracted.

• args – The list of keys and indices to be extracted.

• kwargs – The default value to be returned if an item is not found can be specified using the named option default. If this option is not provided, then an error is raised should the item not be found.

Returns:

The object found at the given nested position in the data structure d. If one of the provided keys does not exist in the associated sub-collection of d, then the default option is returned.

Return type:

object

Raises:

KeyError – If the given sequence of keys cannot be found in the nested data structure and no default option is provided.

class numapi(fn)

An interface for defining functions that expect all arguments to be either numpy arrays or pytorch tensors.

A function decorated with @numapi is a placeholder for two subfunctions: one that is called when any of the arguments are pytorch tensors (all of whose arguments, when possible, are converted into pytorch tensors), and a version called otherwise, all of whose arguments are converted into numpy arrays when possible. The body of the decorated function is usually pass, but, if desired, it can return either the pytorch or the numpy modules to indicate that a particular version of the function should be called (if necessary, tensors are converted into numpy arrays for this).

Once a function has been decorated with @numapi, that function should be used to decorate two other functions. If, for example, the function f is decorated with @numapi, then @f.array should be used to decorate the version of the function that accepts numpy arrays and @f.tensor should be used to decorate the version of the function that accepts pytorch tensors.

Examples

>>> @numapi
... def l2_distance(pt1, pt2):
...     "Calculates the L2 distance between two points."
...     pass

>>> @l2_distance.array
... def _(pt1, pt2):
...     return np.sqrt(np.sum((pt1 - pt2)**2, axis=0))

>>> @l2_distance.tensor
... def _(pt1, pt2):
...     return torch.sqrt(torch.sum((pt1 - pt2)**2, axis=0))

>>> l2_distance(torch.tensor([0,0]), [0,1])
tensor(1.)

>>> l2_distance([0,0], torch.tensor([0,1]))
tensor(1.)

>>> l2_distance([0,0], [0,1])
1.0

numeric_args(fn=None, /, *args)

Converts arguments of the decorated function into either NumPy arrays or PyTorch tensors.

The decorator @numeric_args, when applied to a function, will convert all of that function’s arguments into numeric collections–either PyTorch tensors or NumPy arrays–prior to invoking the function. Either all arguments are converted into either NumPy arrays or all arguments are converted into PyTorch tensors; the former only occurs when no PyTorch tensors occur in the argument list. numeric_args considers pint.Quantity objects whose magnitudes are numeric collections to be numeric collections and will convert arguments that are quantitites into new quantities with numeric magnitudes if necessary.

If a function is decorated with @numeric_args('arg1', 'arg2' ...) then only the arguments whose names are given (arg1, arg2, …) are converted into numeric collections.

When arguments are converted into PyTorch tensors, the first object in the argument list that is already a tensor is found and its device is used as the device for all converted objects. If no such object is found, then None is used for the device.

osfpath(obj, *args, client=None, cache_path=Ellipsis, file_cache_mode=Ellipsis, mkdir_mode=Ellipsis, pagesize=Ellipsis, local_cache_dir=None)

Creates and returns an OSFPath representing an OSF.io repository.

osfpath(p) creates and returns an OSFPath object, which is a type of cloudpathlib.CloudPath object, from the path or path-string p. If p is an OSFPath, then it is returned as-is. Otherwise str(p) is converted into an OSFPath; str(p) may start with 'osf://' (not case-sensitive) or, if it does not have a scheme specifier, 'osf://' will be prepended to it.

osfpath(p, a1, a2...) converts p into an OSFPath then joins the a1, a2, etc. values to the end of the path and returns the joined path.

OSF paths take the format 'osf://<project-ID>:<storage>/<path>' where the storage is optional (defaulting to 'osfstorage') and an empty path refers to the project storage’s root. The project-ID is the code used to find the project online. For example, the webpage reached at the website https://osf.io/tery8/ is the project page for the project whose ID is tery8.

If any of the optional keyword arguments are given, then the returned path will always use the specified options; a new path is returned with updated options if necessary; this is done before joining paths if multiple positional arguments are given. If the client keyword is given, then it is modified by the keyword options before being used in the path. All optional keyword arguments have a default value of Ellipsis, which indicates that the value of the client for that option should be used.

Parameters:

• obj (path-like) – The path or path-like object to convert into an OSFPath, typically a string.

• client (OSFClient or None, optional) – The OSFClient object to use. The OSFClient is responsible primarily for the caching of data locally. If OSFClient is None, then an OSFClient object is created for the project using a temporary cache directory.

• cache_path (path-like or None, optional) – The local directory in which cache files should be stored. This option is ignored if client is not None; otherwise it is passed to the created client object. The cache directory is the root cache directory for the entire OSF project.

• file_cache_mode (cloudpathlib.enums.FileCacheMode, optional) – How often to clear the file cache; see [cloudpathlib’s caching docs](https://cloudpathlib.drivendata.org/stable/caching/) for more information about the options in cloudpathlib.enums.FileCacheMode.

• mkdir_mode (int, optional) – The mode to use when making directories in the cache. By default this is 0o775. This option is ignored if the client option is not None.

• pagesize (int, optional) – The number of items to include in a single page when paging directory contents from the OSF server. The default is 100. This option is ignored if the client option is not None.

path(arg0, *args, **kwargs)

Convenience function for instantiating Path objects.

path(arg) returns a Path-like object that references the path given by the argument arg. The arg is converted into a string prior to conversion into a path if it is not a path already.

path(arg, *args) joins the list of arguments in args to the path created from the arg.

Optional keyword arguments may be given as well,

pathdict(arg, all=False, filter=None, ondir=None, onfile=None)

Returns a lazy dictionary of the nested paths beginning at the argument.

Searches a directory and all its subdirectories, lazily, for all contents, uniting them in a single nested lazy dictionary. The dictionaries in the nest represent directories whose keys are the filenames of their contents. The value of a key is another lazy dictionaries or a path object if the file is not a directory.

Parameters:

• arg (path-like) – The path from which to begin the search.

• all (boolean, optional) – Whether to include hidden files (True) or not (False) in the directory lists. The default is False.

• filter (None or function, optional) – A filter that must return either True (indicating that the path should be included in the pathdict) or False (indicating that the path should not be included in the pathdict) for each path that is scanned. The default is None, meaning that no filter is applied.

• ondir (function, optional) – A function to run on any path encountered during the pathdict search that is a directory. When pathdict is given a path that is a directory, it returns a lazy dictionary whose keys are the filenames of the contents of that directory. For subdirectories, their filenames are mapped to the return value of ondir(path) where path is the path object for the subdirectory. By default this is pathdict itself, resulting in a nested structure for subdirectories. The all, filter, ondir, and onfile parameters are all passed to this function.

• onfile (function, optional) – A function to run on any path encountered during the pathdict search that is a file. When pathdict is given a path that is a directory, it returns a lazy dictionary whose keys are the filenames of the contents of that directory. For files in the directory, their filenames are mapped to the return value of onfile(path) where path is the path object for the file. By default this is None, indicating that the path itself should be returned.

Returns:

A lazy dictionary of the contents of the argument.

Return type:

ldict

pathstr(obj)

Returns a string or bytes representation of a path.

pathstr(obj) returns obj itself if obj is either a str or bytes object. If obj is a CloudPath object, then str(obj) is returned. Otherwise, If obj is a PathLike object, then os.fspath(obj) is returned.

pdictmap(f, keys, /, *args, **kw)

Returns a pdict with the given keys and the values map(f, keys).

pdictmap(f, keys) returns a pdict object whose keys are the elements of iter(keys) and whose values are the elements of map(f, keys).

pdictmap(f, keys, *args, **kw) returns a dict object whose keys are the elements of iter(keys) and whose values are the elements of [f(k, *args, **kw) for k in iter(keys)].

Parameters:

• f (function) – The function used to create the values in the new dictionary; it must accept one arguments (f(k)) plus any additional arguments provided in *args and **kwargs.

• keys (iterable) – An iterable object whose values are to become the keys of the new dictionary.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

A persistent dictionary of the given keys with each key k mapped to f(k).

Return type:

pcollections.pdict

class plan(*args, **kwargs)

Represents a directed acyclic graph of calculations.

The plan class encapsulates individual functions that require parameters as inputs and produce outputs in the form of named values. Plan objects can be called as functions with a dictionary and/or a keyword arguments providing the plan’s parameters; they always return a type of lazy dictionary called a plandict of the values they calculate, even if they calculate only a single value.

Superficially, a plan is a pdict object whose values must all be calc objects. However, under the hood, every plan object maintains a directed acyclic graph of dependencies of the inputs and outputs of the calculation objects such that it can create plandict objects that reify the outputs of the various calculations lazily.

The keys that are used in a plan must be strings but are not otherwise restricted.

For a plan p = plan(calc_key1=calc1, calc_key2=calc2, ...), a plandict can be instantiated using the following syntax:

pd = p(param1=val1, param2=val2, ...)

This plandict is an enhanced ldict that evaluates components of the plan as requested based on laziness requirements of the calculations in the plan and on dictionary lookups of plan outputs. (ldict is the lazy dictionary type from the pcollections library.)

All plans implicitly contains the parameter 'cache_path' with the default value of None. This parameter is used by the plan’s plandict objects, to cache the outputs of calculations that were constructed with the option pathcache=True.

inputs

A pset of the input parameter names, as defined by the plan’s calculations. Note that the union of the inputs and the outputs is equivalent to the keys in any plan-dictionary.

Type:

pset of strs

outputs

A pset of the output parameter names, as defined by the plan’s calculations. Note that the union of the inputs and the outputs is equivalent to the keys in any plan-dictionary.

Type:

pset of strs

defaults

A dictionary whose keys consist of a subset of the inputs to the plan and whose values are the default values those parameters should take if they are not provided explicitly to the plan.

Type:

pdict

calcs

A persistent dictionary whose keys are the names of the various calculations in the plan and whose values are the calculation objects themselves.

Type:

pdict

input_docs

A dictionary whose keys are input parameter names and whose values are the combined documentation for the associated parameter across all calculations in the plan.

Type:

pdict

output_docs

A dictionary whose keys are output value names and whose values are the combined documentation for the associated outputs across all calculations in the plan.

Type:

pdict

requirements

A pset of the names of the required calculations of the plan (i.e., those with option lazy=False).

Type:

pset

__doc__

Every plan object is given a set of documentation which includes sections for the inputs and outputs as well as a listing of all the calculation steps.

Type:

str

class CalcData(names, calcs, args, sources, index)

args

Alias for field number 2

calcs

Alias for field number 1

index

Alias for field number 4

names

Alias for field number 0

sources

Alias for field number 3

class DepData(inputs, calcs)

calcs

Alias for field number 1

inputs

Alias for field number 0

filtercall(*args, **kwargs)

Calls the plan object, but filters out args that aren’t in the plan.

plan_obj.filtercall(dict1, dict2, ..., k1=v1, k2=v2, ...) is equivalent to plan_obj(dict1, dict2, ..., k1=v1, k2=v2, ...) except that any keys in the argument list to filtercall that aren’t in the parameter list of plan_obj are automatically filtered out.

class plandict(*args, **kwargs)

A persistent dict type that manages the outputs of executing a plan.

plandict(plan, params) instantiates a plan object with the given dict-like object of parameters, params.

plandict(plan, params, k1=v1, k2=v2, ...) additional merges all keyword arguments into parameters.

plandict(plan, k1=v1, k2=v2, ...) uses only the keyword arguments as the plan parameters.

Note that plandict(plan, args...) is equivalent to plan(args...).

plandict is a subclass of lazydict, but it has some unique behavior, primarily in that only the parameters of a plandict may be updated; the rest of the items are consequences of the plan and parameter.

Parameters:

• plan (plan) – The plan object that is to be instantiated.

• *params (dict-like, optional) – The dict-like object of the parameters of the plan. All and only plan parameters must be provided, after the params argument is merged with the kwargs options. This may be a lazydict, and this dict’s laziness is respected as much as possible.

• **kwargs (optional keywords) – Optional keywords that are merged into params to form the set of parameters for the plan.

plan

The plan object on which this plandict is based or alternatively a plandict or object to copy.

Type:

immlib.plan

inputs

The parameters that fulfill the plan. Note that these are the only keys in the plandict that can be updated using methods like set and setdefault.

Type:

pdict

delete(k)

Returns a copy of the mersistent mapping the excludes the given key.

If the key is not found in the mapping, a KeyError is raised.

set(k, v)

Returns a copy of the pdict that maps the given key to the given value.

setdefault(k, v=None)

Returns a copy of the persistent mapping with the key inserted with a value of default, if key is not already in the mapping.

transient()

Returns a transient copy of the dict in constant time.

class planobject(*args, **kwargs)

Base class for objects that are based on lazy calculation plans.

planobject is the base-class for all objects that use immlib.plan objects as their base type. Objects that inherit from planobject (which uses metaclass plantype) are defined in the same way that calculation plans are defined. Any attributes of the class (including those inherited from base classes) that are calculations (see immlib.calc and immlib.plan) are turned into a plan. The inputs of the plan are the required parameters for the class, and the outputs of the plan become the attributes of the class, which are resolved lazily like in plandict s.

The __init__ function of a planobject is special. During the __init__ function only, the parameters of a planobject function can be set using the usual setattr interface. All planobject``s are immutable once they have been initialized, however. At the end of the ``__init__ function, the object must have all of its parameters set, otherwise an error is raised. If no __init__ function is defined, then the planobject default init function calls merge on its arguments and keywords; the resulting dict must be a dictionary of the class’s parameters.

planobject types must not overload the following methods, as they are used by the planobject / plantype system. These are: * __new__ * __setattr__ * __getattr__ * __dir__

copy(**kwargs)

Creates a copy of the planobject with optional parameter updates.

is_persistent()

Returns True if the planobject is persistent and False if it is transient.

persistent()

Returns a persistent copy of the planobject. If the planobject is already persistent, it is returned unchanged.

transient()

Returns a transient copy of the planobject.

class plantype(name, bases, attrs, **kwargs)

A metaclass that allows one to create lazy types from calculation plans.

The plantype metaclass handles classes with the base-class planobject. In general, one should create a plan-object by inheriting from planobject, not by providing the plantype metaclass, but passing plantype has the same effect (all classes created with metaclass plantype will inherit from planobject).

See planobject for more information.

class planobject_base(*args, **kwargs)

The base-class for the immlib.planobject class.

plantype.planobject_base is a simple class that implements the basic features of the planobject class. The separation for certain methods from the planobject type itself is required due to details of how the planobject class, which has the plantype meta-class, gets initialized while the plantype.__new__ method depends on methods in the planobject class (which hasn’t been initialized/defined at the time that the planobject.__new__ method is called. This class shouldn’t be used directly and shouldn’t be inherited. Use the planobject class instead.

promote(*args, ureg=None)

Promotes all arguments into quantities with compatible magnitudes.

promote(a, b, c...) converts all of the passed arguments into numerical quantity objects and returns them as a list. The returned arguments will all have compatible (promoted) types or magnitude types.

Promotion is determined based on the object type. If any of the objects are PyTorch tensors or quantities with tensor magnitudes, then all of the returned quantities will have for their magnitudes PyTorch tensors with the same profile (e.g, device) as the tensor argument(s). Otherwise, the returned quantities will be converted into array types. The purpose of this promotion is to ensure that all arguments can be combined into an expression (PyTorch tensor operations generally require that all arguments are tensors).

Parameters:

• args – The arguments that are to be promoted.

• ureg (pint.UnitRegistry or None, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then no specific coersion to a pint.UnitRegistry is performed.

Returns:

A list of the arguments after each has been promoted.

Return type:

list of tensors or arrays

quant(mag, /, unit=Ellipsis, *, ureg=None)

Returns a pint.Quantity object with the given magnitude and unit.

quant(mag, unit) returns a pint.Quantity object with the given magnitude mag and unit. If mag is alreaady a pint.Quantity, then it is converted into the given units and returned (a copy of mag is made only if necessary); if the units of mag in this case are not compatible with unit, then an error is raised. If mag is not a quantity, then the given unit is used to create the quantity.

quant(mag) is equivalent to quant(mag, Ellipsis). Both return mag if mag is already a pint.Quantity object; otherwise they return a quantity with dimensionless units.

Warning

The value unit=None is not equivalent to unit='dimensionless'; rather, unit=None is used throughout immlib to indicate a non-quantity such as a plain PyTorch tensor or a NumPy array. Accordingly, an exception is raised when unit=None is given.

Parameters:

• mag (object) – The magnitude to be given a unit.

• unit (unit-like, Ellipsis, optional) – The units to use in the returned quantity. If Ellipsis is given (the default), then dimensionless units are assumed unless the mag argument already is a quantity with its own units. If None is given, then an exception is raised.

• ureg (pint.UnitRegistry, None, Ellipsis, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then no specific coersion to a pint.UnitRegistry is performed, and immlib.units is used when a unit name is given.

Returns:

A quantity object representing the given magnitude and unit.

Return type:

pint.Quantity

Raises:

ValueError – If unit is None.

reload_immlib()

Reload and return the entire immlib package.

immlib.reload_immlib() reloads every submodule in the immlib package then reloads immlib itself, and returns the reloaded package.

Warning

This function exists primarily for debugging purposes; its use is not generally needed or advised by users of the library.

Returns:

The newly reloaded immlib module.

Return type:

module

Examples

>>> import immlib as il
>>> il.units = None  # This will break parts of the library.
>>> il = il.reload_immlib()  # But this resets it.
>>> il.units is not None
True

rmerge(*args, **kwargs)

Merges dictionary objects right-to-left. See also merge.

rmerge(...) collapses all arguments, which must be python Mapping objects of some kind, into a single mapping from right-to-left. The mapping that is returned depends on the inputs: if any of the input mappings are lazydict objects, then a lazydict is returned (and the laziness of arguments is respected); otherwise, a frozendict object is retuend.

Named arguments may be passed after the dictionaries; these are collectively considered equivalent to one additional dictionary argument to the right of the positional mapping arguments.

Note

The rmerge function is identical to the merge function but with reversed arguments. In other words, merge(*args, **kw) is equivalent to rmerge(kw, **reversed(args)).

Parameters:

• args – A sequence of collections.abc.Mapping objects such as dict objectss.

• kwargs – Additional key-value pairs that are merged into the result first.

Returns:

A dictionary that represents the merger of all given dictionaries and key-value pairs. If any of the arguments are lazy dictionaries (pcollections.ldict) then the return value is also lazy in order to respect the laziness of the arguments.

Return type:

pcollections.pdict or pcollections.ldict

See also

merge

Merges dictionaries from left to right.

s3path(obj, *args, **kwargs)

Creates and returns an S3Path representing an AWS S3 repository.

s3path(p) creates and returns an S3Path object, which is a type of cloudpathlib.CloudPath object, from the path or path-string p. If p is an S3Path, then it is returned as-is. Otherwise pathstr(p) is converted into an S3Path; pathstr(p) may start with 's3://' (not case-sensitive) or, if it does not have a scheme specifier, 's3://' will be prepended to it.

s3path(p, a1, a2...) converts p into an S3Path then joins the a1, a2, etc. values to the end of the path and returns the joined path.

The s3path function accepts all the optional arguments of the S3Client type from cloudpathlib as well as the client option. If the client option is given along with additional optional arguments, then the optional arguments are ignored.

Additionally, s3path parses the option cache_path, which is not normally accepted by S3Path, which instead requires the option local_cache_dir. Any time that a local_cache_dir is given, it overrides the cache_path; however, if local_cache_dir is not given and cache_path is, then the directory os.path.join(cache_path, "s3") is given as the local_cache_dir option.

sparse_find(arr, /)

Returns the indices and values of nonzero elements of a sparse object.

sparse_find(sp_array) is equivalent to scipy.sparse.find(sp_array) for a sparse array sp_array.

sparse_find(sp_tensor) is equivalent to s.indices() + (s.values(),) for a sparse PyTorch tensor sp_tensor and a version of it that has been coalesced, s = sp_tensor.coalesce(). Note that the s.values() tensor is cloned and detached before being returned.

sparse_find(q) for a quantity q returns the equivalent of sparse_find(q.m) except that the returned value array will have the same magnitude as q.

Raises:

TypeError – If arr is not a sparse array or sparse tensor.

See also

sparse_data, sparse_indices

strcmp(a, b, /, case=True, *, unicode=None, strip=False, split=False)

Determines if the given objects are strings and compares them if so.

strcmp(a, b) returns None if either a or b is not a string; otherwise, it returns -1, 0, or 1 if a is less than, equal to, or greater than b, respectively, subject to the constraints of the parameters.

Parameters:

• a (object) – The first argument.

• b (object) – The second argument.

• case (bool, optional) – Whether to perform case-sensitive (case=True) or case-insensitive (case=False) string comparison. The default is False.

• unicode (bool or None, optional) – Whether to run unicode normalization on a and b prior to comparison. By default, this is None, which is interpreted as a True value when case is False and as False value when case is True. In other words, unicode normalization is performed when case-insensitive comparison is being performed but not when standard string comparison is being performed. Unicode normalization is always performed both before and after casefolding. Unicode normalization is performed using the unicodedata package’s normalize(unicode, string) function. If this argument is a string, it is instead passed to the normalize function as the first argument. When unicode is not a string but normalization is performed, them the default string is 'NFD'.

• strip (bool, optional) – If set to True, then a.strip() and b.strip() are used in place of a and b. If set to False (the default), then no stripping is performed. If a non-boolean value is given, then it is passed as an argument to the strip() method.

• split (bool, optional) – If set to True, then a.split() and b.split() are used in place of a and b. The lists of strings that result from a.split() and b.split() are rejoined with no separator prior to comparison. If this option is set to False (the default), then no splitting is performed. If a non-boolean value is given, then it is passed as an argument to the split() method.

Returns:

None if either a is not a string or b is not a string; otherwise, -1 if a is lexicographically less than b, 0 if a == b, and 1 if a is lexicographically greater than b, subject to the constraints of the optional parameters.

Return type:

bool or None

See also

strnorm, streq

strends(a, b, /, case=True, *, unicode=None, strip=False)

Determines whether or not the string a ends with the string b.

strends(a, b) returns True if a and b are both strings and if a ends with b, subject to the constraints of the parameters.

Parameters:

• case (bool, optional) – Whether to perform case-sensitive (case=True) or case-insensitive (case=False) string comparison. The default is False.

• unicode (bool or None, optional) – Whether to run unicode normalization on a and b prior to comparison. By default, this is None, which is interpreted as a True value when case is False and as False value when case is True. In other words, unicode normalization is performed when case-insensitive comparison is being performed but not when standard string comparison is being performed. Unicode normalization is always performed both before and after casefolding. Unicode normalization is performed using the unicodedata package’s normalize(unicode, string) function. If this argument is a string, it is instead passed to the normalize function as the first argument. When unicode is not a string but normalization is performed, them the default string is 'NFD'.

• strip (bool, optional) – If set to True, then a.strip() and b.strip() are used in place of a and b. If set to False (the default), then no stripping is performed. If a non-boolean value is given, then it is passed as an argument to the strip() method.

Returns:

If a and b are both strings then True is returned if a ends with b and False is returned otherwise. If either a or b is not a string, then None is returned.

Return type:

bool or None

streq(a, b, /, case=True, *, unicode=None, strip=False, split=False)

Determines if the given objects are equal strings or not.

streq(a, b) returns True if a and b are both strings and are equal to each other, subject to the constraints of the options.

Parameters:

• a (object) – The first argument.

• b (object) – The second argument.

• case (bool, optional) – Whether to perform case-sensitive (case=True) or case-insensitive (case=False) string comparison. The default is False.

• unicode (bool or None, optional) – Whether to run unicode normalization on a and b prior to comparison. By default, this is None, which is interpreted as a True value when case is False and as False value when case is True. In other words, unicode normalization is performed when case-insensitive comparison is being performed but not when standard string comparison is being performed. Unicode normalization is always performed both before and after casefolding. Unicode normalization is performed using the unicodedata package’s normalize(unicode, string) function. If this argument is a string, it is instead passed to the normalize function as the first argument. When unicode is not a string but normalization is performed, them the default string is 'NFD'.

• strip (bool, optional) – If set to True, then a.strip() and b.strip() are used in place of a and b. If set to False (the default), then no stripping is performed. If a non-boolean value is given, then it is passed as an argument to the strip() method.

• split (bool, optional) – If set to True, then a.split() and b.split() are used in place of a and b. The lists of strings that result from a.split() and b.split() are rejoined with no separator prior to comparison. If this option is set to False (the default), then no splitting is performed. If a non-boolean value is given, then it is passed as an argument to the split() method.

Returns:

If a and b are both strings then True is returned if a equals b and False is returned otherwise. If either a or b is not a string, then None is returned.

Return type:

bool or None

See also

strnorm, strcmp

striskey(s)

Determines if the given string is a valid keyword.

strissym(s) returns True if s is both a string and a valid keyword (such as 'if' or 'while'). Otherwise, it returns False if s is a string and None if not.

See also

strissym, strisvar

strissym(s)

Determines if the given string is a valid symbol (identifier).

strissym(s) returns True if s is both a string and a valid identifier. Otherwise, it returns False if s is a string and None if not.

See also

striskey, strisvar

strisvar(s)

Determines if the given string is a valid variable name.

strissym(s) returns True if s is both a string and a valid name (i.e., a symbol but not a keyword). Otherwise, it returns False if s is a string and None if not.

See also

strissym, striskey

strnorm(s, /, case=False, *, unicode=True)

Normalizes a string using the unicodedata package.

strnorm(s) returns a version of s that has been unicode-normalized using the unicodedata.normalize(s) function. Case-normalization can also be requested via the case option.

Parameters:

• s (object) – The string to be normalized.

• case (bool, optional) – Whether to perform case-normalization (case=True) or not (case=False, the default). If two strings are case-normalized, then an equality comparison will reveal whether the original (unnormalized strings) were equal up to the case of the characters. Case normalization is performed using the str.casefold() method.

• unicode (bool or str, optional) – Whether to perform unicode normalization via the unicodedata.normalize function. The default behavior (unicode=True) is to perform normalization, but this can be disabled with unicode=False. Alternatively, a string may be given, in which case it is passed to the unicodedata.normalize function as the first argument; when unicode is True, the string used is 'NFD'.

Returns:

A normalized version of s.

Return type:

str

strstarts(a, b, /, case=True, *, unicode=None, strip=False)

Determines whether or not the string a starts with the string b.

strstarts(a, b) returns True if a and b are both strings and if a starts with b, subject to the constraints of the parameters.

Parameters:

• case (bool, optional) – Whether to perform case-sensitive (case=True) or case-insensitive (case=False) string comparison. The default is False.

• unicode (bool or None, optional) – Whether to run unicode normalization on a and b prior to comparison. By default, this is None, which is interpreted as a True value when case is False and as False value when case is True. In other words, unicode normalization is performed when case-insensitive comparison is being performed but not when standard string comparison is being performed. Unicode normalization is always performed both before and after casefolding. Unicode normalization is performed using the unicodedata package’s normalize(unicode, string) function. If this argument is a string, it is instead passed to the normalize function as the first argument. When unicode is not a string but normalization is performed, them the default string is 'NFD'.

• strip (bool, optional) – If set to True, then a.strip() and b.strip() are used in place of a and b. If set to False (the default), then no stripping is performed. If a non-boolean value is given, then it is passed as an argument to the strip() method.

Returns:

True` if `a` and `b` are both strings and if `a` startss with `b`, subject to the constraints of the optional parameters. If either `a` or `b` is not a string, then ``None is returned.

Return type:

bool or None

tensor_args(fn=None, /, *args, keep_arrays=False)

Converts arguments of the decorated function into PyTorch tensors.

The decorator @tensor_args, when applied to a function, will convert all of that function’s arguments into PyTorch tensors prior to invoking the function. tensor_args considers pint.Quantity objects whose magnitudes are tensors to be tensors and will convert arguments that are quantitites into new quantities with tensor magnitudes.

If a function is decorated with @tensor_args('arg1', 'arg2' ...) then only the arguments whose names are given (arg1, arg2, …) are converted into tensors.

When arguments are converted into PyTorch tensors, the first object in the argument list that is already a tensor is found and its device is used as the device for all converted objects. If no such object is found, then None is used for the device.

The optional argument keep_arrays (default: False) can be set to True to indicate that the function should convert tensor return values back into NumPy arrays if none of the arguments to the function were originally tensors. This allows a function to be written using one numerical interface (PyTorch) but to work for either PyTorch tensors or NumPy arrays while returning values whose types match the input types.

to_array(obj, /, dtype=None, *, order=None, copy=False, sparse=None, frozen=None, quant=None, ureg=None, unit=Ellipsis, detach=True)

Reinterprets obj as a NumPy array or quantity with an array magnitude.

immlib.to_array is roughly equivalent to the numpy.asarray function with a few exceptions:

• to_array(obj) allows quantities for obj and, in such a case, will return a quantity whose magnitude has been reinterpreted as an array, though this behavior can be altered with the quant parameter;

• to_array(obj) can extract the numpy array from torch tensor objects.

Parameters:

• obj (object) – The object that is to be reinterpreted as, or if necessary covnerted to, a NumPy array object.

• dtype (data-type, optional) – The dtype that is passed to numpy.asarray().

• order ({'C', 'F'}, optional) – The array order that is passed to numpy.asarray().

• copy (boolean, optional) – Whether to copy the data in obj or not. If False, then obj is only copied if doing so is required by the optional parameters. If True, then obj is always copied if possible.

• sparse (bool, 'csr', 'coo', or None, optional) – If None, then the sparsity of obj is the same as the sparsity of the array that is returned. Otherwise, the return value will always be either a scipy.spase matrix (sparse=True) or a numpy.ndarray (sparse=False) based on the given value of sparse. The sparse parameter may also be set to 'bsr', 'coo', 'csc', 'csr', 'dia', or 'dok' to return specific sparse matrix types.

• frozen (bool or None, optional) – Whether the return value should be read-only or not. If None, then no changes are made to the return value; if a new array is allocated in the to_array() function call, then it is returned in a writeable form. If frozen=True, then the return value is always a read-only array; if obj is not already read-only, then a copy of obj is always returned in this case. If frozen=False, then the return-value is never read-only.

• quant (bool or None, optional) – Whether the return value should be a Quantity object wrapping the array (quant=True) or the array itself (quant=False). If quant is None (the default) then the return value is a quantity if either obj is a quantity or an explicit unit parameter is given and is not a quantity if obj is not a quantity.

• ureg (pint.UnitRegistry or None, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then no specific coersion to a UnitRegistry is performed (i.e., the same quantity class is returned).

• unit (unit-like, bool, or Ellipsis, optional) – The unit that should be used in the return value. When the return value of this function is a Quantity (see the quant parameter), the returned quantity always has a unit matching the unit parameter; if the provided obj is not a quantity, then its unit is presumed to be that requested by unit. When the return value of this function is not a Quantity object and is instead is a NumPy array object, then when obj is not a quantity the unit parameter is ignored, and when obj is a quantity, its magnitude is returned after conversion into unit. The default value of unit, Ellipsis, indicates that, if obj is a quantity, its unit should be used, and unit should be considered dimensionless otherwise.

• detach (bool, optional) – If the argument is a PyTorch tensor that requires gradient tracking, then it must be detached from the gradient tracking system before it can be turned into an array. If detach is True (the default), then this detachment is performed automatically. Otherwise, an error is raised if a tensor would need to be detached.

Returns:

Either a NumPy array equivalent to obj or a Quantity whose magnitude is a NumPy array equivalent to obj.

Return type:

numpy.ndarray or pint.Quantity

Raises:

ValueError – If invalid parameter values are given or if the parameters conflict.

See also

to_tensor, to_numeric

to_dense(obj, /, dtype=None, *, quant=None, ureg=None, unit=Ellipsis)

Returns a dense version of the numerical object obj.

to_dense(obj) returns obj if it is already a PyTorch dense tensor or a NumPy ndarray or a quantity whose magnitude is one of these. Otherwise, it converts obj into a dense representation and returns this. Additional requirements on the output format of the return value can be added using the optional parameters.

Parameters:

• obj (object) – The object that is to be converted into a dense representation.

• dtype (dtype-like or None, optional) – The dtype that is passed to torch.as_tensor(obj) or np.asarray(obj).

• quant (bool or None, optional) – Whether the return value should be a pint.Quantity object wrapping wrapping the object (quant=True) or the object itself (quant=False). If quant is None (the default) then the return value is a quantity if obj is a quantity and is not a quantity if obj is not a quantity.

• ureg (pint.UnitRegistry or None, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then no specific coersion to a pint.UnitRegistry is performed (i.e., the same quantity class is returned).

• unit (unit-like, bool, None, or Ellipsis, optional) – The unit that should be used in the return value. When the return value of this function is a pint.Quantity (see the quant parameter), the returned quantity always has a unit matching the unit parameter; if the provided obj is not a quantity, then its unit is presumed to be those requested by unit. When the return value of this function is not a pint.Quantity object and is instead a numeric object, then when obj is not a quantity the unit parameter is ignored, and when obj is a quantity, its magnitude is returned after conversion into unit. The default value of unit, Ellipsis, indicates that, if obj is a quantity, its unit should be used, and unit should be considered to be None otherwise.

Returns:

A dense version of the argument obj.

Return type:

dense tensor, dense array, or quantity with a dense magnitude

to_mcoll(obj, /, copy=True)

Returns a mutable copy of obj.

to_mcoll(obj) returns a mutable copy of the given collection obj. If obj is already a mutable collection, then a duplicate is returned. If obj is not a collection that can be converted into a mutable collection, then an error is raised.

A mutable collection, according to this function, is a Python list, set, or dict, depending on the type of obj. When obj is a Sequence, the result is a list; when obj is a Set, the result is a set; and when obj is a Mapping, the result is a dict.

Parameters:

• obj (collection) – An object that is to be converted into a persistent collection.

• copy (boolean, optional) – If obj is already a mutable collection, then a copy is made if and only if copy is True; otherwise, obj is returned as-is when it is already a mutable type. The default is True.

Returns:

A mutable version of obj; the return value’s type will always be one of list, set, or dict.

Return type:

object

Raises:

TypeError – If obj cannot be converted into a mutable collection.

to_number(obj, /, unit=Ellipsis, *, ureg=None)

Converts the argument into a simple Python number.

to_number(obj) returns a simple Python number representation of obj (in other words, obj will be a subtype of Python’s numbers.Number type). Any number, any NumPy array with only one element, and any PyTorch tensor with only one element can be converted into a scalar. If obj is a pint.Quantity then the return value is a quantity with the same unit as obj and whose magnitude is to_number(obj.m).

Parameters:

• obj (object) – The object that is to be converted into a scalar number.

• unit (unit-like, bool, or None, optional) – If obj is a pint.Quantity object, the unit parameter determines how it is handled by to_number. If unit is None and obj is a quantity, then an error will be raised. If unit is a valid pint.Unit or the an object that can be converted into a unit vis that immlib.unit function. then an error is raised if obj is not a quantity with alike units. If unit is Ellipsis (the default value), then the behavior depends on whether obj is a quantity: if obj is a quantity, the to_number function is run on its magnitude and a quantity with the same unit is returned; if obj is not a quantity, then the a non-quantity is returned.

• ureg (pint.UnitRegistry, None, or Ellipsis, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then the registry of obj is used if obj is a quantity, and immlib.units is used if not.

Returns:

A scalar number that is an object whose class is a subtype of numbers.Number or a pint.Quantity object whose magnitude is such a number.

Return type:

number or pint.Quantity

Raises:

TypeError – If the argument is not like a scalar number.

to_numeric(obj, /, dtype=None, *, copy=False, sparse=None, quant=None, ureg=None, unit=Ellipsis)

Reinterprets obj as a numeric type or quantity with such a magnitude.

immlib.to_numeric is roughly equivalent to the torch.as_tensor or numpy.asarray function with a few exceptions:

• to_numeric(obj) allows quantities for obj and, in such a case, will return a quantity whose magnitude has been reinterpreted as a numeric object, though this behavior can be altered with the quant parameter;

• to_numeric(obj) correctly handles SciPy sparse matrices, NumPy arrays, and PyTorch tensors.

If the object obj passed to immlib.to_numeric(obj) is a PyTorch tensor or a quantity whose magnitude is a PyTorch tensor, then a PyTorch tensor or a quantity with a PyTorch tensor magnitude is returned. Otherwise, a NumPy array, SciPy sparse matrix, or quantity with a magnitude matching one of these types is returned.

Parameters:

• obj (object) – The object that is to be reinterpreted as, or if necessary covnerted to, a numeric object.

• dtype (dtype-like or None, optional) – The dtype that is passed to torch.as_tensor(obj) or np.asarray(obj).

• copy (bool, optional) – Whether to copy the data in obj or not. If False, then obj is only copied if doing so is required by the optional parameters. If True, then obj is always copied if possible.

• sparse (bool, {'csr','csc','bsr','bsc','coo'}, or None, optional) – If None, then the sparsity of obj is the same as the sparsity of the object that is returned. Otherwise, the return value will always be either a spase object (sparse=True) or a dense object (sparse=False) based on the given value of sparse. The sparse parameter may also be set to 'coo', 'csr', or other sparse matrix names to return specific sparse layouts.

• quant (bool or None, optional) – Whether the return value should be a pint.Quantity object wrapping wrapping the object (quant=True) or the object itself (quant=False). If quant is None (the default) then the return value is a quantity if obj is a quantity and is not a quantity if obj is not a quantity.

• ureg (pint.UnitRegistry or None, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then no specific coersion to a pint.UnitRegistry is performed (i.e., the same quantity class is returned).

• unit (unit-like, bool, None, or Ellipsis, optional) – The unit that should be used in the return value. When the return value of this function is a pint.Quantity (see the quant parameter), the returned quantity always has a unit matching the unit parameter; if the provided obj is not a quantity, then its unit is presumed to be those requested by unit. When the return value of this function is not a pint.Quantity object and is instead a numeric object, then when obj is not a quantity the unit parameter is ignored, and when obj is a quantity, its magnitude is returned after conversion into unit. The default value of unit, Ellipsis, indicates that, if obj is a quantity, its unit should be used, and unit should be considered to be None otherwise.

Returns:

Either a NumPy array or PyTorch tensor equivalent to obj or a pint.Quantity whose magnitude is such an object.

Return type:

NumPy array or PyTorch tensor or Quantity

Raises:

ValueError – If invalid parameter values are given or if the parameters conflict.

See also

to_array, to_tensor

to_pcoll(obj)

Returns a persistent copy of obj.

to_pcoll(obj) returns obj itself if obj is a persistent collection; otherwise, it returns a persistent copy of obj. If obj is not a collection that can be converted into a persistent collection, then an error is raised.

Parameters:

obj (collection) – An object that is to be converted into a persistent collection.

Returns:

A persistent version of obj.

Return type:

object

Raises:

TypeError – If obj cannot be converted into a persistent collection.

to_sparse(obj, /, dtype=None, *, quant=None, ureg=None, unit=Ellipsis)

Returns a sparse version of the numerical object obj.

to_sparse(obj) returns obj if it is already a PyTorch sparse tensor or a SciPy sparse matrix or a quantity whose magnitude is one of these. Otherwise, it converts obj into a sparse representation and returns this. Additional requirements on the output format of the return value can be added using the optional parameters.

Parameters:

• obj (object) – The object that is to be converted into a sparse representation.

• dtype (dtype-like or None, optional) – The dtype that is passed to torch.as_tensor(obj) or np.asarray(obj).

• quant (bool or None, optional) – Whether the return value should be a pint.Quantity object wrapping wrapping the object (quant=True) or the object itself (quant=False). If quant is None (the default) then the return value is a quantity if obj is a quantity and is not a quantity if obj is not a quantity.

• ureg (pint.UnitRegistry or None, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then no specific coersion to a pint.UnitRegistry is performed (i.e., the same quantity class is returned).

• unit (unit-like, bool, None, or Ellipsis, optional) – The unit that should be used in the return value. When the return value of this function is a pint.Quantity (see the quant parameter), the returned quantity always has a unit matching the unit parameter; if the provided obj is not a quantity, then its unit is presumed to be those requested by unit. When the return value of this function is not a pint.Quantity object and is instead a numeric object, then when obj is not a quantity the unit parameter is ignored, and when obj is a quantity, its magnitude is returned after conversion into unit. The default value of unit, Ellipsis, indicates that, if obj is a quantity, its unit should be used, and unit should be considered to be None otherwise.

Returns:

A sparse version of the argument obj.

Return type:

sparse tensor, sparse array, or quantity with a sparse magnitude

to_tcoll(obj, /, copy=True)

Returns a transient copy of obj.

to_tcoll(obj) returns a copy of obj as a transient collection. If obj is not a collection that can be converted into a transient collection, then an error is raised.

Note

If obj is already a transient collection, then a copy of obj is returned.

Note

If to_tcoll is given a lazy dict (pcollections.ldict) or a lazi list (pcollections.llist), the resulting transient dictionary is made using obj.transient() and so respects the laziness of the elements.

Parameters:

• obj (collection) – An object that is to be converted into a persistent collection.

• copy (boolean, optional) – If obj is already a transient collection, then a copy is made if and only if copy is True; otherwise, obj is returned as-is when it is already a transient type. The default is True.

Returns:

A transient version of obj. The returned value is always either a tlist, tset, or tdict object.

Return type:

object

Raises:

TypeError – If obj cannot be converted into a transient collection.

to_tensor(obj, /, dtype=None, *, device=None, requires_grad=None, copy=False, sparse=None, quant=None, ureg=None, unit=Ellipsis)

Reinterprets obj as a PyTorch tensor or as a pint quantity with a tensor magnitude.

immlib.to_tensor is roughly equivalent to the torch.as_tensor function with a few exceptions:

• to_tensor(obj) allows quantities for obj and, in such a case, will return a quantity whose magnitude has been reinterpreted as a tensor, though this behavior can be altered with the quant parameter;

• to_tensor(obj) can convet a SciPy sparse matrix into a sparse tensor.

Parameters:

• obj (object) – The object that is to be reinterpreted as or covnerted to, a PyTorch tensor object.

• dtype (dtype-like, optional) – The dtype that is passed to torch.as_tensor(obj).

• device (device name or None, optional) – The device parameter that is passed to torch.as_tensor(obj), None by default.

• requires_grad (bool or None, optional) – Whether the returned tensor should require gradient calculations or not. If None (the default), then the objecct obj is not changed from its current gradient settings, if obj is a tensor, and obj is not made to track its gradient if it is converted into a tensor. If the requires_grad parameter does not match the given tensor’s requires_grad field, then a copy is always returned.

• copy (bool, optional) – Whether to copy the data in obj or not. If False, then obj is only copied if doing so is required by the optional parameters. If True, then obj is always copied if possible.

• sparse (bool, {'csr','csc','bsr','bsc','coo'}, or None, optional) – If None, then the sparsity of obj is the same as the sparsity of the tensor that is returned. Otherwise, the return value will always be either a spase tensor (sparse=True) or a dense tensor (sparse=False) based on the given value of sparse. The sparse parameter may also be set to the name of a sparse layout in order to convert the object into that layout. Possible sparse layouts include 'coo', 'csr', 'csc', 'bsr', and 'bsc'.

• quant (bool or None, optional) – Whether the return value should be a Quantity object wrapping the array (quant=True) or the tensor itself (quant=False). If quant is None (the default) then the return value is a quantity if either obj is a quantity or an explicit unit parameter is given and is not a quantity if obj is not a quantity.

• ureg (pint.UnitRegistry or None, optional) – The pint.UnitRegistry object to use for units. If ureg is Ellipsis, then immlib.units is used. If ureg is None (the default), then no specific coersion to a pint.UnitRegistry is performed (i.e., the specific subclass of pint.Quantity used by obj is not changed).

• unit (unit-like, bool, None, or Ellipsis, optional) – The unit that should be used in the return value. When the return value of this function is a pint.Quantity (see the quant parameter), the returned quantity always has units matching the unit parameter; if the provided obj is not a quantity, then its unit is presumed to be that requested by unit. When the return value of this function is not a pint.Quantity object and is instead a PyTorch tensor object, then when obj is not a quantity the unit parameter is ignored, and when obj is a quantity, its magnitude is returned after conversion into unit. The default value of unit, Ellipsis, indicates that, if obj is a quantity, its unit should be used, and unit should be considered dimensionless otherwise.

Returns:

Either a PyTorch tensor equivalent to obj or a pint.Quantity whose magnitude is a PyTorch tensor equivalent to obj.

Return type:

torch.Tensor or pint.Quantity

Raises:

ValueError – If invalid parameter values are given or if the parameters conflict.

See also

to_array, to_numeric

unit(obj, /, ureg=None)

Converts the argument into a a pint.Unit object.

unit(obj) returns the immlib-library unit object for the given unit object obj (which may be from a separate pint.UnitRegistry instance).

unit(unitname) returns the unit object for the given unit name string unitname.

unit(q) returns the unit of the given quantity object q.

Note

immlib considers an object to be “unit-like” if unit(obj) returns a valid pint.Unit object.

Parameters:

• obj (object) – The object that is to be converted to a unit.

• ureg (ping.UnitRegistry, None, or Ellipsis, optional) – The unit registry to convert the object into. If Ellipsis, then immlib.units is used. If None (the default), then the unit registry for obj is used if obj is a quantity or unit already, and immlib.units is used if not. Otherwise, must be a unit registry.

Returns:

The Unit object associated with the given argument.

Return type:

pint.Unit

Raises:

TypeError – When the argument cannot be converted to a pint.Unit object.

url_download(url, /, destpath=None, *, mkdirs=True, mkdir_mode=509, expanduser=True)

Returns the contents of the given URL as a byte-string.

url_download(url) returns the contents of the given url as a byte-string.

url_download(url, destpath) downloads the given url to the given destination path, destpath, and returns that path on success.

Parameters:

• url (str or URL) – The URL to be downloaded.

• destpath (PathLike or None, optional) – A string, pathlib.Path object, or any object that can be converted into a Path, which details the local destination path to which the URL should be saved. The default, None, indicates that the file should not be downloaded to a path but should instead just be returned as a byte string.

• mkdirs (boolean, optional) – Whether to make directories that do not exist in order to save the URL to the path destpath. The default is True.

• mkdir_mode (int, optional) – The mode to give any directory created by this function. The default is 0o775. If mkdirs is set to False, then this option is ignored.

• expanduser (bool, optional) – Whether to expand the ~ character into the user’s directory in the destination path. The default is True.

Returns:

If destpath is None, then a bytes object containing the URL contents is returned; otherwise, the pathlib.Path object representing the downloaded file is returned.

Return type:

bytes or Path

valmap(f, d, /, *args, **kwargs)

Returns a dictionary object whose values are transformed by a function.

valmap(f, d) returns a dict whose keys are the same as those of the given dict object and whose values, for each key k are f(d[k]).

valmap(f, d, *args, **kw) additionally passes the given arguments to the function f, such that in the resulting map, each key k is mapped to f(d[k], *args, **kw).

Unlike lazyvalmap, this function returns either a dict, a pdict, or an ldict depending on the input argument d. If d is a (lazy) ldict, then an ldict is returned; if d is a pdict, a pdict is returned, and otherwise, a dict is returnd.

Parameters:

• f (function) – The function used to create the values in the new dictionary.

• d (collections.abc.Mapping) – A mapping whose keys are to be preserved and remapped to a function of their values.

• args – Additional positional arguments to pass to f.

• kwargs – Additional named arguments to pass to f.

Returns:

This function always returns a dictionary object whose type is either pcollections.pdict, pcollections.ldict, or dict, depending on the type of d.

Return type:

collections.abc.Mapping object