decisionengine.framework.util package

Submodules

decisionengine.framework.util.countdown module

class decisionengine.framework.util.countdown.Countdown(wait_up_to)

Bases: object

Countdown is a context manager that keeps track of elapsed time.

It is designed to be used for cases where a sequence of operations should not take longer than a specified period of time. This is done by occasionally querying the ‘time_left’ attribute (e.g.):

countdown = Countdown(wait_up_to=10)
for p in processes:
    with countdown:
        rc = p.join(countdown.time_left)
        if rc is None:
            p.terminate()

In the above example, the time it takes to shutdown all processes should not exceed 10 seconds. Upon entering the countdown context, a timer starts. Once that context is exited, the timer stops and the elapsed time is subtracted from the initial ‘wait_up_to’ value. If it takes 10 seconds for the first process to join, then the ‘time_left’ value will be 0 when joining all subsequent processes. In this way, the entire sequence of operations is constrained to occur in roughly 10 seconds.

decisionengine.framework.util.fs module

decisionengine.framework.util.fs.files_with_extensions(dir_path, *extensions)

Return all files in dir_path that match the provided extensions.

If no extensions are given, then all files in dir_path are returned.

Results are sorted by channel name to ensure stable output.

decisionengine.framework.util.logparser module

decisionengine.framework.util.logparser.console_scripts_main(args_to_parse=None)

This is the entry point for the setuptools auto generated scripts. Setuptools thinks a return from this function is an error message.

decisionengine.framework.util.logparser.create_parser()

decisionengine.framework.util.logparser.execute_command_from_args(argsparsed, logfile=None, constraint=None)

Parse the log file as requested.

Parameters:

• argsparsed (Namespace) – Parsed arguments from create_parser in this file.

• logfile (path) – Log file path.

• constraint (dict) – Combined constraints dictionary

Returns:

Output of the command.

Return type:

str

decisionengine.framework.util.logparser.main(args_to_parse=None)

Main function for logparser

Parameters:

• args_to_parse (list, optional) – If you pass a list of args, they will be used instead of sys.argv.

• None. (Defaults to)

Returns:

Parsing result

Return type:

str

decisionengine.framework.util.logparser.matches_constraint(constraint, linelist, linedict)

Return True if all constraints are marched

Parameters:

• constraint (dict|None) – combined constraints

• linelist (list) – List of line fields

• linedict (dict) – Dictionary with structured elements

Returns:

True is all constraints are matched, False otherwise

Return type:

bool

decisionengine.framework.util.logparser.parse_constraints(constraints, loglevel=None)

Parse and combine the constraints

Parameters:

• constraints (list|None) – List of constraints

• loglevel (str) – Logging level, e.g. DEBUG, INFO, …

Returns:

combined constraint dictionary

Return type:

dict

decisionengine.framework.util.metrics module

class decisionengine.framework.util.metrics.Counter(name: str, documentation: str, labelnames: ~typing.Iterable[str] = (), namespace: str = '', subsystem: str = '', unit: str = '', registry: ~prometheus_client.registry.CollectorRegistry | None = <prometheus_client.registry.CollectorRegistry object>, _labelvalues: ~typing.Sequence[str] | None = None)

Bases: Counter

_abc_impl = <_abc._abc_data object>

class decisionengine.framework.util.metrics.Gauge(*args, **kwargs)

Bases: Gauge

Override prometheus client Gauge so that muliproccess_mode ‘liveall” is the default as opposed to ‘all’

_DEFAULT_MULTIPROC_MODE = 'liveall'

__determine_multiprocess_mode_existence(*args, **kwargs)

_abc_impl = <_abc._abc_data object>

class decisionengine.framework.util.metrics.Histogram(name: str, documentation: str, labelnames: ~typing.Iterable[str] = (), namespace: str = '', subsystem: str = '', unit: str = '', registry: ~prometheus_client.registry.CollectorRegistry | None = <prometheus_client.registry.CollectorRegistry object>, _labelvalues: ~typing.Sequence[str] | None = None, buckets: ~typing.Sequence[float | str] = (0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0, inf))

Bases: Histogram

_abc_impl = <_abc._abc_data object>

class decisionengine.framework.util.metrics.Summary(name: str, documentation: str, labelnames: ~typing.Iterable[str] = (), namespace: str = '', subsystem: str = '', unit: str = '', registry: ~prometheus_client.registry.CollectorRegistry | None = <prometheus_client.registry.CollectorRegistry object>, _labelvalues: ~typing.Sequence[str] | None = None)

Bases: Summary

_abc_impl = <_abc._abc_data object>

decisionengine.framework.util.metrics.display_metrics()

decisionengine.framework.util.reaper module

A stand-alone script purges data in database older than specified in configuration. Configuration file has to have this bit added:

{
    "dataspace" : {
        "retention_interval_in_days" : 365,
        "datasource" :  {}
    }
}

Can be used in a cron job.

decisionengine.framework.util.reaper.main()

decisionengine.framework.util.redis_stats module

decisionengine.framework.util.redis_stats.redis_stats(broker_url, exchange)

decisionengine.framework.util.singleton module

class decisionengine.framework.util.singleton.ScopedSingleton

Bases: Singleton

Singleton pattern using Metaclass with weak refs

_instances = <WeakValueDictionary>

class decisionengine.framework.util.singleton.ScopedSingletonABC(name, bases, namespace, **kwargs)

Bases: ABCMeta, ScopedSingleton

class decisionengine.framework.util.singleton.Singleton

Bases: type

Singleton pattern using Metaclass with strong refs

_instances = {}

class decisionengine.framework.util.singleton.SingletonABC(name, bases, namespace, **kwargs)

Bases: ABCMeta, Singleton

decisionengine.framework.util.sockets module

decisionengine.framework.util.sockets.get_random_port()

decisionengine.framework.util.subclasses module

decisionengine.framework.util.subclasses._derived_class(cls, base_class)

Only matches subclasses that are not equal to the base class.

decisionengine.framework.util.subclasses.all_subclasses(module, base_class)

Return all of a module’s subclasses of the given base class.

Module contents