decisionengine.framework.modules package

Subpackages

• decisionengine.framework.modules.tests package
  • Submodules
  • decisionengine.framework.modules.tests.test_EmptySource module
    • test_empty_source_structure()
    • test_missing_data_product_name_not_supported()
  • decisionengine.framework.modules.tests.test_Module module
    • test_module_structure()
  • decisionengine.framework.modules.tests.test_Publisher module
    • test_publisher_structure()
  • decisionengine.framework.modules.tests.test_QueueLogger module
    • handler_setup()
    • log_setup()
    • queue_logger_setup()
    • setup_queue_logging()
    • test_setup_queue_logging()
    • test_start_queue_logger()
    • test_stop_queue_logger()
  • decisionengine.framework.modules.tests.test_Source module
    • test_source_structure()
  • decisionengine.framework.modules.tests.test_Transform module
    • test_transform_structure()
  • decisionengine.framework.modules.tests.test_de_logger module
    • log_setup()
    • test_by_nonsense_is_err()
    • test_by_size()
    • test_by_time()
  • decisionengine.framework.modules.tests.test_module_decorators module
    • test_multiple_consumes_declarations()
    • test_multiple_produces_declarations()
    • test_supports_config()
    • test_wrong_product_names()
    • test_wrong_product_types()
  • decisionengine.framework.modules.tests.test_translate_product_name module
    • test_translate_all()
    • test_translate_illegal_characters()
    • test_translate_none()
    • test_translate_simple()
    • test_translate_with_underscores()
  • Module contents

Submodules

decisionengine.framework.modules.EmptySource module

This dummy source takes the name of a source datablock from config file as parameter “data_product_name” and produces an empty pandas DataFrame as a datablock with that name

class decisionengine.framework.modules.EmptySource.EmptySource(config)

Bases: Source

_supported_config = {'data_product_name': (<class 'str'>, '', None)}

acquire()

decisionengine.framework.modules.Module module

class decisionengine.framework.modules.Module.Module(set_of_parameters)

Bases: object

A skeleton of a module

get_data_block()

get_parameters()

set_data_block(data_block)

decisionengine.framework.modules.Module.consumes(**kwargs)

decisionengine.framework.modules.Module.produces(**kwargs)

decisionengine.framework.modules.Module.verify_products(producer, data)

decisionengine.framework.modules.Publisher module

class decisionengine.framework.modules.Publisher.Parameter(name, type=None, default=None, comment=None)

Bases: object

class decisionengine.framework.modules.Publisher.Publisher(set_of_parameters)

Bases: Module

_consumes = {}

publish(data_block=None)

shutdown()

decisionengine.framework.modules.Publisher.consumes(**kwargs)

decisionengine.framework.modules.Publisher.describe(cls, program_options=<class 'decisionengine.framework.modules.describe.ModuleProgramOptions'>)

decisionengine.framework.modules.Publisher.supports_config(*args)

decisionengine.framework.modules.QueueLogger module

class decisionengine.framework.modules.QueueLogger.QueueLogger

Bases: object

configure_listener(handlers)

format_logger(logger)

initialize_q()

setup_queue_logging(logger, handlers)

start()

stop()

decisionengine.framework.modules.Source module

class decisionengine.framework.modules.Source.Parameter(name, type=None, default=None, comment=None)

Bases: object

class decisionengine.framework.modules.Source.Source(set_of_parameters)

Bases: Module

_produces = {}

acquire()

decisionengine.framework.modules.Source.describe(cls, sample_config=None)

decisionengine.framework.modules.Source.produces(**kwargs)

decisionengine.framework.modules.Source.supports_config(*args)

decisionengine.framework.modules.Transform module

class decisionengine.framework.modules.Transform.Parameter(name, type=None, default=None, comment=None)

Bases: object

class decisionengine.framework.modules.Transform.Transform(set_of_parameters)

Bases: Module

_consumes = {}

_produces = {}

transform()

decisionengine.framework.modules.Transform.consumes(**kwargs)

decisionengine.framework.modules.Transform.describe(cls, program_options=<class 'decisionengine.framework.modules.describe.ModuleProgramOptions'>)

decisionengine.framework.modules.Transform.produces(**kwargs)

decisionengine.framework.modules.Transform.supports_config(*args)

decisionengine.framework.modules.de_logger module

Logger to use in all modules

decisionengine.framework.modules.de_logger.configure_logging(log_level='DEBUG', file_rotate_by='size', rotation_time_unit='D', rotation_interval=1, max_backup_count=6, max_file_size=200000000, log_file_name='/tmp/decision_engine_logs/decisionengine.log', start_q_logger='True')

Parameters:

• log_level (str) – log level

• file_rotate_by – files rotation by size or by time

• rotation_time_unit (str) – unit of time for file rotation

• rotation_interval (int) – time in rotation_time_units between file rotations

• log_file_name (str) – log file name

• max_file_size (int) – maximal size of log file. If reached save and start new log.

• max_backup_count (int) – start rotation after this number is reached

Return type:

None

decisionengine.framework.modules.de_logger.get_logger()

get default logger - “decisionengine” :rtype: logging.Logger - rotating file logger

decisionengine.framework.modules.de_logger.get_queue_logger()

get QueueLogger which owns the logging queues and listeners :rtype: decisionengine.framework.modules.QueueLogger`

decisionengine.framework.modules.de_logger.stop_queue_logger()

decisionengine.framework.modules.describe module

class decisionengine.framework.modules.describe.ModuleProgramOptions(module_spec, cls)

Bases: object

process_args()

class decisionengine.framework.modules.describe.Parameter(name, type=None, default=None, comment=None)

Bases: object

decisionengine.framework.modules.describe._par_default(par_type, default_value)

decisionengine.framework.modules.describe._par_type(par_type, default_value)

decisionengine.framework.modules.describe.main_wrapper(cls, program_options=<class 'decisionengine.framework.modules.describe.ModuleProgramOptions'>)

decisionengine.framework.modules.describe.supports_config(*args)

decisionengine.framework.modules.logging_configDict module

Global Logger config dictionary used by all loggers (in their own subkeys)

decisionengine.framework.modules.print_description module

decisionengine.framework.modules.print_description._print_comment(comment)

decisionengine.framework.modules.print_description._print_type(type_or_value)

decisionengine.framework.modules.print_description._print_value(v)

decisionengine.framework.modules.print_description._spec_from_file_name(filename)

decisionengine.framework.modules.print_description.print_consumes(cls)

decisionengine.framework.modules.print_description.print_produces(cls)

decisionengine.framework.modules.print_description.print_supported_config(module_spec, cls)

decisionengine.framework.modules.print_description.spec_if_main(cls)

decisionengine.framework.modules.translate_product_name module

decisionengine.framework.modules.translate_product_name.translate(spec)

Break apart the string ‘old -> new’ into a tuple (‘old’, ‘new’)

decisionengine.framework.modules.translate_product_name.translate_all(specs)

Module contents