Bulk Run

This stores BulkRun class used for running multiple concurrent simulations

Written By: Matthew Stadelman

Date Written: 2016/06/16

Last Modifed: 2016/06/16

class apmapflow.run_model.bulk_run.BulkRun(init_input_file, num_CPUs=2, sys_RAM=4.0, **kwargs)

Handles generating a collection of input files from the provided parameters and then running multiple instances of the LCL model concurrently to produce simulation data for each simulation parameter combination. A comprehensive example of this class and the associated script is avilable under the Usage Examples page.

Parameters:

• init_input_file (apmapflow.run_model.InputFile) – An inital InputFile instance to define the static parameters of the bulk run.
• num_CPUs (int, optional) – The maximum number of CPUs to utilize
• sys_RAM (float, optional) – The maximum amount of RAM avilable for use.
• **kwargs (multiple) –

  • delim : string

    The expected delimiter in the aperture map files

  • spawn_delay : float

    The minimum time between spawning of new LCL instances in seconds

  • retest_delay : float

    The time to wait between checking for completed processes.

Examples

>>> from apmapflow import BulkRun, InputFile
>>> inp_file = InputFile('./input-file-path.inp')
>>> blk_run = BulkRun(inp_file, num_CPUs=16, sys_RAM=32.0, spawn_delay=10.0)

Notes

spawn_delay is useful to help ensure shared resources are not accessed at the same time.

__init__(init_input_file, num_CPUs=2, sys_RAM=4.0, **kwargs)

Setting properties of the class.

__weakref__

list of weak references to the object (if defined)

static _check_processes(processes, RAM_in_use, retest_delay=5, **kwargs)

Checks the list of currently running processes for any that have completed removing and them from a list. If no processes have completed then the routine sleep for a specified amount of time before checking again.

Parameters:

• processes (list of Popen instances) – The list of processes to curate.
• RAM_in_use (list of floats) – The list of maximum RAM each process is estimated to use.
• retest_delay (floats) – The time delay between testing for completed processes.

static _combine_run_params(run_params)

Generates all possible unique combinations from a set of parameter arrays.

Parameters:run_params (dictionary) – A dictionary of parameter lists to combine together Returns:parameter combinations – A list of dictionaries where each parameter only has a single value Return type:dictionary

_initialize_run()

Assesses RAM requirements of each aperture map in use and registers the value with the InputFile instance. This RAM measurement is later used when determining if there is enough space available to begin a simulation.

_start_simulations(processes, RAM_in_use, spawn_delay=5, **kwargs)

This starts additional simulations if there is enough free RAM and avilable CPUs.

Parameters:

• processes (list of Popen instances) – The list of processes to add any new simulations to.
• RAM_in_use (list of floats) – The list of maximum RAM to a new simulations requirement to.
• spawn_delay (floats) – The time delay between spawning of processes.

dry_run()

Steps through the entire simulation creating directories and input files without actually starting any of the simulations. This Allows the LCL input files to be inspected before actually starting the run.

Examples

>>> from apmapflow import BulkRun, InputFile
>>> inp_file = InputFile('./input-file-path.inp')
>>> blk_run = BulkRun(inp_file, num_CPUs=16, sys_RAM=32.0, spawn_delay=10.0)
>>> blk_run.dry_run()

See also

start()

generate_input_files(default_params, default_name_formats, case_identifer='', case_params=None, append=False)

Generates the input file list based on the default parameters and case specific parameters. An InputFile instance is generated for each unique combination of model parameters which is then written to disk to be run by the LCL model.

Parameters:

• default_params (dictionary) – A dictionary containing lists of parameter values to use in the simulations.
• default_name_formats (dictionary) – A dictionary containing the infile and outfile name formats to use.
• case_identifer (string, optional) – A format string used to identify cases that need special parameters
• case_params (dictionary, optional) – A dictionary setup where each key is an evaluation of the case_identifer format string and the value is a dictionary containing lists of parameter values used to update the default params for that case.
• append (boolean, optional) – When True the BulkRun.input_file_list attribute is appended to instead of reset by this method.

Notes

The default_name_formats parameter is passed directly to the InputFile instance initialization and is no modified in any way. When using a case_identifier only the evaluations that matter need to be added to the case_params dictionary. Missing permuatations of the identifer are

start()

Starts the bulk run, first creating the input files and then managing the multiple processes until all input files have been processed. The input file list must have already been generated prior to calling this method.

See also

generate_input_files()