spinetoolbox.project_items.tool.executable_item

Contains Tool’s executable item and support functionality.

authors:

1. Soininen (VTT)

date:

30.3.2020

Module Contents

Classes

ExecutableItem	Tool project item’s executable parts.
_ExecutionToken	A token that acts as a callback after the tool process has finished execution.

Functions

_count_files_and_dirs(paths)	Counts the number of files and directories in given paths.
_create_output_dir_timestamp()	Creates a new timestamp string that is used as Tool output
_database_urls_from_resources(resources)	Pries database URLs and their providers’ names from resources.
_execution_directory(work_dir, tool_specification)	Returns the path to the execution directory, depending on execute_in_work.
_find_files_in_pattern(pattern, available_file_paths)	Returns a list of files that match the given pattern.
_unique_dir_name(tool_specification)	Builds a unique name for Tool’s work directory.

class spinetoolbox.project_items.tool.executable_item.ExecutableItem(name, work_dir, output_dir, tool_specification, cmd_line_args, logger)

Bases: spinetoolbox.executable_item_base.ExecutableItemBase

Tool project item’s executable parts.

Parameters:

• name (str) – item’s name
• work_dir (str) – an absolute path to Spine Toolbox work directory or None if the Tool should not execute in work directory
• output_dir (str) – path to the directory where output files should be archived
• tool_specification (ToolSpecification) – a tool specification
• cmd_line_args (list) – a list of command line argument to pass to the tool instance
• logger (LoggerInterface) – a logger

static item_type()

Returns the item’s type identifier string.

execution_finished(self, execution_token, return_code, execution_dir)

Handles things after execution has finished.

stop_execution(self)

Stops executing this Tool.

_copy_input_files(self, paths, execution_dir)

Copies input files from given paths to work or source directory, depending on where the Tool specification requires them to be.

Parameters:

• paths (dict) – key is path to destination file, value is path to source file
• execution_dir (str) – absolute path to the execution directory

Returns:

True if the operation was successful, False otherwise

Return type:

bool

_copy_optional_input_files(self, paths)

Copies optional input files from given paths to work or source directory, depending on where the Tool specification requires them to be.

Parameters:paths (dict) – key is the source path, value is the destination path

_copy_output_files(self, target_dir, execution_dir)

Copies Tool specification output files from work directory to given target directory.

Parameters:

• target_dir (str) – Destination directory for Tool specification output files
• execution_dir (str) – path to the execution directory

Returns:

Contains two lists. The first list contains paths to successfully copied files. The second list contains paths (or patterns) of Tool specification output files that were not found.

Return type:

tuple

Raises:

OSError – If creating a directory fails.

_copy_program_files(self, execution_dir)

Copies Tool specification source files to base directory.

_create_input_dirs(self, execution_dir)

Iterates items in required input files and check if there are any directories to create. Create found directories directly to work or source directory.

Parameters:execution_dir (str) – the execution directory Returns:True if the operation was successful, False otherwiseBoolean variable depending on success Return type:bool

_create_output_dirs(self, execution_dir)

Makes sure that work directory has the necessary output directories for Tool output files. Checks only “outputfiles” list. Alternatively you can add directories to “inputfiles” list in the tool definition file.

Parameters:execution_dir (str) – a path to the execution directory Returns:True for success, False otherwise. Return type:bool Raises:OSError – If creating an output directory to work fails.

_execute_backward(self, resources)

Stores resources for forward execution.

_execute_forward(self, resources)

Executes the Tool according to the Tool specification.

Before launching the tool script in a separate instance, prepares the execution environment by creating all necessary directories and copying input files where needed. After execution archives the output files.

Parameters:resources (list) – a list of resources from direct predecessor items Returns:True if execution succeeded, False otherwise

_find_input_files(self, resources)

Iterates required input files in tool specification and looks for them in the given resources.

Parameters:resources (list) – resources available Returns:Dictionary mapping required files to path where they are found, or to None if not found

_find_optional_input_files(self, resources)

Tries to find optional input files from previous project items in the DAG.

Parameters:resources (list) – resources available Returns:

Dictionary of optional input file paths or an empty dictionary if no files found. Key is the

optional input item and value is a list of paths that matches the item.

Return type:dict

_handle_output_files(self, return_code, execution_dir)

Copies Tool specification output files from work directory to result directory.

Parameters:

• return_code (int) – Tool specification process return value
• execution_dir (str) – path to the execution directory

_optional_output_destination_paths(self, paths, execution_dir)

Returns a dictionary telling where optional output files should be copied to before execution.

Parameters:

• paths (dict) – key is the optional file name pattern, value is a list of paths to source files
• execution_dir (str) – a path to the execution directory

Returns:

a map from source path to destination path

Return type:

dict

_output_resources_forward(self)

Returns a list of resources, i.e. the output files produced by the tool.

Returns the files that were actually created during the execution. The URL points to the archive directory.

Returns:a list of Tool’s output resources Return type:list

classmethod from_dict(cls, item_dict, name, project_dir, app_settings, specifications, logger)

See base class.

spinetoolbox.project_items.tool.executable_item._count_files_and_dirs(paths)

Counts the number of files and directories in given paths.

Parameters:paths (list) – list of paths Returns:Tuple containing the number of required files and directories.

spinetoolbox.project_items.tool.executable_item._create_output_dir_timestamp()

Creates a new timestamp string that is used as Tool output directory.

Returns:Timestamp string or empty string if failed.

spinetoolbox.project_items.tool.executable_item._database_urls_from_resources(resources)

Pries database URLs and their providers’ names from resources.

Parameters:resources (list) – a list of ProjectItemResource objects Returns:a mapping from resource provider’s name to a database URL. Return type:dict

spinetoolbox.project_items.tool.executable_item._execution_directory(work_dir, tool_specification)

Returns the path to the execution directory, depending on execute_in_work.

If execute_in_work is True, a new unique path will be returned. Otherwise, the main program file path from tool specification is returned.

Returns:a full path to next basedir Return type:str

spinetoolbox.project_items.tool.executable_item._find_files_in_pattern(pattern, available_file_paths)

Returns a list of files that match the given pattern.

Parameters:

• pattern (str) – file pattern
• available_file_paths (list) – list of available file paths from upstream items

Returns:

List of (full) paths

Return type:

list

spinetoolbox.project_items.tool.executable_item._unique_dir_name(tool_specification)

Builds a unique name for Tool’s work directory.

class spinetoolbox.project_items.tool.executable_item._ExecutionToken(tool_executable, execution_dir)

A token that acts as a callback after the tool process has finished execution.

Parameters:

• tool_executable (spinetoolbox.project_items.tool.executable_item.ExecutableItem) – the object that has initiated the execution
• execution_dir (str) – absolute path to the execution working directory

handle_execution_finished(self, return_code)

Handles Tool specification execution finished.

Parameters:return_code (int) – Process exit code