spinetoolbox.project

Spine Toolbox project class.

Module Contents

Classes

ItemNameStatus	Generic enumeration.
SpineToolboxProject	Class for Spine Toolbox projects.

Functions

node_successors(g)	Returns a dict mapping nodes in topological order to a list of successors.
_ranks(node_successors)	Calculates node ranks.

class spinetoolbox.project.ItemNameStatus

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

OK

INVALID

EXISTS

SHORT_NAME_EXISTS

class spinetoolbox.project.SpineToolboxProject(toolbox, p_dir, plugin_specs, app_settings, settings, logger)

Bases: spinetoolbox.metaobject.MetaObject

Class for Spine Toolbox projects.

Parameters

• toolbox (ToolboxUI) – toolbox of this project

• p_dir (str) – Project directory

• plugin_specs (Iterable of ProjectItemSpecification) – specifications available as plugins

• app_settings (QSettings) – Toolbox settings

• settings (ProjectSettings) – project settings

• logger (LoggerInterface) – a logger instance

property all_item_names

property n_items

property settings

property connections

property app_settings

project_about_to_be_torn_down

Emitted before project is being torn down.

project_execution_about_to_start

Emitted just before the entire project is executed.

project_execution_finished

Emitted after the entire project execution finishes.

connection_established

Emitted after new connection has been added to project.

connection_about_to_be_removed

Emitted before connection removal.

connection_updated

Emitted after a connection has been updated.

jump_added

Emitted after a jump has been added.

jump_about_to_be_removed

Emitted before a jump is removed.

jump_updated

Emitted after a jump has been replaced by another.

item_added

Emitted after a project item has been added.

item_about_to_be_removed

Emitted before project item removal.

item_renamed

Emitted after project item has been renamed.

specification_added

Emitted after a specification has been added.

specification_about_to_be_removed

Emitted before a specification will be removed.

specification_replaced

Emitted after a specification has been replaced.

specification_saved

Emitted after a specification has been saved.

LOCAL_EXECUTION_JOB_ID = '1'

toolbox()

Returns Toolbox main window.

Returns

main window

Return type

ToolboxUI

has_items()

Returns True if project has project items.

Returns

True if project has items, False otherwise

Return type

bool

get_item(name)

Returns project item.

Parameters

name (str) – Item’s name

Returns

Project item

Return type

ProjectItem

get_items()

Returns all project items.

Returns

All project items

Return type

list of ProjectItem

get_items_by_type(_type)

Returns all project items with given _type.

Parameters

_type (str) – Project Item type

Returns

Project Items with given type or an empty list if none found

Return type

list of ProjectItem

_create_project_structure(directory)

Makes the given directory a Spine Toolbox project directory. Creates directories and files that are common to all projects.

Parameters

directory (str) – Abs. path to a directory that should be made into a project directory

Returns

True if project structure was created successfully, False otherwise

Return type

bool

call_set_description(description)

set_description(description)

Set object description.

Parameters

description (str) – Object description

save()

Collects project information and objects into a dictionary and writes it to a JSON file.

_save_all_specifications(local_path)

Writes all specifications except plugins to disk.

Local specification data is also written to disk, including local data from plugins.

Parameters

local_path (Path) –

Returns

specification local data that is supposed to be stored in a project specific place

Return type

dict

_pop_local_data_from_items_dict(items_dict)

Pops local data from project items dict.

Parameters

items_dict (dict) – items dict

Returns

local project item data

Return type

dict

static _dump(target_dict, out_stream)

Dumps given dict into output stream.

Parameters

• target_dict (dict) – dictionary to dump

• out_stream (IOBase) – output stream

load(spec_factories, item_factories)

Loads project from its project directory.

Parameters

• spec_factories (dict) – Dictionary mapping specification name to ProjectItemSpecificationFactory

• item_factories (dict) – mapping from item type to ProjectItemFactory

Returns

True if the operation was successful, False otherwise

Return type

bool

static _merge_local_data_to_project_info(local_data_dict, project_info)

Merges local data into project info.

Parameters

• local_data_dict (dict) – local data

• project_info (dict) – project dict

connection_from_dict(connection_dict)

jump_from_dict(jump_dict)

add_specification(specification, save_to_disk=True)

Adds a specification to the project.

Parameters

• specification (ProjectItemSpecification) – specification to add

• save_to_disk (bool) – if True, save the specification to disk

Returns

A unique identifier for the specification or None if the operation was unsuccessful

Return type

int

is_specification_name_reserved(name)

Checks if specification exists.

Parameters

name (str) – specification’s name

Returns

True if project has given specification, False otherwise

Return type

bool

specifications()

Yields project’s specifications.

Yields

ProjectItemSpecification – specification

_specification_id()

Creates an id for specification.

Returns

new id

Return type

int

get_specification(name_or_id)

Returns project item specification.

Parameters

name_or_id (str or int) – specification’s name or id

Returns

specification or None if specification was not found

Return type

ProjectItemSpecification

specification_name_to_id(name)

Returns identifier for named specification.

Parameters

name (str) – specification’s name

Returns

specification’s id or None if no such specification exists

Return type

int

remove_specification(id_or_name)

Removes a specification from project.

Parameters

id_or_name (int or str) – specification’s id or name

replace_specification(name, specification, save_to_disk=True)

Replaces an existing specification.

Refreshes the spec in all items that use it.

Parameters

• name (str) – name of the specification to replace

• specification (ProjectItemSpecification) – a specification

• save_to_disk (bool) – If True, saves the given specification to disk

Returns

True if operation was successful, False otherwise

Return type

bool

save_specification_file(specification, previous_name=None)

Saves the given project item specification.

Save path is determined by specification directory and specification’s name.

Parameters

• specification (ProjectItemSpecification) – specification to save

• previous_name (str, optional) – specification’s earlier name if it has been renamed/replaced

Returns

True if operation was successful, False otherwise

Return type

bool

_update_specification_local_data_store(specification, local_data, previous_name)

Updates the file containing local data of project’s specifications.

Parameters

• specification (ProjectItemSpecification) – specification

• local_data (dict) – local data serialized into dict

• previous_name (str, optional) – specification’s earlier name if it has been renamed/replaced

_default_specification_file_path(specification)

Determines a path inside project directory to save a specification.

Parameters

specification (ProjectItemSpecification) – specification

Returns

valid path or None if operation failed

Return type

str

add_item(item)

Adds a project item to project.

Parameters

item (ProjectItem) – item to add

rename_item(previous_name, new_name, rename_data_dir_message)

Renames a project item

Parameters

• previous_name (str) – item’s current name

• new_name (str) – item’s new name

• rename_data_dir_message (str) – message to show when renaming item’s data directory

Returns

True if item was renamed successfully, False otherwise

Return type

bool

validate_project_item_name(name)

Validates item name.

Parameters

name (str) – proposed project item’s name

Returns

validation result

Return type

ItemNameStatus

find_connection(source_name, destination_name)

Searches for a connection between given items.

Parameters

• source_name (str) – source item’s name

• destination_name (str) – destination item’s name

Returns

connection instance or None if there is no connection

Return type

Connection

connections_for_item(item_name)

Returns connections that have given item as source or destination.

Parameters

item_name (str) – item’s name

Returns

connections connected to item

Return type

list of Connection

add_connection(*args, silent=False, notify_resource_changes=True)

Adds a connection to the project.

A single argument is expected to be the Logging connection instance. Optionally, source name and position, and destination name and position can be provided instead.

Parameters

• *args – connection to add

• silent (bool) – If False, prints ‘Link establ…’ msg to Event Log

• notify_resource_changes (bool) – If True, updates resources of successor and predecessor items

Returns

True if connection was added successfully, False otherwise

Return type

bool

_notify_rsrc_changes(destination, source)

Notifies connection destination and connection source item that resources may have changed.

Parameters

• destination (ProjectItem) – Destination item

• source (ProjectItem) – Source item

remove_connection(connection)

Removes a connection from the project.

Parameters

connection (LoggingConnection) – connection to remove

update_connection(connection, source_position, destination_position)

Updates existing connection between items.

Updating does not trigger any updates to the DAG or project items.

Parameters

• connection (LoggingConnection) – connection to update

• source_position (str) – link’s position on source item’s icon

• destination_position (str) – link’s position on destination item’s icon

jumps_for_item(item_name)

Returns jumps that have given item as source or destination.

Parameters

item_name (str) – item’s name

Returns

jumps connected to item

Return type

list of Jump

add_jump(jump, silent=False)

Adds a jump to project.

Parameters

• jump (Jump) – jump to add

• silent (bool) – if True, don’t log messages

find_jump(source_name, destination_name)

Searches for a jump between given items.

Parameters

• source_name (str) – source item’s name

• destination_name (str) – destination item’s name

Returns

connection instance or None if there is no jump

Return type

Jump

remove_jump(jump)

Removes a jump from the project.

Parameters

jump (Jump) – jump to remove

update_jump(jump, source_position, destination_position)

Updates an existing jump between items.

Parameters

• jump (LoggingJump) – jump to update

• source_position (str) – link’s position on source item’s icon

• destination_position (str) – link’s position on destination item’s icon

_update_jump_icons()

Updates icons for all jumps in the project.

jump_issues(jump)

Checks if jump is OK.

Parameters

jump (Jump) – jump to check

Returns

list of issues, if any

Return type

list of str

_dag_iterator()

Iterates directed graphs in the project.

Yields

DiGraph

dag_with_node(node)

Returns the DiGraph that contains the given node (project item) name (str).

restore_project_items(items_dict, item_factories)

Restores project items from dictionary.

Parameters

• items_dict (dict) – a mapping from item name to item dict

• item_factories (dict) – a mapping from item type to ProjectItemFactory

remove_item_by_name(item_name, delete_data=False)

Removes project item by its name.

Parameters

• item_name (str) – Item’s name

• delete_data (bool) – If set to True, deletes the directories and data associated with the item

execute_dags(dags, execution_permits_list, msg)

Executes given dags.

Parameters

• dags (list of DiGraph) – List of DAGs

• execution_permits_list (list of dict) –

• msg (str) – Message to log before execution

_execute_dags(dags, execution_permits_list)

create_engine_worker(dag, execution_permits, dag_identifier, settings, job_id)

Creates and returns a SpineEngineWorker to execute given validated dag.

Parameters

• dag (DiGraph) – The dag

• execution_permits (dict) – mapping item names to a boolean indicating whether to execute it or skip it

• dag_identifier (str) – A string identifying the dag, for logging

• settings (dict) – project and app settings to send to the spine engine.

• job_id (str) – job id

Returns

SpineEngineWorker

_handle_engine_worker_finished(worker)

execute_selected(names)

Executes DAGs corresponding to given project items.

Parameters

names (Iterable of str) – Names of selected items

_split_to_subdags(dag, selected_items)

Checks if given dag contains weakly connected components. If it does, the weakly connected components are split into their own (sub)dags. If not, the dag is returned unaltered.

Parameters

• dag (DiGraph) – Dag that is checked if it needs to be split into subdags

• selected_items (list) – Names of selected items

Returns

List of dags, ready for execution

Return type

list of DiGraph

execute_project()

Executes all dags in the project.

_validate_dags(dags)

Validates dags and logs error messages.

Parameters

dags (Iterable) – dags to validate

Returns

validated dag

Return type

list

stop()

Stops execution.

notify_resource_changes_to_predecessors(item)

Updates resources for direct predecessors of given item.

Parameters

item (ProjectItem) – item whose resources have changed

_update_incoming_connection_and_jump_resources(item_name, trigger_resources)

notify_resource_changes_to_successors(item)

Updates resources for direct successors and outgoing connections of given item.

Parameters

item (ProjectItem) – item whose resources have changed

_update_outgoing_connection_and_jump_resources(item_name, trigger_resources)

_notify_resource_changes(trigger_name, target_names, provider_connections, update_resources, trigger_resources)

Updates resources in given direction for immediate neighbours of an item.

Parameters

• trigger_name (str) – item whose resources have changed

• target_names (Iterable of str) – items to be notified

• provider_connections (Callable) – function that receives a target item name and returns a list of Connections from resource providers

• update_resources (Callable) – function that takes an item name, a list of provider names, and a dictionary of resources, and does the updating

• trigger_resources (list of ProjectItemResource) – resources from the trigger item

notify_resource_replacement_to_successors(item, old, new)

Replaces resources for direct successors and outgoing connections of given item.

Parameters

• item (ProjectItem) – item whose resources have changed

• old (list of ProjectItemResource) – old resource

• new (list of ProjectItemResource) – new resource

notify_resource_replacement_to_predecessors(item, old, new)

Replaces resources for direct predecessors.

Parameters

• item (ProjectItem) – item whose resources have changed

• old (list of ProjectItemResource) – old resources

• new (list of ProjectItemResource) – new resources

_update_item_resources(target_item, direction)

Updates up or downstream resources for a single project item. Called in both directions after removing a Connection.

Parameters

• target_item (ProjectItem) – item whose resource need update

• direction (ExecutionDirection) – FORWARD updates resources from upstream, BACKWARD from downstream

predecessor_names(name)

Collects direct predecessor item names.

Parameters

name (str) – name of the project item whose predecessors to collect

Returns

direct predecessor names

Return type

set of str

successor_names(name)

Collects direct successor item names.

Parameters

name (str) – name of the project item whose successors to collect

Returns

direct successor names

Return type

set of str

descendant_names(name)

Yields descendant item names.

Parameters

name (str) – name of the project item whose descendants to collect

Yields

str – descendant name

outgoing_connections(name)

Collects outgoing connections.

Parameters

name (str) – name of the project item whose connections to collect

Returns

outgoing connections

Return type

set of Connection

_outgoing_jumps(name)

Collects outgoing jumps.

Parameters

name (str) – name of the project item whose jumps to collect

Returns

outgoing jumps

Return type

set of Jump

_outgoing_connections_and_jumps(name)

Collects outgoing connections and jumps.

Parameters

name (str) – name of the project item whose connections and jumps to collect

Returns

outgoing connections and jumps

Return type

set of Connection/Jump

incoming_connections(name)

Collects incoming connections.

Parameters

name (str) – name of the project item whose connections to collect

Returns

incoming connections

Return type

set of Connection

_incoming_jumps(name)

Collects incoming jumps.

Parameters

name (str) – name of the project item whose jumps to collect

Returns

incoming jumps

Return type

set of Jump

_incoming_connections_and_jumps(name)

Collects incoming connections and jumps.

Parameters

name (str) – name of the project item whose connections and jumps to collect

Returns

incoming connections

Return type

set of Connection/Jump

_update_successor(successor, incoming_connections, resource_cache)

_update_predecessor(predecessor, outgoing_connections, resource_cache)

_is_dag_valid(dag)

_update_ranks(dag)

prepare_remote_execution()

Pings the server and sends the project as a zip-file to server.

Returns

Job Id if server is ready for remote execution, empty string if something went wrong

or LOCAL_EXECUTION_JOB_ID if local execution is enabled.

Return type

str

finalize_remote_execution(job_id)

Sends a request to server to remove the project directory. In addition, removes the project ZIP file from client machine.

Parameters

job_id (str) – job id

tear_down()

Cleans up project.

spinetoolbox.project.node_successors(g)

Returns a dict mapping nodes in topological order to a list of successors.

Parameters

g (DiGraph) –

Returns

dict

spinetoolbox.project._ranks(node_successors)

Calculates node ranks.

Parameters

node_successors (dict) – a mapping from successor name to a list of predecessor names

Returns

a mapping from node name to rank

Return type

dict