Executing Projects

This section describes how executing a project works. Execution happens by pressing the or the buttons in the main window tool bar. A project consists of project items and connections (yellow arrows) that are visualized on the Design View. You use the project items and the connections to build a Directed Acyclic Graph (DAG), with the project items as nodes and the connections as edges. A DAG is traversed using the breadth-first-search algorithm.

Rules of DAGs:

1. A single project item with no edges is a DAG.
2. All project items that are connected, are considered as a single DAG. It does not matter, which direction the edges go. If there is a path between two nodes, they are considered as belonging to the same DAG.
3. Loops are not allowed (this is what acyclic means).

You can connect the nodes in the Design View how ever you want but you cannot execute the resulting DAGs if they break the rules above. Here is an example project with three DAGs.

• DAG 1: nodes: a, b, c, d. edges: a-b, a-c, b-d, c-d
• DAG 2: nodes: e, f. edges: e-f
• DAG 3: nodes: g. edges: None

When you press the button, all three DAGs are executed in a row. You can see the progress and the current executed node in the Event Log. Execution order of DAG 1 is a->b->c->d or a->c->b->d since nodes b and c are siblings. DAG 2 execution order is e->f and DAG 3 is just g. If you have a DAG in your project that breaks the rules above, that DAG is skipped and the execution continues with the next DAG.

You can execute a single DAG in the project by selecting a node in the desired DAG and pressing the button in the tool bar. For example, to execute only DAG 1 you can select (in Design View or in the project item list in Project dock widget) any of the nodes a, b, c, or d and then press the button.

Tip

If you are not sure how execution works in your DAG, you can test the execution just like in the above picture by adding and connecting empty Data Connection items and then pressing the play buttons.

Executing Tools as a part of the DAG

All project items in the DAG are executed, but the real processing only happens when a Tool project item is executed. When you have created at least one Tool template, you can execute a Tool as part of the DAG. The Tool template defines the process that is depicted by the Tool project item. As an example, below we have two project items; Julia Model Tool and Data File Data Connection connected as a DAG with an edge between the nodes.

Selecting the Julia Model shows its properties in the Properties dock widget. In the top of the Tool Properties, there is a template drop-down menu. From this drop-down menu, you can select the Tool template for this particular Tool item. The Example Tool Template tool template has been selected for the Tool Julia Model. Below the drop-down menu, you can see the details of the Tool template, command line arguments, Source files (the first one is the main program file), Input files, Optional input files and Output files. Results… button opens the Tool’s result archive directory in the File Explorer (all Tools have their own result directory). The Execute in radio buttons control, whether this Tool is first copied to a work directory and executed there, or if the execution should happen in the source directory where the main program file is located.

Note

The Example Tool Template has been modified a bit from previous sections. It now requires an Input file data.csv.

When you click on the button, the execution starts from the Data File Data Connection. When executed, Data Connection items advertise their files and references to project items that are in the same DAG and executed after them. In this particular example, the Data File item contains a file called data.csv as depicted in the below picture.

When it’s the Julia Model tools turn to be executed, it checks if it finds the file data.csv from project items, that have already been executed. When the DAG is set up like this, the Tool finds the input file that it requires and then starts processing the Tool template starting with the main program file script.jl. Note that if the edge would be the other way around (from Julia Model to Data File) execution would start from the Julia Model and it would fail because it cannot find the required file data.csv. The same thing happens if there is no edge between the two project items. In this case the project items would be in separate DAGs.

Since the Tool template type was set as Julia and the main program is a Julia script, Spine Toolbox starts the execution in the Julia Console (if you have selected this in the application Settings, See Settings section).

Tool execution algorithm

The below figure depicts what happens when a Tool item with a valid Tool template is executed.