spinetoolbox.spine_io.exporters.gdx

For exporting a database to GAMS .gdx file.

Currently, this module supports databases that are “GAMS-like”, that is, they follow the EAV model but the object classes, objects, relationship classes etc. directly reflect the GAMS data structures. Conversions e.g. from Spine model to TIMES are not supported at the moment.

This module contains low level functions for reading a database into an intermediate format and for writing that intermediate format into a .gdx file. A higher lever function to_gdx_file() that does basically everything needed for exporting is provided for convenience.

author:

1. Soininen (VTT)

date:

30.8.2019

Module Contents

Classes

NoneExport	Options to export None values.
NoneFallback	Options load None values from the database.
Set	Represents a GAMS domain, set or a subset.
Record	Represents a GAMS set element in a Set.
Parameter	Represents a GAMS parameter.
Picking	An interface for picking objects.
FixedPicking	Picking from a fixed boolean array.
GeneratedPicking	Picking using a Python expression.
Records	An interface for records used in SetSettings.
LiteralRecords	Shufflable records with fixed keys.
GeneratedRecords	Non-shuffleable records where keys are generated by a Python expression.
ExtractedRecords	Records that are extracted from an indexed parameter.
MergingSetting	Holds settings needed to merge a single parameter.
IndexingSetting	Settings for indexed value expansion for a single Parameter.
SetSettings	This class holds the settings for domains, sets and records needed by to_gdx_file() for .gdx export.
ExportFlag	Options for exporting Set objects.
Origin	Domain or set origin.
SetMetadata	This class holds some additional configuration for Sets.

Functions

_picking_from_dict(picking_dict)	Deserializes pickings from dictionary.
_update_records(old, new)	Updates records where appropriate.
_records_from_dict(record_dict)	Deserializes records from a dict.
_python_interpreter_bitness()	Returns 64 for 64bit Python interpreter or 32 for 32bit interpreter.
_read_value(value_in_database)	Converts a parameter from its database representation to a value object.
_windows_dlls_exist(gams_path)	Returns True if required DLL files exist in given GAMS installation path.
find_gams_directory()	Returns GAMS installation directory or None if not found.
expand_indexed_parameter_values(parameters, indexing_settings, sets)	Expands the dimensions of indexed parameter values.
update_merging_settings(merging_settings, set_settings, db_map)	Returns parameter merging settings updated according to new export settings.
merging_records(merging_setting)	Constructs records which contain the merged parameters’ names.
merge_parameters(parameters, merging_settings)	Merges multiple parameters into a single parameter.
sets_to_gams(gdx_file, sets, omitted_set=None)	Writes Set objects to .gdx file as GAMS sets.
parameters_to_gams(gdx_file, parameters, none_export)	Writes parameters to .gdx file as GAMS parameters.
domain_parameters_to_gams_scalars(gdx_file, parameters, domain_name)	Adds the parameter from given domain as a scalar to .gdx file.
object_classes_to_domains(db_map, domain_names)	Converts object classes and objects from a database to the intermediate format.
relationship_classes_to_sets(db_map, domain_names, set_names)	Converts relationship classes and relationships from a database to the intermediate format.
object_parameters(db_map, domains_with_ids, fallback_on_none, logger)	Converts object parameters from database to Parameter objects.
relationship_parameters(db_map, sets_with_ids, fallback_on_none, logger)	Converts relationship parameters from database to Parameter objects.
_default_values(db_map, subquery, sets_with_ids, classes_with_ignored_parameters)	Reads default parameter values from the database.
_update_using_existing_relationship_parameter_values(parameters, db_map, sets_with_ids, classes_with_ignored_parameters)	Updates an existing relationship parameter dict using actual parameter values.
domain_names_and_records(db_map)	Returns a list of domain names and a map from a name to list of record keys.
set_names_and_records(db_map)	Returns a list of set names and a map from a name to list of record keys.
make_indexing_settings(db_map, none_fallback, logger)	Constructs skeleton indexing settings for parameter indexed value expansion.
_object_indexing_settings(db_map, none_fallback, logger)	Constructs skeleton indexing settings from object parameters.
_relationship_indexing_settings(db_map, none_fallback, logger)	Constructs skeleton indexing settings from relationship parameters.
_add_to_indexing_settings(settings, parameter_name, entity_class_name, dimensions, parsed_value, index_keys, classes_with_unsupported_value_types)	Adds parameter to indexing settings.
update_indexing_settings(old_indexing_settings, new_indexing_settings, set_settings)	Returns new indexing settings merged from old and new ones.
indexing_settings_to_dict(settings)	Stores indexing settings to a JSON compatible dictionary.
indexing_settings_from_dict(settings_dict, db_map, none_fallback, logger)	Restores indexing settings from a json compatible dictionary.
_find_indexed_parameter(parameter_name, db_map, none_fallback, logger=None)	Searches for parameter_name in db_map and returns Parameter and its entity_class name.
_create_additional_domains(set_settings)	Generates additional domains found in the settings.
_exported_set_names(names, set_settings)	Returns a set of names of the domains that are marked for exporting.
sort_sets(sets, order)	Sorts a list of sets according to sorted_names
sort_records_inplace(sets, set_settings)	Sorts the record lists of given domains according to the order given in settings.
extract_domain(domains, name_to_extract)	Extracts the domain with given name from a list of domains.
to_gdx_file(database_map, file_name, set_settings, indexing_settings, merging_settings, none_fallback, none_export, gams_system_directory=None, logger=None)	Exports given database map into .gdx file.
make_set_settings(database_map)	Builds a SetSettings object from given database.

class spinetoolbox.spine_io.exporters.gdx.NoneExport

Bases: enum.Enum

Options to export None values.

Create and return a new object. See help(type) for accurate signature.

DO_NOT_EXPORT = 0

Does not export Nones.

EXPORT_AS_NAN = 1

Replace Nones with NaNs while exporting.

class spinetoolbox.spine_io.exporters.gdx.NoneFallback

Bases: enum.Enum

Options load None values from the database.

Create and return a new object. See help(type) for accurate signature.

USE_IT = 0

Keep using the value.

USE_DEFAULT_VALUE = 1

Replace None by the default value.

exception spinetoolbox.spine_io.exporters.gdx.GdxExportException(message)

Bases: Exception

An exception raised when something goes wrong within the gdx module.

Parameters:message (str) – a message detailing the cause of the exception

message

A message detailing the cause of the exception.

__str__(self)

Returns the message detailing the cause of the exception.

exception spinetoolbox.spine_io.exporters.gdx.GdxUnsupportedValueTypeException(message)

Bases: spinetoolbox.spine_io.exporters.gdx.GdxExportException

An exception raised when an unsupported parameter type is read from the database.

Parameters:message (str) – a message detailing the cause of the exception

class spinetoolbox.spine_io.exporters.gdx.Set(name, description='', domain_names=None)

Represents a GAMS domain, set or a subset.

description

set’s explanatory text

Type:str

domain_names

a list of superset (domain) names, None if the Set is a domain

Type:list of str

name

set’s name

Type:str

records

set’s elements as a list of Record objects

Type:list of Record

Parameters:

• name (str) – set’s name
• description (str) – set’s explanatory text
• domain_names (list of str) – a list of indexing domain names

dimensions

Number of dimensions of this Set.

is_domain(self)

Returns True if this set is a domain set.

to_dict(self)

Stores Set to a dictionary.

static from_dict(set_dict)

Restores Set from a dictionary.

class spinetoolbox.spine_io.exporters.gdx.Record(keys)

Represents a GAMS set element in a Set.

keys

a tuple of record’s keys

Type:tuple

Parameters:keys (tuple) – a tuple of record’s keys

name

Record’s ‘name’ as a comma separated list of its keys.

__eq__(self, other)

Returns True if other is equal to self.

Parameters:other (Record) – a record to compare to

to_dict(self)

Stores Record to a dictionary.

static from_dict(record_dict)

Restores Record from a dictionary.

class spinetoolbox.spine_io.exporters.gdx.Parameter(domain_names, indexes, values)

Represents a GAMS parameter.

domain_names

indexing domain names (currently Parameters can be indexed by domains only)

Type:list

data

a map from index tuples to parsed values

Type:dict

Parameters:

• domain_names (list) – indexing domain names (currently Parameters can be indexed by domains only)
• indexes (list) – parameter’s indexes
• values (list) – parameter’s values

indexes

indexing key tuples

Type:list

values

parsed values

Type:list

__eq__(self, other)

Compares two Parameter objects for equality.

Parameters:other (Parameter) – a parameter Returns:True if the parameters are equal, False otherwise Return type:bool

is_consistent(self)

Checks that all values are IndexedValue objects or scalars.

slurp(self, parameter)

Appends the indexes and values from another parameter.

Parameters:parameter (Parameter) – a parameter to append from

is_scalar(self)

Returns True if this parameter seems to contain scalars.

is_indexed(self)

Returns True if this parameter seems to contain indexed values.

expand_indexes(self, indexing_setting, sets)

Expands indexed values to scalars in place by adding a new dimension (index).

The indexes and values attributes are resized to accommodate all scalars in the indexed values. A new indexing domain is inserted to domain_names and the corresponding keys into indexes. Effectively, this increases parameter’s dimensions by one.

Parameters:

• indexing_setting (IndexingSetting) – description of how the expansion should be done
• sets (dict) – mapping from set name to Set

class spinetoolbox.spine_io.exporters.gdx.Picking

An interface for picking objects.

Picking object are used to select indexes from an indexing domain when performing parameter index expansion.

pick(self, i)

Returns pick for given indexing domain record.

Parameters:i (int) – record index Returns:True if the record is picked, False otherwise Return type:bool

to_dict(self)

Serializes the picking to a dict.

Returns:serialized picking Return type:dict

static from_dict(picking_dict)

Deseriealizes the picking from a dict.

Parameters:picking_dict (dict) – serialized picking Returns:deserialized picking Return type:Picking

class spinetoolbox.spine_io.exporters.gdx.FixedPicking(picked)

Bases: spinetoolbox.spine_io.exporters.gdx.Picking

Picking from a fixed boolean array.

Parameters:picked (list of bool) – a list of booleans, where True picks and False drops a record

__eq__(self, other)

Compared pickings for equality.

Parameters:other (FixedPicking) – another picking Returns:True if the pickings are equal, False otherwise Return type:bool

pick(self, i)

See base class.

to_dict(self)

See base class.

static from_dict(picking_dict)

See base class.

class spinetoolbox.spine_io.exporters.gdx.GeneratedPicking(expression)

Bases: spinetoolbox.spine_io.exporters.gdx.Picking

Picking using a Python expression.

The expression should return a value that can be cast to bool. It has a single parameter, i, at its disposal. This is a one-based index to the pick list.

Parameters:expression (str) – the expression used for picking

expression

the picking expression

pick(self, i)

See base class.

to_dict(self)

See base class.

static from_dict(picking_dict)

See base class.

spinetoolbox.spine_io.exporters.gdx._picking_from_dict(picking_dict)

Deserializes pickings from dictionary.

Parameters:picking_dict (dict) – a serialized picking Returns:a FixedPicking or GeneratedPicking Return type:Picking

class spinetoolbox.spine_io.exporters.gdx.Records

An interface for records used in SetSettings.

records

stored records as a list of key tuples

__eq__(self, other)

Tests for equality.

Returns:True if the records are equal, False otherwise. Return type:bool

__len__(self)

Gives the number of records

Returns:number of records Return type:int

shuffle(self, new_order)

Reorders the records if the order is not fixed, otherwise raises NotImplementedError.

Parameters:new_order (list of tuple) – new records

is_shufflable(self)

Tells if the records can be shuffled.

Returns:True if the records can be shuffled, False otherwise Return type:bool

static update(old, new)

Merges two records.

Parameters:

• old (Records) – the ‘original’ records
• new (Records) – ‘new’ records

Returns:

merged records

Return type:

Records

to_dict(self)

Serializes the records to a dict.

Returns:serialized records. Return type:dict

static from_dict(record_dict)

Deserializes records from a dict.

Parameters:record_dict – serialized records Returns:deserialized records Return type:Records

class spinetoolbox.spine_io.exporters.gdx.LiteralRecords(records)

Bases: spinetoolbox.spine_io.exporters.gdx.Records

Shufflable records with fixed keys.

Parameters:records (list of tuple) – list of key tuples

records

See base class.

__eq__(self, other)

Compares two LiteralRecords for equality.

Parameters:other (LiteralRecords) – records to compare to Returns:True if the key lists are equal, False otherwise Return type:bool

__len__(self)

See base class.

shuffle(self, new_order)

See base class.

is_shufflable(self)

Returns True; LiteralRecords is shufflable.

static update(old, new)

Updates the keys from another LiteralRecords.

Common keys are kept in their old order while new keys are added last. Keys present only in old records are dropped.

Parameters:

• old (LiteralRecords) – original records
• new (LiteralRecords) – new records

Returns:

updated records

Return type:

LiteralRecords

to_dict(self)

See base class.

static from_dict(record_dict)

See base class.

class spinetoolbox.spine_io.exporters.gdx.GeneratedRecords(expression, length)

Bases: spinetoolbox.spine_io.exporters.gdx.Records

Non-shuffleable records where keys are generated by a Python expression.

The expression should return a string.The expression has a single parameter, i, at it disposal. i is a one-based index to the pick list.

Parameters:

• expression (str) – key generator expression
• length (int) – number of records to generate

expression

the expression used to generate the records

records

See base class.

__eq__(self, other)

Compares to another GeneratedRecords for equality

Parameters:other (GeneratedRecords) – records Returns:True if the record expressions and lengths are equal, False otherwise Return type:bool

__len__(self)

See base class.

shuffle(self, new_order)

See base class.

is_shufflable(self)

Returns False; GeneratedRecords is not shuffleable.

static update(old, new)

Updating is not supported by GeneratedRecords.

to_dict(self)

See base class.

static from_dict(record_dict)

See base class.

_record_list(self)

Generates records according to given Python expression.

Returns:generated records Return type:list

class spinetoolbox.spine_io.exporters.gdx.ExtractedRecords(parameter_name, indexes)

Bases: spinetoolbox.spine_io.exporters.gdx.Records

Records that are extracted from an indexed parameter.

Parameters:

• parameter_name (str) – name of the parameter from which the records were extracted
• indexes (list of tuple) – records

parameter_name

name of the parameter from which the records were extracted

records

See base class.

__eq__(self, other)

Compares two ExtractedRecords for equality.

Parameters:other (ExtractedRecords) – records to compare to Returns:True if the records and paramter name are equal, False otherwise Return type:bool

__len__(self)

See base class.

shuffle(self, new_order)

See base class.

is_shufflable(self)

Returns False; ExtractedRecords is never shufflable.

static extract(parameter_name, db_map)

Gets the record keys from a given indexed parameter.

Parameters:

• parameter_name (str) – parameter’s name
• db_map (DatabaseMappingBase) – a database map

Returns:

extracted records

Return type:

ExtractedRecords

static update(old, new)

Takes the parameter name from old and the records from new.

Parameters:

• old (ExtractedRecords) – original records
• new (ExtractedRecords) – new records

Returns:

merged records

Return type:

ExtractedRecords

to_dict(self)

See base class.

static from_dict(record_dict)

See base class.

spinetoolbox.spine_io.exporters.gdx._update_records(old, new)

Updates records where appropriate.

Parameters:

• old (Records) – original records
• new (Records) – new records

Returns:

updated records

Return type:

Records

spinetoolbox.spine_io.exporters.gdx._records_from_dict(record_dict)

Deserializes records from a dict.

Parameters:record_dict (dict) – serialized records Returns:deserialized records Return type:Records

spinetoolbox.spine_io.exporters.gdx._python_interpreter_bitness()

Returns 64 for 64bit Python interpreter or 32 for 32bit interpreter.

spinetoolbox.spine_io.exporters.gdx._read_value(value_in_database)

Converts a parameter from its database representation to a value object.

spinetoolbox.spine_io.exporters.gdx._windows_dlls_exist(gams_path)

Returns True if required DLL files exist in given GAMS installation path.

spinetoolbox.spine_io.exporters.gdx.find_gams_directory()

Returns GAMS installation directory or None if not found.

On Windows systems, this function looks for gams.location in registry; on other systems the PATH environment variable is checked.

Returns:a path to GAMS installation directory or None if not found.

spinetoolbox.spine_io.exporters.gdx.expand_indexed_parameter_values(parameters, indexing_settings, sets)

Expands the dimensions of indexed parameter values.

Parameters:

• parameters (dict) – a map from parameter names to Parameters
• indexing_settings (dict) – mapping from parameter name to IndexingSetting
• sets (dict) – mapping from domain name to Set

class spinetoolbox.spine_io.exporters.gdx.MergingSetting(parameter_names, new_domain_name, new_domain_description, previous_set, previous_domain_names)

Holds settings needed to merge a single parameter.

parameter_names

parameters to merge

Type:list

new_domain_name

name of the additional domain that contains the parameter names

Type:str

new_domain_description

explanatory text for the additional domain

Type:str

previous_set

name of the set containing the parameters before merging; not needed for the actual merging but included here to make the parameters’ origing traceable

Type:str

Parameters:

• parameter_names (list) – parameters to merge
• new_domain_name (str) – name of the additional domain that contains the parameter names
• new_domain_description (str) – explanatory text for the additional domain
• previous_set (str) – name of the set containing the parameters before merging
• previous_domain_names (list) – list of parameters’ original indexing domains

domain_names(self)

Composes a list of merged parameter’s indexing domains.

Returns:a list of indexing domains including the new domain containing the merged parameters’ names Return type:list

to_dict(self)

Stores the settings to a dictionary.

static from_dict(setting_dict)

Restores settings from a dictionary.

spinetoolbox.spine_io.exporters.gdx.update_merging_settings(merging_settings, set_settings, db_map)

Returns parameter merging settings updated according to new export settings.

Parameters:

• merging_settings (dict) – old merging settings
• set_settings (SetSettings) – new set settings
• db_map (spinedb_api.DatabaseMapping or spinedb_api.DiffDatabaseMapping) – a database map

Returns:

updated merging settings

Return type:

dict

spinetoolbox.spine_io.exporters.gdx.merging_records(merging_setting)

Constructs records which contain the merged parameters’ names.

Parameters:merging_setting (MergingSetting) – settings Returns:records needed to index merged parameters Return type:Records

spinetoolbox.spine_io.exporters.gdx.merge_parameters(parameters, merging_settings)

Merges multiple parameters into a single parameter.

Note, that the merged parameters will be removed from the parameters dictionary.

Parameters:

• parameters (dict) – a mapping from existing parameter name to its Parameter object
• merging_settings (dict) – a mapping from the merged parameter name to its merging settings

Returns:

a mapping from merged parameter name to its Parameter object

Return type:

dict

spinetoolbox.spine_io.exporters.gdx.sets_to_gams(gdx_file, sets, omitted_set=None)

Writes Set objects to .gdx file as GAMS sets.

Records and Parameters contained within the Sets are written as well.

Parameters:

• gdx_file (GdxFile) – a target file
• sets (list) – a list of Set objects
• omitted_set (Set) – prevents writing this set even if it is included in given sets

spinetoolbox.spine_io.exporters.gdx.parameters_to_gams(gdx_file, parameters, none_export)

Writes parameters to .gdx file as GAMS parameters.

Parameters:

• gdx_file (GdxFile) – a target file
• parameters (dict) – a list of Parameter objects
• none_export (NoneExport) – option how to handle None values

spinetoolbox.spine_io.exporters.gdx.domain_parameters_to_gams_scalars(gdx_file, parameters, domain_name)

Adds the parameter from given domain as a scalar to .gdx file.

The added parameters are erased from parameters.

Parameters:

• gdx_file (GdxFile) – a target file
• parameters (dict) – a map from parameter name to Parameter object
• domain_name (str) – name of domain whose parameters to add

Returns:

a list of non-scalar parameters

spinetoolbox.spine_io.exporters.gdx.object_classes_to_domains(db_map, domain_names)

Converts object classes and objects from a database to the intermediate format.

Object classes get converted to Set objects while objects are stored as Record objects in the Set objects.

Parameters:

• db_map (DatabaseMapping or DiffDatabaseMapping) – a database map
• domain_names (set) – names of domains to convert

Returns:

a map from object_class id to corresponding Set.

Return type:

dict

spinetoolbox.spine_io.exporters.gdx.relationship_classes_to_sets(db_map, domain_names, set_names)

Converts relationship classes and relationships from a database to the intermediate format.

Relationship classes get converted to Set objects while relationships are stored as Record objects in corresponding Set objects.

Parameters:

• db_map (DatabaseMapping or DiffDatabaseMapping) – a database map
• domain_names (set) – names of domains (a.k.a object classes) the relationships connect
• set_names (set) – names of sets to convert

Returns:

a map from relationship_class ids to the corresponding Set objects

Return type:

dict

spinetoolbox.spine_io.exporters.gdx.object_parameters(db_map, domains_with_ids, fallback_on_none, logger)

Converts object parameters from database to Parameter objects.

Parameters:

• db_map (DatabaseMapping or DiffDatabaseMapping) – a database map
• domains_with_ids (dict) – mapping from object_class ids to corresponding Set objects
• fallback_on_none (NoneFallback) – fallback when encountering Nones
• logger (LoggingInterface, optional) – a logger; if not None, some errors are logged and ignored instead of raising an exception

Returns:

a map from parameter name to corresponding Parameter

Return type:

dict

spinetoolbox.spine_io.exporters.gdx.relationship_parameters(db_map, sets_with_ids, fallback_on_none, logger)

Converts relationship parameters from database to Parameter objects.

Parameters:

• db_map (DatabaseMapping or DiffDatabaseMapping) – a database map
• sets_with_ids (dict) – mapping from relationship_class ids to corresponding Set objects
• fallback_on_none (NoneFallback) – fallback when encountering Nones
• logger (LoggingInterface, optional) – a logger; if not None, some errors are logged and ignored instead of raising an exception

Returns:

a map from parameter name to corresponding Parameter

Return type:

dict

spinetoolbox.spine_io.exporters.gdx._default_values(db_map, subquery, sets_with_ids, classes_with_ignored_parameters)

Reads default parameter values from the database.

Parameters:

• db_map (DatabaseMapping or DiffDatabaseMapping) – a database map
• subquery (Alias) – object_parameter_definition_sq or relationship_parameter_definition_sq
• sets_with_ids (dict) – mapping from relationship_class ids to corresponding Set objects
• classes_with_ignored_parameters (set, optional) – a set of problematic relationship_class names; if not None, relationship_class names are added to this set in case of errors instead of raising an exception

Returns:

a map from parameter name to the parsed default value

Return type:

dict

spinetoolbox.spine_io.exporters.gdx._update_using_existing_relationship_parameter_values(parameters, db_map, sets_with_ids, classes_with_ignored_parameters)

Updates an existing relationship parameter dict using actual parameter values.

Parameters:

• parameters (dict) – a mapping from relationship parameter names to Parameter objects to update
• db_map (DatabaseMapping or DiffDatabaseMapping) – a database map
• sets_with_ids (dict) – mapping from relationship_class ids to corresponding Set objects
• classes_with_ignored_parameters (set, optional) – a set of problematic relationship_class names; if not None, class names are added to this set in case of errors instead of raising an exception

spinetoolbox.spine_io.exporters.gdx.domain_names_and_records(db_map)

Returns a list of domain names and a map from a name to list of record keys.

Parameters:db_map (DatabaseMapping or DiffDatabaseMapping) – a database map Returns:a tuple containing set of domain names and a dict from domain name to its records Return type:tuple

spinetoolbox.spine_io.exporters.gdx.set_names_and_records(db_map)

Returns a list of set names and a map from a name to list of record keys.

Parameters:db_map (spinedb_api.DatabaseMapping or spinedb_api.DiffDatabaseMapping) – a database map Returns:a tuple containing a set of set names and a dict from set name to its records Return type:tuple

class spinetoolbox.spine_io.exporters.gdx.IndexingSetting(indexed_parameter, set_name)

Settings for indexed value expansion for a single Parameter.

parameter

a parameter containing indexed values

Type:Parameter

indexing_domain_name

indexing domain’s name

Type:str

picking

index picking

Type:FixedPicking or GeneratePicking

index_position

where to insert the new index when expanding a parameter

Type:int

set_name

name of the domain or set to which this parameter belongs

Type:str

Parameters:

• indexed_parameter (Parameter) – a parameter containing indexed values
• set_name (str) – name of the original entity_class to which this parameter belongs

append_parameter(self, parameter)

Adds indexes and values from another parameter.

Parameters:parameter (Parameter) – parameter to slurp

to_dict(self)

Serializes settings to dict.

Returns:serialized settings Return type:dict

static from_dict(setting_dict, parameter, set_name)

Restores serialized setting from dict.

Parameters:

• setting_dict (dict) – serialized settings
• parameter (Parameter) – indexed parameter
• set_name (str) – name of the set containing the parameter

Returns:

restored setting

Return type:

IndexingSetting

spinetoolbox.spine_io.exporters.gdx.make_indexing_settings(db_map, none_fallback, logger)

Constructs skeleton indexing settings for parameter indexed value expansion.

Parameters:

• db_map (spinedb_api.DatabaseMapping or spinedb_api.DiffDatabaseMapping) – a database mapping
• none_fallback (NoneFallback) – how to handle None values
• logger (LoggerInterface, optional) – a logger

Returns:

a mapping from parameter name to IndexingSetting

Return type:

dict

spinetoolbox.spine_io.exporters.gdx._object_indexing_settings(db_map, none_fallback, logger)

Constructs skeleton indexing settings from object parameters.

Parameters:

• db_map (spinedb_api.DatabaseMapping or spinedb_api.DiffDatabaseMapping) – a database mapping
• none_fallback – how to handle Nones
• logger (LoggingInterface, optional) – a logger

Returns:

a mapping from parameter name to IndexingSetting

Return type:

dict

spinetoolbox.spine_io.exporters.gdx._relationship_indexing_settings(db_map, none_fallback, logger)

Constructs skeleton indexing settings from relationship parameters.

Parameters:

• db_map (spinedb_api.DatabaseMapping or spinedb_api.DiffDatabaseMapping) – a database mapping
• none_fallback (NoneFallback) – how to handle Nones
• logger (LoggingInterface, optional) – a logger

Returns:

a mapping from parameter name to IndexingSetting

Return type:

dict

spinetoolbox.spine_io.exporters.gdx._add_to_indexing_settings(settings, parameter_name, entity_class_name, dimensions, parsed_value, index_keys, classes_with_unsupported_value_types)

Adds parameter to indexing settings.

Parameters:

• settings (dict) – indexing settings
• parameter_name (str) – parameter’s name
• entity_class_name (str) – name of the object or relationship_class the parameter belongs to
• dimensions (list) – a list of parameter’s domain names
• parsed_value (IndexedValue) – parsed parameter_value
• index_keys (tuple) – parameter’s keys
• classes_with_unsupported_value_types (set, optional) – entity_class names with unsupported value types

spinetoolbox.spine_io.exporters.gdx.update_indexing_settings(old_indexing_settings, new_indexing_settings, set_settings)

Returns new indexing settings merged from old and new ones.

Entries that do not exist in old settings will be removed. If entries exist in both settings the old one will be chosen if both entries are ‘equal’, otherwise the new entry will override the old one. Entries existing in new settings only will be added.

Parameters:

• old_indexing_settings (dict) – settings to be updated
• new_indexing_settings (dict) – settings used for updating
• set_settings (SetSettings) – new set settings

Returns:

merged old and new indexing settings

Return type:

dict

spinetoolbox.spine_io.exporters.gdx.indexing_settings_to_dict(settings)

Stores indexing settings to a JSON compatible dictionary.

Parameters:settings (dict) – a mapping from parameter name to IndexingSetting. Returns:a JSON serializable dictionary Return type:dict

spinetoolbox.spine_io.exporters.gdx.indexing_settings_from_dict(settings_dict, db_map, none_fallback, logger)

Restores indexing settings from a json compatible dictionary.

Parameters:

• settings_dict (dict) – a JSON compatible dictionary representing parameter indexing settings.
• db_map (DatabaseMapping) – database mapping
• none_fallback (NoneFallback) – how to handle None parameter values
• logger (LoggerInterface, optional) – a logger

Returns:

a dictionary mapping parameter name to IndexingSetting.

Return type:

dict

spinetoolbox.spine_io.exporters.gdx._find_indexed_parameter(parameter_name, db_map, none_fallback, logger=None)

Searches for parameter_name in db_map and returns Parameter and its entity_class name.

spinetoolbox.spine_io.exporters.gdx._create_additional_domains(set_settings)

Generates additional domains found in the settings.

Parameters:set_settings (SetSettings) – settings Returns:a list of additional Set objects Return type:list

spinetoolbox.spine_io.exporters.gdx._exported_set_names(names, set_settings)

Returns a set of names of the domains that are marked for exporting.

Parameters:

• names (set) – list of all domain or set names
• set_settings (SetSettings) – settings

Returns:

names that should be exported

Return type:

set of str

spinetoolbox.spine_io.exporters.gdx.sort_sets(sets, order)

Sorts a list of sets according to sorted_names

Parameters:

• sets (list) – Set objects to be sorted
• order (dict) – a mapping from set name to index

Returns:

sorted Set objects

Return type:

list

spinetoolbox.spine_io.exporters.gdx.sort_records_inplace(sets, set_settings)

Sorts the record lists of given domains according to the order given in settings.

Parameters:

• sets (list of Set) – a list of Set objects whose records are to be sorted
• set_settings (SetSettings) – settings that define the sorting order

spinetoolbox.spine_io.exporters.gdx.extract_domain(domains, name_to_extract)

Extracts the domain with given name from a list of domains.

Parameters:

• domains (list) – a list of Set objects
• name_to_extract (str) – name of the domain to be extracted

Returns:

a tuple (list, Set) of the modified domains list and the extracted Set object

spinetoolbox.spine_io.exporters.gdx.to_gdx_file(database_map, file_name, set_settings, indexing_settings, merging_settings, none_fallback, none_export, gams_system_directory=None, logger=None)

Exports given database map into .gdx file.

Parameters:

• database_map (spinedb_api.DatabaseMapping or spinedb_api.DiffDatabaseMapping) – a database to export
• file_name (str) – output file name
• set_settings (SetSettings) – export settings
• indexing_settings (dict) – a dictionary containing settings for indexed parameter expansion
• merging_settings (dict) – a list of merging settings for parameter merging
• none_fallback (NoneFallback) – options how to handle none parameter values on database read
• none_export (NoneExport) – option how to handle none parameter values on export
• gams_system_directory (str, optional) – path to GAMS system directory or None to let GAMS choose one for you
• logger (LoggingInterface, optional) – a logger; if None given all error conditions raise GdxExportException otherwise some errors are logged and ignored

spinetoolbox.spine_io.exporters.gdx.make_set_settings(database_map)

Builds a SetSettings object from given database.

Parameters:database_map (spinedb_api.DatabaseMapping or spinedb_api.DiffDatabaseMapping) – a database from which domains, sets, records etc are extracted Returns:settings needed for exporting the entities and class from the given database_map Return type:SetSettings

class spinetoolbox.spine_io.exporters.gdx.SetSettings(domain_names, set_names, records, domain_tiers=None, set_tiers=None, metadatas=None, global_parameters_domain_name='')

This class holds the settings for domains, sets and records needed by to_gdx_file() for .gdx export.

SetSettings keeps track which domains, sets and records are exported into the .gdx file and in which order they are written to the file. This order is paramount for some models, like TIMES.

Parameters:

• domain_names (set of str) – domain names
• set_names (set of str) – set names
• records (dict) – a mapping from domain or set name to Records
• domain_tiers (dict, optional) – a mapping from domain name to tier
• set_tiers (dict, optional) – a mapping from set name to tier
• metadatas (dict, optional) – a mapping from domain or set name to SetMetadata
• global_parameters_domain_name (str, optional) – name of the domain whose parameters should be exported as scalars

domain_names

domain names

domain_tiers

a mapping from domain name to tier

set_names

set names

set_tiers

a mapping from set name to tier

global_parameters_domain_name

the name of the domain, parameters of which should be exported as GAMS scalars

metadata(self, name)

Returns the metadata for given domain/set.

Parameters:name (str) – set/domain name Returns:metadata Return type:Metadata

is_exportable(self, set_name)

Returns True if the domain or set with the given name is exportable, False otherwise.

Parameters:set_name (str) – domain/set name

add_or_replace_domain(self, domain_name, records, metadata)

Adds a new domain or replaces an existing domain’s records and metadata.

Parameters:

• domain_name (str) – a domain to add/replace
• records (Records) – domain’s records
• metadata (SetMetadata) – domain’s metadata

Returns:

True if a new domain was added, False if an existing domain was replaced

Return type:

bool

remove_domain(self, domain_name)

Erases domain.

Parameters:domain_name (str) – name of the domain to remove

records(self, name)

Returns the records of a given domain or set.

Parameters:name (str) – domain or set name Returns:domain’s or set’s records Return type:Records

update_records(self, set_name, records)

Updates the records of given domain or set.

Parameters:

• set_name (str) – domain or set name
• records (Records) – updated records

update(self, updating_settings)

Updates the settings by merging with another one.

All domains, sets and records that are in both settings (common) or in updating_settings (new) are retained. Common elements are ordered the same way they were ordered in the original settings. New elements are appended to the common ones in the order they were in updating_settings

Parameters:updating_settings (SetSettings) – settings to merge with

to_dict(self)

Serializes the this object to a dict.

Returns:serialized settings Return type:dict

static from_dict(dictionary)

Deserializes SetSettings from a dict.

Parameters:dictionary (dict) – serialized settings Returns:restored settings Return type:SetSettings

class spinetoolbox.spine_io.exporters.gdx.ExportFlag

Bases: enum.Enum

Options for exporting Set objects.

Create and return a new object. See help(type) for accurate signature.

EXPORTABLE = 1

User has declared that the set should be exported.

NON_EXPORTABLE = 2

User has declared that the set should not be exported.

class spinetoolbox.spine_io.exporters.gdx.Origin

Bases: enum.Enum

Domain or set origin.

Create and return a new object. See help(type) for accurate signature.

DATABASE = 1

Set exists in the database.

INDEXING = 2

Set has been generated for indexed parameter indexing.

MERGING = 3

Set has been generated for parameter merging.

class spinetoolbox.spine_io.exporters.gdx.SetMetadata(exportable=ExportFlag.EXPORTABLE, origin=Origin.DATABASE)

This class holds some additional configuration for Sets.

exportable

set’s export flag

Type:ExportFlag

origin

True if the domain does not exist in the database but is supplied separately.

Type:bool

description

set’s description or None if its origin is from database

Type:str

Parameters:

• exportable (ExportFlag) – set’s export flag
• origin (Origin) – where the set comes from

__eq__(self, other)

Returns True if other is equal to this metadata.

is_additional(self)

Returns True if Set does not originate from the database.

is_exportable(self)

Returns True if Set should be exported.

to_dict(self)

Serializes metadata to a dictionary.

static from_dict(metadata_dict)

Deserializes metadata from a dictionary.