Data Control

Why data control?

The contents of domain sets in your model may change through running procedures or performing other actions from within the graphical user interface. When elements are removed from sets, there may be data for domain elements that are no longer in the domain sets. In addition, data may exist for intermediate parameters, which is no longer used in the remainder of your model session. For these situations, AIMMS offers facilities to eliminate or activate data elements that fall outside their current domain of definition. This section provides you with housekeeping data control statements, which can be combined with ordinary assignments to keep your model data consistent and maintained.

Facilities

AIMMS offers the following data control tools:

  • the EMPTY statement to remove the contents from all or a selected number of identifiers,

  • the CLEANUP and CLEANDEPENDENTS statements to clean up all, or a selected number of identifiers,

  • the REBUILD statement to manually instruct AIMMS to reclaim unused memory from the internal data structures used to the store the data of a selected number of identifiers,

  • the procedure FindUsedElements to find all elements of a particular set that are in use in a given collection of indexed model identifiers, and

  • the procedure RestoreInactiveElements to find and restore all inactive elements of a particular set for which inactive data exists in a given collection of indexed model identifiers.

The EMPTY statement

The EMPTY statement can be used to discard the complete contents of all or selected identifiers in your model. Its syntax follows.

empty-statement:

image/svg+xmlEMPTY reference , IN database-table ;

Empty AIMMS identifiers

The EMPTY operator operates on a list of references to AIMMS identifiers and takes the following actions.

  • For parameters, variables (arcs) and constraints (nodes) AIMMS discards their values plus the contents of all their suffices.

  • For sets, AIMMS will discard their contents plus the contents of all corresponding subsets. If a set is a domain set, AIMMS will remove the data from all parameters and variables that are defined over this set or any of its subsets.

  • For slices of an identifier, AIMMS will discard all values associated with the slice.

  • For sections in your model text, AIMMS will discard the contents of all sets, parameters and variables declared in this section.

  • For a subset of the predefined set AllIdentifiers, AIMMS will discard the contents of all identifiers contained in this subset.

Use in databases

You can also use the EMPTY statement in conjunction with databases. With the EMPTY statement you can either empty single columns in a database table, or discard the contents of an entire table. This use is discussed in detail in Data Removal. You should note, however, that applying the EMPTY statement to a subset of AllIdentifiers does not apply to any database table contained in the subset to avoid inadvertent deletion of data.

Examples

The following statements illustrate the use of the EMPTY operator.

  • Remove all data of the variable Transport.

    empty Transport ;

  • Remove all data in the set Cities, but also all data depending on Cities, like e.g. Transport.

    empty Cities ;

  • Remove all the data of the indicated slice of the variable Transport

    empty Transport(DiscardedCity, j);

  • Remove all data of all identifiers in the model tree node CityData.

    empty CityData ;

Inactive data

When you remove some but not all elements from a domain set, AIMMS will not automatically discard the data associated with those elements for every identifier defined over the particular domain set. AIMMS will also not automatically discard data that does not satisfy the current domain restriction of a given identifier. Instead, it will consider such data as inactive. During the execution of your model, no reference will be made to inactive data, but such data may still be visible in the user interface. In addition, AIMMS will not directly reclaim the memory that is freed up when the cardinality of a multidimensional identifier in your model decreases.

When useful

The facility to create inactive data in AIMMS allows you to temporarily remove elements from domain sets when this is required by your model. You can then restore the data after the relevant parts of the model have been executed.

Discard inactive data

If you want to discard inactive data that has been introduced in a particular data set, you can apply the CLEANUP statement to parameters and variables, or the CLEANDEPENDENTS statement to root sets in your model. Through the REBUILD statement you can instruct AIMMS to reclaim the unused memory associated with one or more identifiers in your model. The syntax follows.

cleanup-statement:

image/svg+xmlCLEANDEPENDENTS CLEANUP REBUILD identifier , ;

Rules

The following rules apply when you call the CLEANUP statement.

  • When you apply the CLEANDEPENDENTS statement to a set, all inactive elements are discarded from the set itself and from all of its subsets. In addition, AIMMS will discard all inactive data throughout the model caused by the changes to the set.

  • When you apply the CLEANUP statement to a parameter or variable, all inactive data associated with the identifier is removed. This includes inactive data that is caused by changes in domain and range sets, as well as data that has become inactive by changes in the domain condition of the identifier.

  • When you apply the CLEANDEPENDENTS, CLEANUP, or REBUILD statement to a section, AIMMS will remove the inactive data of all sets, or parameters and variables declared in it, respectively.

After using the CLEANUP or CLEANDEPENDENTS statement for a particular identifier, all its associated inactive data is permanently lost.

Resorting root set elements

In addition to discarding inactive data from your model that is caused by the existence of inactive elements in a root set, the CLEANDEPENDENTS operator will also completely resort a root set and all data defined of it whenever possible and necessary. The following rules apply.

  • Resorting will only take place if the current storage order of a root set differs from its current ordering principle.

  • AIMMS will not resort sets for which explicit elements are used in the model formulation.

As a call to CLEANDEPENDENTS requires a complete rebuild of all identifiers defined over the root sets involved, the CLEANDEPENDENTS statement may take a relatively long time to complete. For a more detailed description of the precise manner in which root set elements and multidimensional data is stored in AIMMS refer to Set Element Ordering. This section also explains the benefits of resorting a root set.

Generated Mathematical Programs

The CLEANDEPENDENTS statement will also check whether any variable or constraint is affected; and if so will remove any generated mathematical program that is generated from such a variable or constraint.

Restricted usage in AIMMS GUI

You should not call the CLEANDEPENDENTS statement in procedures that have been linked to edit actions in graphical objects in an AIMMS end-user GUI via the Procedures tab of the object Properties dialog box. During these actions, AIMMS does not expect the element numbering to change.

Efficiency considerations

If you want to apply the CLEANDEPENDENTS statement to multiple sets, applying the operation to all sets in a single call of the CLEANDEPENDENTS statement will, in general, be more efficient than using a separate call for every single set. If an identifier depends on two or more of the sets to which you want to apply the CLEANDEPENDENTS operation, the data of such an identifier will only be traversed and/or rebuild once, rather than multiple times.

Examples

  • The following CLEANDEPENDENTS statement will remove all data from your application that depends on the removed element 'Amsterdam', including, for instance, all previously assigned values to Transport departing from or arriving at 'Amsterdam'.

    Cities -= 'Amsterdam' ;

cleandependents Cities ;

  • The following CLEANUP statement will remove the data of the identifier Transport for all tuples that either lie outside the current contents of Cities, or do not satisfy the domain restriction.

    cleanup Transport;

  • Consider a parameter A(i,j) where i is an index into a set S and j an index into a set T, then

    cleandependents S,T;

    will be more efficient than

    cleandependents S;
cleandependents T;

    because the latter may require A(i,j) to be rebuilt twice.

Finding used elements

When you want to remove the elements in a set that are no longer used in your application, you first have to make sure which elements are currently in use. To find these elements easily, AIMMS provides the procedure FindUsedElements. It has the following three arguments:

  • a set SearchSet for which you want to find the used elements,

  • a subset SearchIdentifiers of the predefined set AllIdentifiers consisting of all identifiers that you want to be investigated, and

  • a subset UsedElements of the set SearchSet containing the result of the search.

Upon execution, AIMMS will return that subset of SearchSet for which the elements are used in the combined data of the identifiers contained in SearchIdentifiers. When the identifiers SearchSet and UsedElements are contained in SearchIdentifiers they are ignored.

Example

The following call to FindUsedElements will find the elements of the set Cities that are used in the identifiers Supply, Demand, and Distance, and store the result in the set UsedCities.

SearchIdentifiers := DATA { Supply, Demand, Distance };

FindUsedElements( Cities, SearchIdentifiers, UsedCities );

If these cities are the only ones of interest, you can place them into the set Cities, and thereby overwrite its previous contents. After that you can cleanup your entire dataset by eliminating data dependent on cities other than the ones currently contained in the set Cities. This process is accomplished through the following two statements.

Cities := UsedCities;

cleandependents Cities;

Finding and restoring inactive elements

Inactive data in AIMMS results when elements are removed from (domain) sets. Such data will be inaccessible, unless the corresponding set elements are brought back into the set. When this is necessary, you can use the procedure RestoreInactiveElements provided by AIMMS. This procedure has the following three arguments:

  • a set SearchSet for which you want to verify whether inactive data exists,

  • a subset SearchIdentifiers of the predefined set AllIdentifiers consisting of those identifiers that you want to be investigated, and

  • a subset InactiveElements of the set SearchSet containing the result of the search.

Upon execution AIMMS will find all elements for which inactive data exists in the identifiers in SearchIdentifiers. The elements found will not only be placed in the result set InactiveElements, but also be added to the search set. This latter extension of SearchSet implies that the corresponding inactive data is restored.

Example

The following call to RestoreInactiveElements will verify whether inactive data exists for the set Cities in AllIdentifiers.

RestoreInactiveElements( Cities, AllIdentifiers, InactiveCities );

After such a call the set InactiveCities could contain the element 'Amsterdam'. In this case, the set Cities has been extended with 'Amsterdam' as well. If you subsequently decide that cleaning up the set Cities is harmless, the following two statements will do the trick.

Cities -= InactiveCities;

cleandependents Cities;

Reclaiming memory

If the cardinality of a multidimensional identifier in your model decreases, AIMMS will not automatically reclaim the memory that is freed up because of the decreased amount of data to store. Instead, it will keep the memory available to store additional data that is associated with subsequent changes to the identifier. If the cardinality of an identifier decreases dramatically during a run the of a model, this may lead to a huge amount of memory getting stuck up with a single identifier in your model.

Memory fragmentation

In addition, if a model is running for a prolonged period of time, and an identifier has undergone huge amounts of structural changes during that time, the memory associated with that identifier may become heavily fragmented. In the long run, memory fragmentation may lead to decreased performance of your model. Rebuilding the internal data structures associated with such an identifier will resolve the fragmentation problem.

Automatic reclamation

Prior to solving a mathematical program, AIMMS will perform a quick check comparing the total amount of memory used by an identifier to the amount of unused memory associated with that identifier. By adding to and removing elements from identifiers, memory may become fragmented and the fraction of unused memory may grow. If the fraction of unused memory compared to the total amount of memory in use becomes too large, AIMMS will automatically rebuild such an identifier in order to reclaim the unused memory. AIMMS will also reclaim the memory of an identifier whenever it becomes empty during the run of a model.

the REBUILD statement

Through the REBUILD statement you can manually instruct AIMMS to rebuild the internal data structures associated with one or more identifiers. During the REBUILD statement AIMMS uses a more thorough check to verify whether a rebuild of an identifier is worthwhile prior to solving a mathematical program.