The scope
A clear scope starts from an extended review of actual users and
prospect users. SimLab has always been the path into sensitivity
analysis for model developers and scientific staff. The JRC has been
working a lot over the years to spread the usage of sensitivity
analysis out to universities, administrations and other communities
worldwide. After that, SimLab needs now to become the tool of choice
for intensive use of uncertainty and sensitivity analysis techniques.
The application concept
A clear application concept starts from a clear understanding of
available technologies and current technology trends and ends into a
precise standard based design choice: SimLab is a development
framework. A development framework is the place where developers find a
set of specific functionalities, i.e. a set of APIs, for a precise
subject domain. The user can find here a few development frameworks.
- Microsoft .NET is a development framework for developing interoperative service/network based software
- Xerces is a development framework for XML
- CppUnit is a development framework for Testing
- Doxygen is a development framework for Documentation
And SimLab? SimLab is a development framework for uncertainty and sensitivity analysis.
Development frameworks are developed and extended in order to support
more features on three main directions:
- More functionalities in the specific subject domain
- Support for different platforms and development environments
- More features to help developers in their task
SimLab supports a set of uncertainty and sensitivity analysis
techniques, supports an initial set of platforms, supports a set of
languages and for the future it is expected that it will have
additional tools for developers such as a user interface and a command
line interpreter.
Technology Stack
On the bottom level, SimLab is developed in such a way to be source
compatible with major available platforms, but it is binary distributed
and supported on Linux and Windows. SimLab supports compatibility for a
set of standard languages (i.e. Matlab, Fortran and C/C++) and with a
proprietary interpreted scripting language for SimLab. Other language
support may be added for the future. On top of the language level there
are test cases i.e. executables or interpreted code that exploit the
three major features of Simlab:
- Sample generation
- Model Execution
- Uncertainty and sensitivity estimation
These features are developed in one of the available environments.

- Note:
- A test case is what we generically call the developer file, the internal testcase, or the more complex and evolute user demo.
The Software Layout

The framework
APIs enlargement
The current set of exported APIs is composed of roughly 200 calls which
are available on all the languages supported. This congruency is
fulfilled in order to:
- Maximize standardization
- Maximize compatibility in the different environments
- Simplify learning and minimize learning time
On the other hand, the APIs support macro calls such as the 'prep' and
'spop' calls, which permit to compact programming effort for the UA/SA
techniques. This means that the user can start with a 'prep' call for
sample generation, then runs his/her specific computational model
(internal or external), and finally s/he executes the 'spop' call to
retrieve all the possible sensitivity measures that are compatible with
the sampling technique selected. This approach may sound nice, but
usage of this calls is discouraged, and the user may consider them as
deprecated features, as they go against the design goals listed below
and they tend to be bug prone:
- The data structure in support of these calls tends to be complex, difficult to learn and use, and unsafe from a memory perspective. The interface is C-compatible and therefore any data structure must be previously initialized by the Fortran and C/C++ users
- It is a "compute all" approach, which does not grant quality and full control of results
These methods will possibly not support additional techniques added in
the future. For instance, some SA techniques, such as the sequential
estimate of indices, are incompatible with this approach.
The actual APIs design has few major goals:
- Separate feature by feature all the possible functionalities
- Simplify usability
- Maximize user control
The general rule is that when the software is required to compute one
value, the software is minimizing the necessary computational effort to
achieve that value, and is computing only that value.
Demos
Any developer needs a quick start into things, a working environment
s/he can play with and few notions to start from: this is what demos
have been made for. The development environment is provided for C++ and
Fortran users. At the moment, a small set of demos is supported. It is
expected that further development effort will be dedicated to this
area.
The two basic demos are a simple one (the hedged portfolio model) and a complex one (the Level E model).
The two basic demos are a simple one (the hedged portfolio model) and a complex one (the Level E model).
Testcases and Testing framework
Test cases are a fundamental part of a development framework: they
grant quality on one end and they provide a set of advanced examples on
the other. The number of features in a development framework such as
SimLab, is very large (i.e. 200 APIs). Hence, the possible ordered
combinations of these is an incredibly large number
.
All this can be faced only with a suitable testing framework, that is also provided to users. This testing framework will be maintained and extended in order that, in future releases, new features and bugs will be traced with specific reproducible testcases.

All this can be faced only with a suitable testing framework, that is also provided to users. This testing framework will be maintained and extended in order that, in future releases, new features and bugs will be traced with specific reproducible testcases.
Benchmarks
Benchmarks are extended testcases that extensively check the quality of
outputs in a standard scientific environment with a specific subject
domain. These may be supported by publications on peer-reviewed
journals. An appropriate section on test cases includes a set of
benchmarks provided by independent sources.
Documentation Facility
A documentation generation facility is included in order to coherently
maintain all the API reference and all the relevant documentation of the
framework. The developer has chosen Doxygen in order to achieve few
goals:
- Support for standard API documentation
- Support for multiple documentation formats (currently we ship with html, chm and pdf
- Support for latex formulas
Sensitivity Analysis Techniques
SIMLAB 3.0 supports a larger set of sensitivity analysis algorithms.
Too read more about this, please refer to Supported Mathematical Techniques
Platform Support
Too read more about the supported platforms, compilers and environments
please refer to Supported
environments
Persistence Layer
The older persistence layer of SimLab (2.2) is based on a set of text
files, that couples together to build the complete state of the
software. This situation will be enhanced in future releases with a
single xml scheme supporting the prep / model / spop configurations.
This will enhance scalability, robustness, will contribute to remove
all the actual diverse set of file formats, and to remove a bug prone
approach. Backward compatibility in order to load SimLab 2.2 files will
be granted.
Matlab Integration
Integration of Matlab and in general of a new language environment implies possibly a new distribution of the software. Currently we have a distributed library for C++ / for Fortran and for Matlab where the distribution is controlled by a preprocessor symbol.
For the case of Matlab there is a precise requirement of using the memory allocation scheme of Matlab and a C file i/o. For the Fortran distribution there is a difference with the main C++ distribution due to the pass by reference typical of Fortran. In the case of Matlab there is also a large code generated interface to Matlab. This is generated from meta tags added to the main interface file gsatoolbox.h .
Requirement, Development, Release process
A standard circular process for Requirement, Development, Release and
validation is finally available. The user has a well defined set of
operations in order to send back a bug, a contributed testcase, or a
contributed technique. The user may have release 3.0.x installed with a
specific setup, e.g. Matlab.
- You have a specific bug or issue or you may want to share your test case
- You send us back your development file (e.g. an M file)
- Your setup will be reproduced
- A set of testcases (e.g. C++) will be developed to test the issue
- Finally useful modifications and testcases will be added to subsequents releases
Quality Assurance plan
A standardized Quality Assurance plan was started with a set of
actions:
- Present the draft software to users at the summer school on sensitivity analysis (Venice 2004)
- A user questionnaire was set up with a set of useful user feedbacks
- A validation phase has been made during 2005
- A demonstration of Simlab 3.0 has been given during the training course on Sensitivity Analysis, held in Ispra in November 2005
- A ChangeLog is being maintained in order to track modifications ans additional features across releases
- A frequent release plan has been established
- Simlab 3.0 will be released public on May 2006