Application, dependency and version management#

context#

Run context: the configuration and services of one simtools run.

class simtools.context.Activity(activity_id=<factory>, activity_name=None, user=<factory>, hostname=<factory>)[source]#

Identity of one simtools activity.

classmethod from_arguments(args)[source]#: Return the activity of parsed arguments (a new ID if none is given).

class simtools.context.RunContext(model_reader=None, paths=<factory>, software=<factory>, activity=<factory>, args=<factory>, logger=None, run_time=None)[source]#

Configuration and services of one simtools run.

Parameters:

model_readerSimulationModelReader or None: Reader of the simulation models.
pathsOutputPaths: Output and model-configuration directories.
softwareSimulationSoftware: Simulation software installations and ray-tracing backend.
activityActivity: Activity identity.
argsMapping: Read-only view of the parsed application arguments.

classmethod from_arguments(args=None, model_reader=None, paths=None, resolve_sim_software_executables=True)[source]#

Return the context of parsed arguments.

Parameters:

argsdict, optional: Parsed arguments; wrapped read-only, not copied.
model_readerSimulationModelReader, optional: Reader of the simulation models.
pathsOutputPaths, optional: Output paths; by default built from output_path in the arguments.
resolve_sim_software_executablesbool, optional: Resolve the CORSIKA executable from the build manifest.

classmethod from_environment()[source]#: Return a fresh context built from environment.

classmethod from_spec(spec)[source]#: Rebuild a worker context from to_spec() output.

property ray_tracing_backend#: Configured optical ray-tracing backend.

replace(**changes)[source]#: Return a copy with the given fields replaced.

to_spec()[source]#: Return the JSON-serializable configuration needed to rebuild this context.

class simtools.context.SimulationSoftware(sim_telarray_root=None, sim_telarray_executable_name=None, corsika_root=None, corsika_interaction_table_root=None, corsika_build_variants=None, corsika_flat_executable=None, he_interaction='epos', le_interaction='urqmd', ray_tracing_backend='sim_telarray')[source]#

Configured CORSIKA and sim_telarray installations and the ray-tracing backend.

Fields hold the configured values; the properties validate them and resolve executables.

property corsika_dummy_file#

Path to a dummy CORSIKA file required by sim_telarray for ray-tracing simulations.

This file does not need to exist; sim_telarray only requires a file path.

property corsika_exe#: Path to the CORSIKA executable (flat atmosphere).

property corsika_exe_curved#: Path to the curved version of the CORSIKA executable.

corsika_executable(geometry)[source]#: Return the CORSIKA executable declared by the build manifest for a geometry.

property corsika_interaction_models#: Configured high- and low-energy CORSIKA interaction models.

property corsika_interaction_table_path#: Path to the CORSIKA interaction table directory.

property corsika_path#: Path to the CORSIKA installation directory.

classmethod from_arguments(args=None, resolve_sim_software_executables=True)[source]#

Return the software configuration from arguments, environment and defaults.

Priority order: argument > environment variable > built-in default.

Parameters:

argsMapping, optional: Parsed arguments.
resolve_sim_software_executablesbool, optional: Resolve the CORSIKA executable from the build manifest.

property sim_telarray_exe#: Path to the sim_telarray executable.

property sim_telarray_exe_debug_trace#: Path to the debug trace version of the sim_telarray executable.

property sim_telarray_path#: Path to the sim_telarray installation directory.

application.control#

Runtime control shared by simtools applications.

class simtools.application.control.RedactFilter(name='')[source]#

Filter to redact sensitive information from log messages.

This filter dynamically retrieves secret values from environment variables and uses pattern matching to identify and redact sensitive key-value pairs.

filter(record)[source]#: Filter log record to redact sensitive information.

simtools.application.control.get_log_file(args_dict, activity_id=None)[source]#

Get log file path.

Generate log file path if needed from application name and application ID.

Returns:

Path, str or None: Log file path, or None if no logging to file is configured.

simtools.application.control.initialize_runtime(args_dict, needs=())[source]#

Initialize common runtime services for parsed application configuration.

Parameters:

args_dictdict: Parsed application configuration.
needstuple of str, optional: Runtime services required by the application.

Returns:

RunContext: Configuration and services for the application run.

simtools.application.control.setup_logging(logger_name=None, log_level='INFO', log_file=None, activity_id=None)[source]#

Set up logging configuration.

Parameters:

logger_namestr, optional: Name for the logger. If None, uses the root logger. Default is None.
log_levelstr, optional: Logging level as a string (e.g., “DEBUG”, “INFO”). Default is “INFO”.
log_filestr or pathlib.Path, optional: Path to a log file. If provided, a file handler is added and log messages are written to this file. If None, logging to file is disabled. Default is None.

application.definition#

Definitions for command-line applications.

class simtools.application.definition.ApplicationDefinition(module_name, description, arguments=(), model_repository=False, needs=None, post_parse=None, defer_required_validation=False, usage=None, excluded_standard_arguments=())[source]#

Command-line and startup definition for an application.

property all_arguments#: Return standard and application-selected arguments in registration order.

build_parser()[source]#: Build the complete parser without reading configuration or starting the application.

classmethod for_module(module_name, **kwargs)[source]#: Define an application using the loaded module’s name and documentation.

property label#: Return the application module label.

start()[source]#: Read configuration and run the standard application startup sequence.

dependencies#

Simtools dependencies version management.

This modules provides two main functionalities:

retrieve the versions of simtools dependencies (e.g., model repositories, sim_telarray, CORSIKA)
provide space for future implementations of version management

simtools.dependencies.build_dependency_manifest(context=None)[source]#

Build a deterministic manifest from the active runtime environment.

Returns:

dict: Discovered dependency manifest.

simtools.dependencies.canonical_manifest_bytes(manifest)[source]#

Serialize a manifest deterministically for hashing.

Parameters:

manifestdict: Dependency manifest.

Returns:

bytes: Canonical UTF-8 JSON representation.

simtools.dependencies.export_build_info(output_file, run_time=None, context=None)[source]#

Export build and version information to a file.

Parameters:

output_filestr: Path to the output file.
run_timelist, optional: Runtime environment command (e.g., Docker).

simtools.dependencies.get_build_options(run_time=None, context=None)[source]#

Return CORSIKA / sim_telarray config and build options.

For CORSIKA / sim_telarray build for simtools version >0.25.0: expects build_opts.yml file in each CORSIKA and sim_telarray directories.

For CORSIKA / sim_telarray build for simtools version <=0.25.0: expects a build_opts.yml file in the sim_telarray directory.

Parameters:

run_timelist, optional: Runtime environment command (e.g., Docker).

Returns:

dict: CORSIKA / sim_telarray build options.

simtools.dependencies.get_corsika_version(run_time=None, context=None)[source]#

Get the version of the CORSIKA package.

Version strings for CORSIKA are of the form “7.7550” (major.minor with 4-digit minor).

Parameters:

run_timelist, optional: Runtime environment command (e.g., Docker).

Returns:

str: Version of the CORSIKA package.

simtools.dependencies.get_dependency_manifest(run_time=None, context=None)[source]#

Return the installed dependency manifest or a discovered fallback.

Parameters:

run_timelist, optional: Runtime command used to read a manifest from a container.

Returns:

dict: Dependency manifest.

Raises:

FileNotFoundError: If an explicitly requested container does not contain a manifest.
ValueError: If the manifest contains invalid JSON.

simtools.dependencies.get_dependency_manifest_digest(run_time=None, context=None)[source]#: Return the SHA-256 digest of the dependency manifest.

simtools.dependencies.get_dependency_metadata(context=None)[source]#: Return compact scalar provenance values for simulation metadata.

simtools.dependencies.get_dependency_summary(run_time=None, context=None)[source]#: Return a compact human-readable dependency summary.

simtools.dependencies.get_direct_python_dependency_versions()[source]#

Return installed versions of direct simtools Python dependencies.

Includes optional dependencies (extras) that are installed.

Returns:

dict: Normalized distribution names mapped to installed versions.

simtools.dependencies.get_sim_telarray_version(run_time=None, context=None)[source]#

Get the version of the sim_telarray package using ‘sim_telarray –version’.

Version strings for sim_telarray are of the form “2024.271.0” (year.day_of_year.patch).

Parameters:

run_timelist, optional: Runtime environment command (e.g., Docker).

Returns:

str: Version of the sim_telarray package.

simtools.dependencies.get_software_version(software, context=None)[source]#

Return the version of the specified software package.

Parameters:

softwarestr: Name of the software package.

Returns:

str: Version of the specified software package.

simtools.dependencies.get_version_string(run_time=None, include_software_versions=True, context=None)[source]#

Print the versions of the dependencies.

Parameters:

run_timelist, optional: Runtime environment command (e.g., Docker).
include_software_versionsbool, optional: If True, query sim_telarray/CORSIKA executables and include their runtime versions. If False, skip executable checks and report these values as None.

Returns:

str: String containing the versions of the dependencies.

simtools.dependencies.validate_simulation_dependencies(simulation_software, context=None)[source]#

Validate external software and data required by a simulation.

Parameters:

simulation_softwarestr: Simulation software selection: sim_telarray, corsika, or corsika_sim_telarray.

Raises:

ValueError: If the selection or CORSIKA interaction-table manifest is invalid, or if a required dependency is unavailable.

simtools.dependencies.write_dependency_manifest(output_file, context=None)[source]#

Write the active dependency manifest and its SHA-256 file.

Parameters:

output_filestr or Path: JSON manifest output path.

simtools.dependencies.write_development_dependency_manifest(output_file, project_file, build_option_files)[source]#

Write provenance for a development image without installing simtools.

Parameters:

output_filestr or Path: JSON manifest output path.
project_filestr or Path: Project file declaring direct Python dependencies.
build_option_filesiterable of str or Path: CORSIKA and sim_telarray build-options files to merge when available.

dependency_versions#

Read, validate, and export the simtools dependency version catalog.

simtools.dependency_versions.build_workflow_matrices(catalog)[source]#: Build GitHub Actions matrices from the dependency catalog.

simtools.dependency_versions.corsika_source_tag_for_build_id(build_id, catalog=None)[source]#

Return the CORSIKA source tag matching a legacy build identifier.

Parameters:

build_idstr or int: Legacy CORSIKA build identifier, for example 78010.
catalogdict, optional: Validated dependency catalog. The installed catalog is loaded when omitted.

Returns:

str or None: Matching CORSIKA source tag, or None when the catalog has no match.

Raises:

ValueError: If more than one catalog entry matches the build identifier.

simtools.dependency_versions.dependency_catalog_environment(catalog)[source]#

Return catalog-managed runtime values as environment assignments.

Parameters:

catalogdict: Validated dependency version catalog.

Returns:

dict: Environment variable names and values for model repository and test-resource configuration. Local paths and credentials are intentionally omitted.

simtools.dependency_versions.dependency_catalog_summary(catalog)[source]#: Return stable scalar build values used by Docker workflows.

simtools.dependency_versions.export_dependency_configuration(pyproject_path=None, output_format='catalog', extras=None)[source]#

Return dependency configuration in a selected export format.

Parameters:

pyproject_pathstr or Path, optional: Explicit project file, required for python-requirements output. The repository is searched when omitted for that output format.
output_formatstr, optional: One of catalog, env, github-output, python-requirements, or summary.
extraslist of str, optional: Optional dependency groups included in python-requirements output.

Returns:

str: Serialized dependency configuration, including a trailing newline.

simtools.dependency_versions.find_dependency_versions(start_path=None)[source]#

Find the dependency catalog matching the running simtools installation.

Parameters:

start_pathstr or Path, optional: Explicit search directory, taking precedence over bundled catalogs. By default, bundled catalogs take precedence over the working directory. SIMTOOLS_DEPENDENCY_VERSIONS overrides either lookup order.

Returns:

pathlib.Path: Path to the matching dependency_versions.yml file.

Raises:

FileNotFoundError: If an explicit environment override is missing or no catalog can be found.

simtools.dependency_versions.find_pyproject(start_path=None)[source]#: Find the nearest pyproject.toml file.

simtools.dependency_versions.load_dependency_catalog(catalog_path=None, validate=True)[source]#

Load the dependency version catalog from YAML.

Parameters:

catalog_pathstr or Path, optional: Explicit catalog file. The repository is searched when omitted.
validatebool, optional: Validate the catalog structure when True.

Returns:

dict: Dependency version catalog.

simtools.dependency_versions.model_repository_revision(catalog)[source]#: Return the configured simulation-model repository ref or revision.

simtools.dependency_versions.project_requirements(pyproject_path, extras)[source]#: Return project requirements, optionally including named extras.

simtools.dependency_versions.update_dependency_source_revisions(catalog_path, updates)[source]#

Update resolved source revisions and snapshot digests in a dependency catalog.

Parameters:

catalog_pathpathlib.Path: Dependency catalog to update.
updatesdict: Resolved source revisions and immutable snapshot digests keyed by source reference.

simtools.dependency_versions.validate_dependency_catalog(catalog)[source]#

Validate required dependency catalog values.

Parameters:

catalogdict: Dependency version catalog.

Returns:

dict: The validated catalog.

Raises:

ValueError: If a required value is missing or invalid.

simtools.dependency_versions.validate_env_template(template_path)[source]#

Validate non-secret runtime defaults against the dependency catalog.

Parameters:

template_pathstr or Path: Environment template to validate.

Raises:

ValueError: If catalog-managed versions are duplicated in the template.

version#

Software version setting.

simtools.version.base_version_for_patch_delta(version_string)[source]#

Return major.minor.0 version for patch releases (x.y.z -> x.y.0) or None.

Parameters:

version_stringstr or None: Version string to inspect, or None.

Returns:

str or None: The base patch version for patch releases, or None for all other inputs.

simtools.version.check_version_constraint(version_string, constraint)[source]#

Check if a version satisfies a constraint.

Parameters:

version_stringstr: The version string to check (e.g., “6.0.2”).
constraintstr: The version constraint to check against (e.g., “>=6.0.0”).

Returns:

bool: True if the version satisfies the constraint, False otherwise.

simtools.version.compare_versions(version_string_1, version_string_2, level='major.minor.patch')[source]#

Compare two versions at the given level: “major”, “major.minor”, “major.minor.patch”.

Parameters:

version_string_1str: First version string to compare.
version_string_2str: Second version string to compare.
levelstr, optional: Level of comparison: “major”, “major.minor”, or “major.minor.patch”

Returns:

int

-1 if version_string_1 < version_string_2: 0 if version_string_1 == version_string_2 1 if version_string_1 > version_string_2

simtools.version.is_valid_model_version(version_string)[source]#

Return whether a value is a bare simulation-model version.

Parameters:

version_stringstr: Version string to validate.

Returns:

bool: True when the value matches the simulation-model version format.

simtools.version.is_valid_package_version(version_string)[source]#

Return whether a value is a bare PEP 440 package version.

Parameters:

version_stringstr: Version string to validate.

Returns:

bool: True when the value is a valid non-v-prefixed PEP 440 version.

simtools.version.is_valid_release_tag(version_string)[source]#

Return whether a value is a v-prefixed semantic release tag.

Parameters:

version_stringstr: Version string to validate.

Returns:

bool: True when the value is a valid v-prefixed semantic release tag.

simtools.version.is_valid_revision(version_string)[source]#

Return whether a value is an exact lower-case Git revision.

Parameters:

version_stringstr: Revision string to validate.

Returns:

bool: True when the value is a valid lower-case Git revision hash.

simtools.version.is_valid_semantic_version(version_string, strict=True)[source]#

Check if a string is a valid semantic version.

Parameters:

version_stringstr: The version string to validate (e.g., “6.0.2”, “1.0.0-alpha”).
strictbool, optional: If True, use PEP 440 validation (packaging.version.Version). If False, use SemVer 2.0.0 regex pattern (allows more flexible pre-release identifiers).

Returns:

bool: True if the version string is valid, False otherwise.

simtools.version.resolve_by_version(config, model_version, preserve_inconsistent_keys=None)[source]#

Resolve version-dependent values in a configuration dictionary.

Fields whose value is a dict with a single by_version key are replaced by the first matching constraint value. All other fields are left unchanged.

Parameters:

configdict: Configuration dictionary (typically loaded from a YAML file).
model_versionstr or list: Semantic version string or list of strings used to evaluate constraints (e.g. “6.0.2” or [“6.0.2”, “6.1.1”]).
preserve_inconsistent_keysset, optional: Keys whose by_version mappings are retained when they resolve differently.

Returns:

dict: New dictionary with all by_version fields resolved to their matching value, or None when no constraint matches.

simtools.version.resolve_version_to_latest_patch(partial_version, available_versions)[source]#

Resolve a partial version (major.minor) to the latest patch version.

Given a partial version string (e.g., “6.0”) and a list of available versions, finds the latest patch version that matches the major.minor pattern.

Parameters:

partial_versionstr: Partial version string in format “major.minor” (e.g., “6.0”, “5.2”)
available_versionslist of str: List of available semantic versions (e.g., [“5.0.0”, “5.0.1”, “6.0.0”, “6.0.2”])

Returns:

str: Latest patch version matching the partial version pattern

Raises:

ValueError: If partial_version is not in major.minor format
ValueError: If no matching versions are found

Examples

>>> versions = ["5.0.0", "5.0.1", "6.0.0", "6.0.2", "6.1.0"]
>>> resolve_version_to_latest_patch("6.0", versions)
'6.0.2'
>>> resolve_version_to_latest_patch("5.0", versions)
'5.0.1'
>>> resolve_version_to_latest_patch("5.0.1", versions)
'5.0.1'

simtools.version.semver_to_int(version_string)[source]#

Convert a semantic version string to an integer.

Parameters:

version_stringstr: Semantic version string (e.g., “6.0.2”)

Returns:

int: Integer representation of the version (e.g., 60002 for “6.0.2”)

simtools.version.sort_versions(version_list, reverse=False)[source]#

Sort a list of semantic version strings.

Parameters:

version_listlist of str: List of semantic version strings (e.g., [“5.0.0”, “6.0.2”, “5.1.0”])
reversebool, optional: Sort in descending order if True (default False)

Returns:

list of str: Sorted list of version strings.

simtools.version.validate_release_tag(version_string)[source]#

Validate and return a v-prefixed semantic release tag.

Parameters:

version_stringstr: Release tag to validate.

Returns:

str: The unchanged release tag.

Raises:

ValueError: If version_string is not a v-prefixed semantic release tag.

simtools.version.version_kind(version_string)[source]#

Determine the kind of version string.

Parameters:

version_stringstr: The version string to analyze.

Returns:

str: The kind of version string (“major.minor”, “major.minor.patch”, or “major”).