I/O

This module include I/O related functionality.

io_handler

Handle input and output directories and file paths.

class simtools.io.io_handler.IOHandler(*args, **kwargs)

Handle input and output directories and file paths.

get_model_configuration_directory(model_version, sub_dir=None)

Get path of the simulation model configuration directory.

This is the directory where the sim_telarray configuration files will be stored.

Parameters:

model_version: str

Model version.

sub_dir: str

subdirectory

Returns:

Path

get_output_directory(sub_dir=None)

Create and get path of an output directory.

Parameters:

sub_dir: str or list of str, optional

Name of the subdirectory (ray_tracing, model etc)

Returns:

Path

Raises:

FileNotFoundError

if the directory cannot be created

get_output_file(file_name, sub_dir=None)

Get path of an output file.

Parameters:

files_name: str

File name.

sub_dir: sub_dir: str or list of str, optional

Name of the subdirectory (ray_tracing, model etc)

Returns:

Path

get_test_data_file(file_name=None)

Get path of a data file in the test resources directory.

Parameters:

files_name: str

File name.

Returns:

Path

set_paths(output_path=None, model_path=None)

Set paths for input and output.

Parameters:

output_path: str or Path

Path pointing to the output directory.

model_path: str or Path

Path pointing to the model file directory.

class simtools.io.io_handler.IOHandlerSingleton

Singleton base class.

ascii_handler

Helper module for ASCII file operations.

class simtools.io.ascii_handler.JsonNumpyEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Convert numpy to python types as accepted by json.dump.

default(o)

Return default encoder.

simtools.io.ascii_handler.collect_data_from_file(file_name, yaml_document=None)

Collect data from file based on its extension.

Parameters:

file_name: str

Name of the yaml/json/ascii file.

yaml_document: None, int

Return list of yaml documents or a single document (for yaml files with several documents).

Returns:

data: dict or list

Data as dict or list.

simtools.io.ascii_handler.collect_data_from_http(url)

Download yaml or json file from url and return it contents as dict.

File is downloaded as a temporary file and deleted afterwards.

Parameters:

url: str

URL of the yaml/json file.

Returns:

dict

Dictionary containing the file content.

Raises:

TypeError

If url is not a valid URL.

FileNotFoundError

If downloading the yaml file fails.

simtools.io.ascii_handler.is_utf8_file(file_name)

Check if a file is encoded in UTF-8.

Parameters:

file_name: str, Path

Name of the file to be checked.

Returns:

bool

True if the file is encoded in UTF-8, False otherwise.

simtools.io.ascii_handler.read_file_encoded_in_utf_or_latin(file_name)

Read a file encoded in UTF-8 or Latin-1.

Parameters:

file_name: str

Name of the file to be read.

Returns:

list

List of lines read from the file.

Raises:

UnicodeDecodeError

If the file cannot be decoded using UTF-8 or Latin-1.

simtools.io.ascii_handler.write_data_to_file(data, output_file, sort_keys=False, numpy_types=False)

Write structured data to JSON or YAML file.

The file type is determined by the file extension.

Parameters:

data: dict or list

Data to be written to the file.

output_file: str or Path

Name of the file to be written.

sort_keys: bool, optional

If True, sort the keys.

numpy_types: bool, optional

If True, convert numpy types to native Python types.

eventio_handler

Read file info and run headers from eventio (CORSIKA IACT, sim_telarray) files.

simtools.io.eventio_handler.get_combined_corsika_run_header(sim_telarray_file)

Return the CORSIKA run header information from an sim_telarray file.

Reads both RunHeader and MCRunHeader object from file and returns a merged dictionary. Adds primary id from the first event.

Parameters:

sim_telarray_file: str

Path to the sim_telarray file.

Returns:

dict, None

CORSIKA run header. Returns None if not found.

simtools.io.eventio_handler.get_corsika_run_and_event_headers(corsika_iact_file)

Return the CORSIKA run and event headers from a CORSIKA IACT eventio file.

Parameters:

corsika_iact_file: str, Path

Path to the CORSIKA IACT eventio file.

Returns:

tuple

CORSIKA run header and event header as dictionaries.

simtools.io.eventio_handler.get_corsika_run_number(file)

Return the CORSIKA run number from an eventio (CORSIKA IACT or sim_telarray) file.

Parameters:

file: str

Path to the eventio file.

Returns:

int, None

CORSIKA run number. Returns None if not found.

simtools.io.eventio_handler.get_simulated_events(event_io_file)

Return the number of shower and MC events from a simulation (eventio) file.

For a sim_telarray file, the number of simulated showers and MC events is determined by counting the number of MCShower (type id 2020) and MCEvent objects (type id 2021). For a CORSIKA IACT file, the number of simulated showers is determined by counting the number of IACTShower (type id 1202).

Parameters:

event_io_file: str, Path

Path to the eventio file.

Returns:

tuple

Number of showers and number of MC events (MC events for sim_telarray files only).

table_handler

IO operations on astropy tables.

simtools.io.table_handler.copy_metadata_to_hdf5(src_file, dst_file, table_name)

Copy metadata (table column meta) from one HDF5 file to another.

For merging tables, this function ensures that the metadata is preserved.

Parameters:

src_filestr or Path

Path to the source HDF5 file.

dst_filestr or Path

Path to the destination HDF5 file.

table_namestr

Name of the table whose metadata is to be copied.

simtools.io.table_handler.merge_tables(input_files, input_table_names, output_file)

Merge multiple astropy tables from different files into a single file.

Handles multiple tables per file and supports both HDF5 and FITS formats. Updates ‘file_id’ column if present to maintain file origin tracking.

Parameters:

input_fileslist of str

List of input file paths to be merged.

input_table_nameslist of str

List of table names to be merged from each input file.

output_filestr

Path to the output file where the merged data will be saved.

Returns:

None

simtools.io.table_handler.read_table_file_type(input_files)

Determine the file type of the input files.

All input files must be of the same type (either all HDF5 or all FITS).

Parameters:

input_fileslist of str

List of input file paths.

Returns:

str

File type (‘HDF5’ or ‘FITS’).

simtools.io.table_handler.read_table_from_hdf5(file, table_name)

Read a single astropy table from an HDF5 file.

Parameters:

filestr or Path

Path to the input HDF5 file.

table_namestr

Name of the table to read.

Returns:

astropy.table.Table

The requested astropy table.

simtools.io.table_handler.read_table_list(input_file, table_names, include_indexed_tables=False)

Read available tables found in the input file.

If table_counter is True, search for tables with the same name but with different suffixes (e.g., “_0”, “_1”, etc.).

simtools.io.table_handler.read_tables(file, table_names, file_type=None)

Read tables from a file.

Parameters:

filestr

Path to the input file.

table_nameslist of str

List of table names to read.

file_typestr

Type of the input file (‘HDF5’ or ‘FITS’).

Returns:

dict

Dictionary with table names as keys and astropy tables as values.

simtools.io.table_handler.write_table_in_hdf5(table, output_file, table_name)

Write or append a single astropy table to an HDF5 file.

Parameters:

tableastropy.table.Table

The astropy table to write.

output_filestr or Path

Path to the output HDF5 file.

table_namestr

Name of the table in the HDF5 file.

Returns:

None

simtools.io.table_handler.write_tables(tables, output_file, overwrite_existing=True, file_type=None)

Write tables to file (overwriting if exists).

Parameters:

tableslist or dict

List or Dictionary with astropy tables as values.

output_filestr or Path

Path to the output file.

overwrite_existingbool

If True, overwrite the output file if it exists.

file_typestr

Type of the output file (‘HDF5’ or ‘FITS’).

Returns:

None

hdf5_handler

Helper module for reading and writing in hd5 format.

simtools.io.hdf5_handler.fill_hdf5_table(hist, x_bin_edges, y_bin_edges, x_label, y_label, meta_data)

Create and fill an hdf5 table with the histogram information.

It works for both 1D and 2D distributions.

Parameters:

hist: numpy.ndarray

The counts of the histograms.

x_bin_edges: numpy.array

The x bin edges of the histograms.

y_bin_edges: numpy.array

The y bin edges of the histograms. Use None for 1D histograms.

x_label: str

X bin edges label.

y_label: str

Y bin edges label. Use None for 1D histograms.

meta_data: dict

Dictionary with the histogram metadata.

simtools.io.hdf5_handler.read_hdf5(hdf5_file_name)

Read a hdf5 output file.

Parameters:

hdf5_file_name: str or Path

Name or Path of the hdf5 file to read from.

Returns:

list

The list with the astropy.Table instances for the various 1D and 2D histograms saved in the hdf5 file.

simtools.io.hdf5_handler.validate_histogram(hist, y_bin_edges)

Validate histogram dimensions and y_bin_edges consistency.

Parameters:

hist (np.ndarray): The histogram array, expected to be 1D or 2D.

y_bin_edges (array-like or None): Bin edges for the second dimension (if applicable).

Raises:

ValueError: If histogram dimensions are invalid or inconsistent with y_bin_edges.

legacy_data_handler

Reading of legacy data files (expect that this will be obsolete in future).

simtools.io.legacy_data_handler.read_legacy_data_as_table(file_path, file_type)

Read legacy data file.

Parameters:

file_path: Path

Path to the legacy data file.

file_type: str

Type of legacy data file.

Returns:

Table

Astropy table.

Raises:

ValueError

If unsupported legacy data file type.

simtools.io.legacy_data_handler.read_legacy_lst_single_pe(file_path)

Read LST single pe file (in legacy data format).

File contains two columns: amplitude (in units of single p.e) and response.

Parameters:

file_path: Path

Path to the legacy data file.

Returns:

Table

Astropy table.