I/O#
This module include I/O related functionality.
io_handler#
Handle input and output paths.
- class simtools.io.io_handler.IOHandler(*args, **kwargs)[source]#
Handle input and output paths.
- get_input_data_file(parent_dir=None, file_name=None, test=False)[source]#
Get path of a data file, using data_path.
- Parameters:
- parent_dir: str
Parent directory of the file.
- files_name: str
File name.
- test: bool
If true, return test resources location
- Returns:
- Path
- Raises:
- IncompleteIOHandlerInitError
if data_path is not set
- get_model_configuration_directory(label, model_version)[source]#
Get path of the simulation model configuration directory.
This is the directory where the sim_telarray configuration files will be stored.
- Parameters:
- label: str
Instance label.
- model_version: str
Model version.
- Returns:
- Path
- get_output_directory(label=None, sub_dir=None)[source]#
Return path to output directory.
- Parameters:
- label: str
Instance label.
- sub_dir: str
Name of the subdirectory (ray_tracing, model etc)
- Returns:
- Path
- Raises:
- FileNotFoundError
if error creating directory
- TypeError
raised for errors while creating directory name
- get_output_file(file_name, label=None, sub_dir=None)[source]#
Get path of an output file.
- Parameters:
- files_name: str
File name.
- label: str
Instance label.
- sub_dir: str
Name of the subdirectory (ray_tracing, model etc)
- Returns:
- Path
- set_paths(output_path=None, data_path=None, model_path=None, use_plain_output_path=False)[source]#
Set paths for input and output.
- Parameters:
- output_path: str or Path
Path pointing to the output directory.
- data_path: str or Path
Path pointing to the data files (e.g., CORSIKA or sim_telarray output).
- model_path: str or Path
Path pointing to the model file directory.
- use_plain_output_path: bool
Use plain output path without adding tool name and date
ascii_handler#
Helper module for ASCII file operations.
- class simtools.io.ascii_handler.JsonNumpyEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]#
Convert numpy to python types as accepted by json.dump.
- simtools.io.ascii_handler.collect_data_from_file(file_name, yaml_document=None)[source]#
Collect data from file based on its extension.
- Parameters:
- file_name: str
Name of the yaml/json/ascii file.
- yaml_document: None, int
Return list of yaml documents or a single document (for yaml files with several documents).
- Returns:
- data: dict or list
Data as dict or list.
- simtools.io.ascii_handler.collect_data_from_http(url)[source]#
Download yaml or json file from url and return it contents as dict.
File is downloaded as a temporary file and deleted afterwards.
- Parameters:
- url: str
URL of the yaml/json file.
- Returns:
- dict
Dictionary containing the file content.
- Raises:
- TypeError
If url is not a valid URL.
- FileNotFoundError
If downloading the yaml file fails.
- simtools.io.ascii_handler.is_utf8_file(file_name)[source]#
Check if a file is encoded in UTF-8.
- Parameters:
- file_name: str, Path
Name of the file to be checked.
- Returns:
- bool
True if the file is encoded in UTF-8, False otherwise.
- simtools.io.ascii_handler.read_file_encoded_in_utf_or_latin(file_name)[source]#
Read a file encoded in UTF-8 or Latin-1.
- Parameters:
- file_name: str
Name of the file to be read.
- Returns:
- list
List of lines read from the file.
- Raises:
- UnicodeDecodeError
If the file cannot be decoded using UTF-8 or Latin-1.
- simtools.io.ascii_handler.write_data_to_file(data, output_file, sort_keys=False, numpy_types=False)[source]#
Write structured data to JSON or YAML file.
The file type is determined by the file extension.
- Parameters:
- data: dict or list
Data to be written to the file.
- output_file: str or Path
Name of the file to be written.
- sort_keys: bool, optional
If True, sort the keys.
- numpy_types: bool, optional
If True, convert numpy types to native Python types.
table_handler#
IO operations on astropy tables.
- simtools.io.table_handler.copy_metadata_to_hdf5(src_file, dst_file, table_name)[source]#
Copy metadata (table column meta) from one HDF5 file to another.
For merging tables, this function ensures that the metadata is preserved.
- Parameters:
- src_filestr or Path
Path to the source HDF5 file.
- dst_filestr or Path
Path to the destination HDF5 file.
- table_namestr
Name of the table whose metadata is to be copied.
- simtools.io.table_handler.merge_tables(input_files, input_table_names, output_file)[source]#
Merge multiple astropy tables from different files into a single file.
Handles multiple tables per file and supports both HDF5 and FITS formats. Updates ‘file_id’ column if present to maintain file origin tracking.
- Parameters:
- input_fileslist of str
List of input file paths to be merged.
- input_table_nameslist of str
List of table names to be merged from each input file.
- output_filestr
Path to the output file where the merged data will be saved.
- Returns:
- None
- simtools.io.table_handler.read_table_file_type(input_files)[source]#
Determine the file type of the input files.
All input files must be of the same type (either all HDF5 or all FITS).
- Parameters:
- input_fileslist of str
List of input file paths.
- Returns:
- str
File type (‘HDF5’ or ‘FITS’).
- simtools.io.table_handler.read_table_from_hdf5(file, table_name)[source]#
Read a single astropy table from an HDF5 file.
- Parameters:
- filestr or Path
Path to the input HDF5 file.
- table_namestr
Name of the table to read.
- Returns:
- astropy.table.Table
The requested astropy table.
- simtools.io.table_handler.read_table_list(input_file, table_names, include_indexed_tables=False)[source]#
Read available tables found in the input file.
If table_counter is True, search for tables with the same name but with different suffixes (e.g., “_0”, “_1”, etc.).
- simtools.io.table_handler.read_tables(file, table_names, file_type=None)[source]#
Read tables from a file.
- Parameters:
- filestr
Path to the input file.
- table_nameslist of str
List of table names to read.
- file_typestr
Type of the input file (‘HDF5’ or ‘FITS’).
- Returns:
- dict
Dictionary with table names as keys and astropy tables as values.
- simtools.io.table_handler.write_table_in_hdf5(table, output_file, table_name)[source]#
Write or append a single astropy table to an HDF5 file.
- Parameters:
- tableastropy.table.Table
The astropy table to write.
- output_filestr or Path
Path to the output HDF5 file.
- table_namestr
Name of the table in the HDF5 file.
- Returns:
- None
- simtools.io.table_handler.write_tables(tables, output_file, overwrite_existing=True, file_type=None)[source]#
Write tables to file (overwriting if exists).
- Parameters:
- tableslist or dict
List or Dictionary with astropy tables as values.
- output_filestr or Path
Path to the output file.
- overwrite_existingbool
If True, overwrite the output file if it exists.
- file_typestr
Type of the output file (‘HDF5’ or ‘FITS’).
- Returns:
- None
hdf5_handler#
Helper module for reading and writing in hd5 format.
- simtools.io.hdf5_handler.fill_hdf5_table(hist, x_bin_edges, y_bin_edges, x_label, y_label, meta_data)[source]#
Create and fill an hdf5 table with the histogram information.
It works for both 1D and 2D distributions.
- Parameters:
- hist: numpy.ndarray
The counts of the histograms.
- x_bin_edges: numpy.array
The x bin edges of the histograms.
- y_bin_edges: numpy.array
The y bin edges of the histograms. Use None for 1D histograms.
- x_label: str
X bin edges label.
- y_label: str
Y bin edges label. Use None for 1D histograms.
- meta_data: dict
Dictionary with the histogram metadata.
legacy_data_handler#
Reading of legacy data files (expect that this will be obsolete in future).