simtools-write-reduced-event-lists#

This application supports the local (default) and htcondor execution backends. See Execution backends for HTCondor setup and configuration.

Write reduced event lists from simulation files.

Overview#

Converts sim_telarray event files (*.simtel.zst) into a compact, analysis-oriented HDF5 product. It removes waveform- and pixel-level data while retaining the event-level quantities needed for trigger-rate, effective-area, and Monte Carlo statistics studies. Input files can be supplied directly, through one text file, or through a glob pattern matching several text files; several input files can be combined into one output file.

Each output file contains the following root-level datasets:

  • SHOWERS: simulated energy, core position, shower direction, event IDs, and area weights for shower records.

  • TRIGGERS: array pointing and the telescope lists for triggered records.

  • FILE_INFO: one row per input file with file IDs and simulation settings, including particle, energy, viewcone, core-scatter, pointing, and NSB values.

  • METADATA: standard simtools product metadata.

  • SIMULATION_METADATA: versioned provenance for the input files and simulation software.

The file_id column joins SHOWERS and TRIGGERS to FILE_INFO; TRIGGERS can contain fewer rows than SHOWERS because it records only triggered events. Tables are stored as HDF5 compound datasets, with physical columns carrying their Astropy units.

Input and output#

Role

Argument

Format

Description

Input

input_files

sim_telarray

One or more *.simtel.zst files.

Input

input_file_list

Text file

One sim_telarray output file per line.

Input

input_file_list_pattern

Glob pattern

Text files containing one sim_telarray output file per line.

Output

output_path

Directory

Reduced-event HDF5 files with embedded metadata.

Provide exactly one of input_files, input_file_list, or input_file_list_pattern. A pattern processes each matching list independently and submits all resulting output batches together. Use max_workers to control parallel output-file processing.

If an HDF5 write fails, the retained incomplete file includes the application activity ID and the per-write staging ID in its name. The activity ID matches the UUID in the generated write_reduced_event_lists_<activity_id>.log file.

Command line arguments#

options#

–help

show this help message and exit

–input_files

Simulation file(s) to process (e.g., ‘*.simtel.zst’).

–input_file_list

Text file containing one simulation file per line.

–input_file_list_pattern

Glob pattern matching text files containing sim_telarray output file lists.

–files_per_reduced_event_file

Number of input files combined into each reduced event file (default: 1).

Default: 1.

–wait

Wait for HTCondor jobs and validate outputs before exiting.

simulation configuration#

–simulation_file_format

Simulation input format. Current IACT and sim_telarray files use ‘eventio’.

Default: eventio.

application#

–show_options

Print available values for a supported option and exit.

Example#

python src/simtools/applications/write_reduced_event_lists.py \
    --input_file_list_pattern "/lustre/fs25/group/cta/prod6/north/reduced_event_list/*.txt" \
    --output_path /lustre/fs25/group/cta/prod6/north/reduced_event_list \
    --files_per_reduced_event_file 10 \
    --backend htcondor \
    --backend_config htcondor.yml

Write reduced event lists from sim_telarray data#

Convert sim_telarray event streams to compact HDF5 event lists.

Application: simtools-write-reduced-event-lists

Integration test: write_reduced_event_lists.yml

Command

simtools-write-reduced-event-lists --config tests/integration_tests/config/write_reduced_event_lists.yml

Configuration

simulation_file_format: eventio
input_files:
- ${generated:proton_run000001_za20deg_azm180deg_North_CTAO-North-Alpha_7.0.0_test.simtel.zst}
output_path: simtools-output

Input file format#

simulation_file_format selects the reader used for the input simulation files. The default eventio supports current CORSIKA IACT and sim_telarray files, including their supported compression formats. The reduced HDF5 tables keep fixed fields and units independently of the input format.

See configuration and file formats for adding another reader. Only eventio is supplied currently.