Database support#
Modules for database access. See the databases sections for details.
db_handler#
Module to handle interaction with DB.
- class db.db_handler.DatabaseHandler(mongo_db_config=None)[source]#
DatabaseHandler provides the interface to the DB.
- Parameters:
- mongo_db_config: dict
Dictionary with the MongoDB configuration (see jsonschema_db_dict for details).
- add_new_parameter(db_name, par_dict, collection_name='telescopes', file_prefix=None)[source]#
Add a new parameter dictionary to the DB.
A new document will be added to the DB, with all fields taken from the input parameters. Parameter dictionaries are validated before submission using the corresponding schema.
- Parameters:
- db_name: str
the name of the DB
- par_dict: dict
dictionary with parameter data
- collection_name: str
The name of the collection to add a parameter to.
- file_prefix: str or Path
where to find files to upload to the DB
- add_production_table(db_name, production_table)[source]#
Add a production table to the DB.
- Parameters:
- db_name: str
the name of the DB.
- production_table: dict
The production table to add to the DB.
- export_model_file(parameter, site, array_element_name, model_version=None, parameter_version=None, export_file_as_table=False)[source]#
Export single model file from the DB identified by the parameter name.
The parameter can be identified by model or parameter version. Files can be exported as astropy tables (ecsv format).
- Parameters:
- parameter: str
Name of the parameter.
- site: str
Site name.
- array_element_name: str
Name of the array element model (e.g. MSTN, SSTS).
- parameter_version: str
Version of the parameter.
- model_version: str
Version of the model.
- export_file_as_table: bool
If True, export the file as an astropy table (ecsv format).
- export_model_files(parameters=None, file_names=None, dest=None, db_name=None)[source]#
Export models files from the DB to given directory.
The files to be exported can be specified by file_name or retrieved from the database using the parameters dictionary.
- Parameters:
- parameters: dict
Dict of model parameters
- file_names: list, str
List (or string) of file names to export
- dest: str or Path
Location where to write the files to.
- Returns:
- file_id: dict of GridOut._id
Dict of database IDs of files.
- get_array_elements(model_version, collection='telescopes')[source]#
Get list array elements for a given model version and collection from the DB.
- Parameters:
- model_version: str
Version of the model.
- collection: str
Which collection to get the array elements from: i.e. telescopes, calibration_devices.
- Returns:
- list
Sorted list of all array elements found in collection
- get_array_elements_of_type(array_element_type, model_version, collection)[source]#
Get array elements of a certain type (e.g. ‘LSTN’) for a DB collection.
Does not return ‘design’ models.
- Parameters:
- array_element_type: str
Type of the array element (e.g. LSTN, MSTS).
- model_version: str
Version of the model.
- collection: str
Which collection to get the array elements from: i.e. telescopes, calibration_devices.
- Returns:
- list
Sorted list of all array element names found in collection
- get_collection(db_name, collection_name)[source]#
Get a collection from the DB.
- Parameters:
- db_name: str
Name of the DB.
- collection_name: str
Name of the collection.
- Returns:
- pymongo.collection.Collection
The collection from the DB.
- get_collections(db_name=None, model_collections_only=False)[source]#
List of collections in the DB.
- Parameters:
- db_name: str
Database name.
- model_collections_only: bool
If True, only return model collections (i.e. exclude fs.files, fs.chunks)
- Returns:
- list
List of collection names
- get_design_model(model_version, array_element_name, collection='telescopes')[source]#
Get the design model used for a given array element and a given model version.
- Parameters:
- model_version: str
Version of the model.
- array_element_name: str
Name of the array element model (e.g. MSTN, SSTS).
- collection: str
Which collection to get the array elements from: i.e. telescopes, calibration_devices.
- Returns:
- str
Design model for a given array element.
- get_model_parameter(parameter, site, array_element_name, parameter_version=None, model_version=None)[source]#
Get a single model parameter using model or parameter version.
Note that this function should not be called in a loop for many parameters, as it each call queries the database.
- Parameters:
- parameter: str
Name of the parameter.
- site: str
Site name.
- array_element_name: str
Name of the array element model.
- parameter_version: str
Version of the parameter.
- model_version: str
Version of the model.
- Returns:
- dict containing the parameter
- get_model_parameters(site, array_element_name, collection, model_version)[source]#
Get model parameters using the model version.
Queries parameters for design and for the specified array element (if necessary).
- Parameters:
- site: str
Site name.
- array_element_name: str
Name of the array element model (e.g. LSTN-01, MSTx-FlashCam, ILLN-01).
- model_version: str, list
Version(s) of the model.
- collection: str
Collection of array element (e.g. telescopes, calibration_devices).
- Returns:
- dict containing the parameters
- get_model_parameters_for_all_model_versions(site, array_element_name, collection)[source]#
Get model parameters for all model versions.
- Parameters:
- site: str
Site name.
- array_element_name: str
Name of the array element model (e.g. LSTN-01, MSTx-FlashCam, ILLN-01).
- collection: str
Collection of array element (e.g. telescopes, calibration_devices).
- Returns:
- dict containing the parameters with model version as first key
- get_model_versions(collection_name='telescopes')[source]#
Get list of model versions from the DB.
- Parameters:
- collection_name: str
Name of the collection.
- Returns:
- list
List of model versions
- get_simulation_configuration_parameters(simulation_software, site, array_element_name, model_version)[source]#
Get simulation configuration parameters from the DB.
- Parameters:
- simulation_software: str
Name of the simulation software.
- site: str
Site name.
- array_element_name: str
Name of the array element model (e.g. MSTN, SSTS).
- model_version: str
Version of the model.
- Returns:
- dict containing the parameters
- Raises:
- ValueError
if simulation_software is not valid.
- insert_file_to_db(file_name, db_name=None, **kwargs)[source]#
Insert a file to the DB.
- Parameters:
- file_name: str or Path
The name of the file to insert (full path).
- db_name: str
the name of the DB
- **kwargs (optional): keyword arguments for file creation.
The full list of arguments can be found in, https://docs.mongodb.com/manual/core/gridfs/#the-files-collection mostly these are unnecessary though.
- Returns:
- file_iD: GridOut._id
If the file exists, return its GridOut._id, otherwise insert the file and return its” “newly created DB GridOut._id.
db_model_upload#
Upload a simulation model (parameters and production tables) to the database.
- db.db_model_upload.add_model_parameters_to_db(args_dict, db)[source]#
Read model parameters from a directory and upload them to the database.
- Parameters:
- args_dictdict
Command line arguments.
- dbDatabaseHandler
Database handler object.
- db.db_model_upload.add_production_tables_to_db(args_dict, db)[source]#
Read production tables from a directory and upload them to the database.
One dictionary per collection is prepared for each model version, containing tables of all array elements, sites, and configuration parameters.
- Parameters:
- args_dictdict
Command line arguments.
- dbDatabaseHandler
Database handler object.
- db.db_model_upload.add_values_from_json_to_db(file, collection, db, db_name, file_prefix)[source]#
Upload new model parameter from json files to db.
- Parameters:
- filelist
Json file to be uploaded to the DB.
- collectionstr
The DB collection to which to add the file.
- dbDatabaseHandler
Database handler object.
- db_namestr
Name of the database to be created.
- file_prefixstr
Path to location of all additional files to be uploaded.