Options YAML

The YAML provided to populate the options object passed to the user provided function, often called options.yaml, is parsed using the following model:

pydantic model Options[source]

A class that holds the user-specified DaVinci options.

This class inherits from the default GaudiConf.LbExec.Options.

This class also configures several PyConf functions, see the list in the apply_binds method, where their keyword arguments are globally bound to the user-specified values. This way, users do not have to manually configure these functions themselves.

The optional parameters that need to be set are : - input_stream (str): Stream name, internally converted to lowercase. Default is “default”.

(Note: for input_process=Hlt2 the stream must be strictly empty. The default value is overwritten in this case.)

  • lumi (bool): Flag to store luminosity information. Default value is True and if Data the “lumiTree” is written by default. If MC the “lumiTree” is not written, EventAccounting is scheduled instead.

  • write_fsr (bool): Flag to write the file summary record. Requires also output_file != None. Default is False.

  • merge_genfsr (bool): Flag to merge the file summary record. Only applies to simulation. Default is False.

  • metainfo_additional_tags: (list): Additional central tags for PyConf.filecontent_metadata.metainfo_repos. Default is [].

_input_config is an ugly hack so that the configuration can be retrieved in config.py This is needed due to the way DaVinci configures itself, which is kind of not compatible with the PyConf spirit

Show JSON schema
{
   "title": "Options",
   "description": "A class that holds the user-specified DaVinci options.\n\nThis class inherits from the default `GaudiConf.LbExec.Options`.\n\nThis class also configures several PyConf functions,\nsee the list in the `apply_binds` method,\nwhere their keyword arguments are globally bound to the user-specified values.\nThis way, users do not have to manually configure these functions themselves.\n\nThe optional parameters that need to be set are :\n- input_stream (str): Stream name, internally converted to lowercase. Default is \"default\".\n  (Note: for `input_process=Hlt2` the stream must be strictly empty. The default value is overwritten in this case.)\n- lumi (bool): Flag to store luminosity information. Default value is True and if Data the \"lumiTree\" is written by default. If MC the \"lumiTree\" is not written, EventAccounting is scheduled instead.\n- write_fsr (bool): Flag to write the file summary record. Requires also output_file != None. Default is False.\n- merge_genfsr (bool): Flag to merge the file summary record. Only applies to simulation. Default is False.\n- metainfo_additional_tags: (list): Additional central tags for `PyConf.filecontent_metadata.metainfo_repos`. Default is [].\n\n_input_config is an ugly hack so that the configuration can be retrieved in config.py\nThis is needed due to the way DaVinci configures itself, which is kind of not compatible with the PyConf spirit",
   "type": "object",
   "properties": {
      "data_type": {
         "anyOf": [
            {
               "$ref": "#/$defs/DataTypeEnum"
            },
            {
               "type": "null"
            }
         ],
         "default": "Upgrade"
      },
      "simulation": {
         "title": "Simulation",
         "type": "boolean"
      },
      "dddb_tag": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Dddb Tag"
      },
      "conddb_tag": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Conddb Tag"
      },
      "geometry_version": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Geometry Version"
      },
      "geometry_backend": {
         "default": "DD4Hep",
         "title": "Geometry Backend",
         "type": "string"
      },
      "conditions_version": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Conditions Version"
      },
      "input_files": {
         "default": [],
         "items": {
            "type": "string"
         },
         "title": "Input Files",
         "type": "array"
      },
      "input_type": {
         "$ref": "#/$defs/FileFormats",
         "default": "NONE"
      },
      "input_raw_format": {
         "default": 0.5,
         "title": "Input Raw Format",
         "type": "number"
      },
      "input_process": {
         "anyOf": [
            {
               "$ref": "#/$defs/InputProcessTypes"
            },
            {
               "type": "null"
            }
         ],
         "default": null
      },
      "input_manifest_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Input Manifest File"
      },
      "input_stream": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": "default",
         "title": "Input Stream"
      },
      "input_run_number": {
         "anyOf": [
            {
               "type": "integer"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Input Run Number"
      },
      "persistreco_version": {
         "default": 1.0,
         "title": "Persistreco Version",
         "type": "number"
      },
      "xml_file_catalog": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Xml File Catalog"
      },
      "evt_max": {
         "default": -1,
         "title": "Evt Max",
         "type": "integer"
      },
      "first_evt": {
         "default": 0,
         "title": "First Evt",
         "type": "integer"
      },
      "ioalg_buffer_nb_events": {
         "default": 20,
         "title": "Ioalg Buffer Nb Events",
         "type": "integer"
      },
      "mdf_ioalg_name": {
         "default": "IOAlgFileRead",
         "title": "Mdf Ioalg Name",
         "type": "string"
      },
      "root_ioalg_name": {
         "default": "RootIOAlg",
         "title": "Root Ioalg Name",
         "type": "string"
      },
      "root_ioalg_opts": {
         "additionalProperties": true,
         "default": {},
         "title": "Root Ioalg Opts",
         "type": "object"
      },
      "gaudipython_mode": {
         "default": false,
         "title": "Gaudipython Mode",
         "type": "boolean"
      },
      "output_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Output File"
      },
      "output_type": {
         "$ref": "#/$defs/FileFormats",
         "default": "ROOT"
      },
      "output_manifest_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Output Manifest File"
      },
      "append_decoding_keys_to_output_manifest": {
         "default": true,
         "title": "Append Decoding Keys To Output Manifest",
         "type": "boolean"
      },
      "write_decoding_keys_to_git": {
         "anyOf": [
            {
               "type": "boolean"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Write Decoding Keys To Git"
      },
      "write_options_to_fsr": {
         "default": false,
         "title": "Write Options To Fsr",
         "type": "boolean"
      },
      "require_specific_decoding_keys": {
         "default": [],
         "items": {
            "type": "string"
         },
         "title": "Require Specific Decoding Keys",
         "type": "array"
      },
      "compression": {
         "anyOf": [
            {
               "$ref": "#/$defs/CompressionSettings"
            },
            {
               "type": "null"
            }
         ],
         "default": null
      },
      "histo_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Histo File"
      },
      "ntuple_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Ntuple File"
      },
      "ntuple_basketsize": {
         "anyOf": [
            {
               "type": "integer"
            },
            {
               "type": "null"
            }
         ],
         "default": 32000,
         "title": "Ntuple Basketsize"
      },
      "xml_summary_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Xml Summary File"
      },
      "xml_summary_svc": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": "XMLSummarySvc",
         "title": "Xml Summary Svc"
      },
      "write_streams_attributes_to_json": {
         "default": false,
         "title": "Write Streams Attributes To Json",
         "type": "boolean"
      },
      "output_streams_attributes_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": "line_attribute_dict.json",
         "title": "Output Streams Attributes File"
      },
      "input_streams_attributes_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": "line_attribute_dict.json",
         "title": "Input Streams Attributes File"
      },
      "n_threads": {
         "default": 1,
         "title": "N Threads",
         "type": "integer"
      },
      "n_event_slots": {
         "title": "N Event Slots",
         "type": "integer"
      },
      "event_store": {
         "$ref": "#/$defs/EventStores",
         "default": "HiveWhiteBoard"
      },
      "memory_pool_size": {
         "default": 10485760,
         "title": "Memory Pool Size",
         "type": "integer"
      },
      "scheduler_legacy_mode": {
         "default": true,
         "title": "Scheduler Legacy Mode",
         "type": "boolean"
      },
      "print_freq": {
         "default": 10000,
         "title": "Print Freq",
         "type": "integer"
      },
      "output_level": {
         "default": 3,
         "title": "Output Level",
         "type": "integer"
      },
      "msg_svc_format": {
         "default": "% F%35W%S %7W%R%T %0W%M",
         "title": "Msg Svc Format",
         "type": "string"
      },
      "msg_svc_time_format": {
         "default": "%Y-%m-%d %H:%M:%S UTC",
         "title": "Msg Svc Time Format",
         "type": "string"
      },
      "python_logging_level": {
         "default": 20,
         "title": "Python Logging Level",
         "type": "integer"
      },
      "monitoring_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Monitoring File"
      },
      "control_flow_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Control Flow File"
      },
      "data_flow_file": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Data Flow File"
      },
      "phoenix_filename": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Phoenix Filename"
      },
      "preamble_algs": {
         "default": [],
         "items": {},
         "title": "Preamble Algs",
         "type": "array"
      },
      "auditors": {
         "default": [],
         "items": {
            "type": "string"
         },
         "title": "Auditors",
         "type": "array"
      },
      "event_timeout": {
         "anyOf": [
            {
               "type": "integer"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Event Timeout"
      },
      "force_odin": {
         "default": false,
         "title": "Force Odin",
         "type": "boolean"
      },
      "lumi": {
         "default": true,
         "title": "Lumi",
         "type": "boolean"
      },
      "write_fsr": {
         "default": false,
         "title": "Write Fsr",
         "type": "boolean"
      },
      "merge_genfsr": {
         "default": false,
         "title": "Merge Genfsr",
         "type": "boolean"
      },
      "metainfo_additional_tags": {
         "anyOf": [
            {
               "items": {},
               "type": "array"
            },
            {
               "type": "null"
            }
         ],
         "default": [],
         "title": "Metainfo Additional Tags"
      }
   },
   "$defs": {
      "CompressionAlgs": {
         "enum": [
            "ZLIB",
            "LZMA",
            "LZ4",
            "ZSTD"
         ],
         "title": "CompressionAlgs",
         "type": "string"
      },
      "CompressionSettings": {
         "additionalProperties": false,
         "properties": {
            "algorithm": {
               "$ref": "#/$defs/CompressionAlgs",
               "default": "ZSTD"
            },
            "level": {
               "default": 4,
               "title": "Level",
               "type": "integer"
            },
            "min_buffer_size": {
               "anyOf": [
                  {
                     "type": "integer"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Min Buffer Size"
            },
            "max_buffer_size": {
               "anyOf": [
                  {
                     "type": "integer"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Max Buffer Size"
            },
            "approx_events_per_basket": {
               "anyOf": [
                  {
                     "type": "integer"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Approx Events Per Basket"
            }
         },
         "title": "CompressionSettings",
         "type": "object"
      },
      "DataTypeEnum": {
         "enum": [
            "Upgrade",
            "2022",
            "2023",
            "2024",
            "2025"
         ],
         "title": "DataTypeEnum",
         "type": "string"
      },
      "EventStores": {
         "enum": [
            "EvtStoreSvc",
            "HiveWhiteBoard"
         ],
         "title": "EventStores",
         "type": "string"
      },
      "FileFormats": {
         "enum": [
            "NONE",
            "RAW",
            "ROOT"
         ],
         "title": "FileFormats",
         "type": "string"
      },
      "InputProcessTypes": {
         "description": "`TurboPass`, `Hlt2` and `Spruce` are standard run-3 input process types.\nThe extra types are available to effectively flag special input types\nand be able to apply a filter only to processes where a `DstData` bank is produced (the standard 3 types).\nSee discussions at https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/3990 and related MR for further details.",
         "enum": [
            "Spruce",
            "TurboPass",
            "Hlt2",
            "TurboSpruce",
            "Gen",
            "Brunel",
            "Boole",
            "Hlt1"
         ],
         "title": "InputProcessTypes",
         "type": "string"
      }
   },
   "additionalProperties": false,
   "required": [
      "simulation",
      "n_event_slots"
   ]
}

Config:

  • use_enum_values: bool = True

  • frozen: bool = True

  • extra: str = forbid

Validators:
field append_decoding_keys_to_output_manifest: bool = True
Validated by:
field auditors: list[str] = []
Validated by:
field compression: CompressionSettings | None = None
Validated by:
field conddb_tag: str | None = None
Validated by:
field conditions_version: str | None = None

Input

Validated by:
field control_flow_file: str | None = None
Validated by:
field data_flow_file: str | None = None
Validated by:
field data_type: DataTypeEnum | None = 'Upgrade'
Validated by:
field dddb_tag: str | None = None
Validated by:
field event_store: EventStores = EventStores.HiveWhiteBoard
Validated by:
field event_timeout: int | None = None
Validated by:
field evt_max: int = -1
Validated by:
field first_evt: int = 0
Validated by:
field force_odin: bool = False
Validated by:
field gaudipython_mode: bool = False

Output

Validated by:
field geometry_backend: str = 'DD4Hep'
Validated by:
field geometry_version: str | None = None
Validated by:
field histo_file: str | None = None
Validated by:
field input_files: list[str] = []
Validated by:
field input_manifest_file: str | None = None
Validated by:
field input_process: InputProcessTypes | None = None
Validated by:
field input_raw_format: float = 0.5
Validated by:
field input_run_number: int | None = None
Validated by:
field input_stream: str | None = 'default'
Validated by:
field input_streams_attributes_file: str | None = 'line_attribute_dict.json'

Processing

Validated by:
field input_type: FileFormats = FileFormats.NONE
Validated by:
field ioalg_buffer_nb_events: int = 20
Validated by:
field lumi: bool = True
Validated by:
field mdf_ioalg_name: str = 'IOAlgFileRead'
Validated by:
field memory_pool_size: int = 10485760
Validated by:
field merge_genfsr: bool = False
Validated by:
field metainfo_additional_tags: list | None = []
Validated by:
field monitoring_file: str | None = None
Validated by:
field msg_svc_format: str = '% F%35W%S %7W%R%T %0W%M'
Validated by:
field msg_svc_time_format: str = '%Y-%m-%d %H:%M:%S UTC'
Validated by:
field n_event_slots: Annotated[int, Field(validate_default=True)] [Required]
Validated by:
field n_threads: int = 1
Validated by:
field ntuple_basketsize: int | None = 32000
Validated by:
field ntuple_file: str | None = None
Validated by:
field output_file: str | None = None
Validated by:
field output_level: int = 3
Validated by:
field output_manifest_file: str | None = None
Validated by:
field output_streams_attributes_file: str | None = 'line_attribute_dict.json'
Validated by:
field output_type: FileFormats = FileFormats.ROOT
Validated by:
field persistreco_version: float = 1.0
Validated by:
field phoenix_filename: str | None = None
Validated by:
field preamble_algs: list = []
Validated by:
field print_freq: int = 10000
Validated by:
field python_logging_level: int = 20

Debugging

Validated by:
field require_specific_decoding_keys: list[str] = []
Validated by:
field root_ioalg_name: str = 'RootIOAlg'
Validated by:
field root_ioalg_opts: dict = {}
Validated by:
field scheduler_legacy_mode: bool = True

Logging

Validated by:
field simulation: bool [Required]
Validated by:
field write_decoding_keys_to_git: bool | None = None
Validated by:
field write_fsr: bool = False
Validated by:
field write_options_to_fsr: bool = False
Validated by:
field write_streams_attributes_to_json: bool = False
Validated by:
field xml_file_catalog: str | None = None
Validated by:
field xml_summary_file: str | None = None
Validated by:
field xml_summary_svc: str | None = 'XMLSummarySvc'
Validated by:
validator glob_input_files  »  input_files[source]
validator n_event_slots_default  »  all fields[source]
validator parse_compression_str  »  compression[source]
apply_binds()[source]

This function configures the following PyConf functions, where their keyword arguments are globally bound to the user-specified values: - default_raw_event - reconstruction - tes_root - tes_root_for_tistos This way users do not have to manually configure these functions themselves.

finalize()[source]
validator validate_fsr_options  »  all fields[source]

Validator for the consistency of writing FSRs and other settings.

validator validate_input  »  all fields[source]
validator validate_input_output  »  all fields[source]

Validator for the consistency of the input process and input/output file formats for Sprucing jobs.

class DataTypeEnum(value)[source]
DT2022 = '2022'
DT2023 = '2023'
DT2024 = '2024'
DT2025 = '2025'
Upgrade = 'Upgrade'
class FileFormats(value)[source]
NONE = 'NONE'
RAW = 'RAW'
ROOT = 'ROOT'
class EventStores(value)[source]
EvtStoreSvc = 'EvtStoreSvc'
HiveWhiteBoard = 'HiveWhiteBoard'