# dorie.parscraper¶

## What is dorie.parscraper?¶

dorie.parscraper is a python package with the purpose of automatic creation of DORiE parameter files in various formats. The developers only need to maintain one single XML file that contains all the relevant information for each parameter.

It provides the following components:

Another purpose is to identify mismatches between XML meta-information and source code and notify the developers. Therefore, a warning is issued if either a parameter is only found in the XML file, or only found in the DORiE source code.

### How to use dorie.parscraper¶

The dorie.parscraper scripts are called via CMake while configuring DORiE. This is to make sure that the generated ini files are accessible as soon as the system tests are created.

## The dorie.parscraper scripts¶

scrape_folder.py

This script basically just parses the command line arguments, implements special handling of warnings and exceptions to keep the output as clean as possible, and calls dorie.parscraper.wrapper.scrape_folder.

Command line options:

--xml

Path to the XML file holding parameter meta-information

--source

The C++ source folder to scrape

--out

Output file(s)

--css

Path to a CSS file to include into HTML output (optional)

--debug

Show warnings (optional, default off)

## The dorie.parscraper core modules¶

### Parameter class¶

class dorie.parscraper.parameter.Parameter(category, key)[source]

Implements a class for storing DORiE parameters.

Parameters: category (str) – The category of the parameter, indicated in the DUNE infiles by square brackets. key (str) – The name of the parameter, at the left hand side of the equal sign in DUNE inifiles.
_code_found = None

Whether the parameter was found in the code at all

_sources = None

List of all the positions in the code where the parameter was found, encoded in a tuple: (filename, lineno, C++ type)

category = None

Parameter category

comment = None

Extra comment showing up in the generated ini files

definition = None

Long explanation of the parameter, shown in the documentation

hidden = None

Do not issue an error if parameter is not found

key = None

Parameter name

suggestion = None

Suggested (default) value

values = None

Possible values

### Warnings¶

exception dorie.parscraper.warnings.OutputWarning[source]
exception dorie.parscraper.warnings.ScraperWarning[source]
exception dorie.parscraper.warnings.XMLWarning[source]

### Matching function¶

dorie.parscraper.match_parameters.match(xml_parameters, source_parameters, base_path)[source]

Expects the output of dorie.parscraper.readers.xml and dorie.parscraper.readers.source() as input, and tries to match the parameters with each other. Records, where the parameters specified in the XML file are found in the code (file name, line number, and which type they are cast to), and supplements the Parameter instances with this information.

Returns an OrderedDict with categories as keys and lists of Parameter instances as values. Parameters that are only found in the source files, but are not specified in XML, are not included in the output. Parameters that are only specified in the XML file but not in the code are returned, but a warning is issued.

Parameters: xml_parameters – Output of dorie.parscraper.readers.xml.parse() source_parameters – Output of dorie.parscraper.readers.source.parse()

### Wrapper¶

dorie.parscraper.wrapper.scrape_folder.scrape(xml_file, source_folder, out, css=None, debug=False)[source]

dorie.parscraper.readers.xml.parse(filename)[source]

Reads and parses a DORiE parameter XML file, using the xml.etree.ElementTree package.

The XML file is expected to have the following structure:

<dorie> ‣ <category name=”…”> ‣ <parameter name=”…”> ‣ (attributes)

Available attributes are all attributes of the dorie.parscraper.parameter.Parameter class, except the ones starting with an underscore.

Parameters: filename (str) – Path to the input XML file OrderedDict all parsed category names as keys, and a list of all parameters in the corresponding category as values (instances of Parameter) XMLWarning – if unknown, redundant, or malformatted XML elements are encountered

#### C++ source reader¶

dorie.parscraper.readers.source.parse(filename)[source]

Reads a source file and looks for patterns like

inifile.get<type>("category.test.parameter",default)


When found, the relevant information gets scraped and assembled in a dict.

Note

You can prevent a parameter to be matched by adding // %HIDDEN% to the end of the corresponding line.

Parameters: filename (str) – Full path to the input file list of dicts (one dict for each matching pattern in the source file), each containing all found information

### Writers¶

#### .ini file writer¶

dorie.parscraper.writers.ini.write(parameters, out, *args, **kwargs)[source]

Writes the contents of the parameters input dict to a .ini file. The output is structured as

[key]
# value.comment
value.key = value.suggestion


to match the format of the DUNE input files. A missing comment attribute leads to omitting of the comment line; a missing suggestion to the value UNDEFINED.

Parameters: parameters (dict) – dict with categories as keys and iterables of Parameter instances as values. out (str) – Path to the output file. Must be writable.

#### .rst file writer¶

dorie.parscraper.writers.rst.write(parameters, out, path_base, *args, **kwargs)[source]

Writes the contents of the parameters input dict to a .rst file. The output is structured as

category

Parameter Definition Possible values Default Queried at
p.key p.definition p.values p.suggestion all sources of p
Parameters: parameters (dict) – dict with categories as keys and iterables of dorie.parscraper.parameter.Parameter instances as values. out (str) – Path to the output file. Must be writable. path_base (str) – Base path to the source files. Needed for creation of working links to the files.

#### .html file writer¶

dorie.parscraper.writers.html.write(parameters, out, path_base, *args, **kwargs)[source]

Writes the contents of the parameters input dict to a .html file. The output is structured as

category

Parameter Definition Possible values Default Queried at
p.key p.definition p.values p.suggestion all sources of p
Parameters: parameters (dict) – dict with categories as keys and iterables of dorie.parscraper.parameter.Parameter instances as values. out (str) – Path to the output file. Must be writable. path_base (str) – Base path to the source files. Needed for creation of working links to the files. css (str) – Path to a CSS file containing the styling of the HTML output. Is included into the HTML output (inline). Optional.