mendevi.database.meta¶

Help to get the good extractor.

Classes

ExtractContext(label, func, is_log)

Create new instance of ExtractContext(label, func, is_log)

Functions

`extract_names`(expr)	Return all the symbols in the python expression.
`get_extractor`(name[, safe])	Get the way to deserialize a raw value.
`merge_extractors`(labels[, alias, select, ...])	Return the source code of the function that extracts all variables.

Details

class mendevi.database.meta.ExtractContext(label, func, is_log): Create new instance of ExtractContext(label, func, is_log)

mendevi.database.meta.extract_names(expr: str) → set[str][source]

Return all the symbols in the python expression.

Examples¶

>>> from mendevi.database.meta import extract_names
>>> extract_names("foo")
{'foo'}
>>> extract_names("[i**2 for i in foo]"")
{'foo'}
>>> extract_names("foo.bar")
{'foo'}
>>> extract_names("bar(foo)")
{'foo'}
>>> extract_names("foo.bar()")
{'foo'}
>>>

mendevi.database.meta.get_extractor(name: str, safe: bool = False) → ExtractContext[source]

Get the way to deserialize a raw value.

Parameters¶

namestr: The label name.
safeboolean, default=False: If True, retrun a stupid value instead of raising KeyError.

Returns¶

labelstr: The description of the physical quantity. This description can be used to label the axes of a graph.
funccallable | str: The function that performs the verification and deserialisation task, or the formula that allows you to find this quantity.
is_logboolean or None: True to display in log space, False for linear. The value None means the axis is not continuous.

mendevi.database.meta.merge_extractors(labels: set[str], alias: dict[str, str] | None = None, select: str | None = None, return_callable: bool = False) → tuple[set[str]][source]

Return the source code of the function that extracts all variables.

Parameters¶

labelsset[str]: The returned variable names. These are the keys to the output dictionary.
aliasdict[str, str], optional: By default, the label extraction method is defined by the function get_extractor(). This list of aliases allows any unknown key to define a customised access method.
selectstr, optional: A Python Boolean expression that raises a RejectError exception if it evaluates to False.
return_callableboolean, default=False: By default, returns the source code of the function. If this option is set to True, an executable function is returned.

Returns¶

lbls_atomset[str]: The name of the primary value to be extracted for the SQL query.
funclist[str] or callable: The function that consumes a line from the SQL query, and returns the dictionary of extracted values.

Examples¶

>>> from mendevi.database.meta import merge_extractors
>>> print("\n".join(merge_extractors({"rate", "enc_scenario"}, select="'yeti' in hostname")[1]))
def line_extractor(raw: dict[str]) -> dict[str]:
    """Get the labels: enc_scenario, rate, reject."""
    hostname = extract.extract_hostname(raw)
    reject = not ('yeti' in hostname)
    if reject:
        raise RejectError("this line must be filtered")
    encode_cmd = extract.extract_encode_cmd(raw)
    size = extract.extract_video_size(raw)
    video_name = extract.extract_video_name(raw)
    video_duration = extract.extract_video_duration(raw)
    enc_scenario = f"cmd: {encode_cmd}, video_name: {video_name}, hostname: {hostname}"
    rate = None if size is None or video_duration is None else 8.0 * size / video_duration
    return {
        'enc_scenario': enc_scenario,
        'rate': rate,
        'reject': reject,
    }