mendevi.database.meta

Help to get the good extractor.

Classes

ExtractContext(label, func, is_log)

Create new instance of ExtractContext(label, func, is_log)

Functions

extract_names(expr)	Return all the symbols in the python expression.
get_extractor(name[, safe])	Get the way to deserialize a raw value.
merge_extractors(labels[, alias, select, ...])	Return the source code of the function that extracts all variables.

Details

class mendevi.database.meta.ExtractContext(label, func, is_log)

Create new instance of ExtractContext(label, func, is_log)

mendevi.database.meta.extract_names(expr: str) → set[str]

Return all the symbols in the python expression.

Examples

>>> from mendevi.database.meta import extract_names
>>> extract_names("foo")
{'foo'}
>>> extract_names("[i**2 for i in foo]"")
{'foo'}
>>> extract_names("foo.bar")
{'foo'}
>>> extract_names("bar(foo)")
{'foo'}
>>> extract_names("foo.bar()")
{'foo'}
>>>

mendevi.database.meta.get_extractor(name: str, safe: bool = False) → ExtractContext

Get the way to deserialize a raw value.

Parameters

namestr

The label name.

safeboolean, default=False

If True, retrun a stupid value instead of raising KeyError.

Returns

labelstr

The description of the physical quantity. This description can be used to label the axes of a graph.

funccallable | str

The function that performs the verification and deserialisation task, or the formula that allows you to find this quantity.

is_logboolean or None

True to display in log space, False for linear. The value None means the axis is not continuous.

mendevi.database.meta.merge_extractors(labels: set[str], alias: dict[str, str] | None = None, select: str | None = None, return_callable: bool = False) → tuple[set[str]]

Return the source code of the function that extracts all variables.

Parameters

labelsset[str]

The returned variable names. These are the keys to the output dictionary.

aliasdict[str, str], optional

By default, the label extraction method is defined by the function get_extractor(). This list of aliases allows any unknown key to define a customised access method.

selectstr, optional

A Python Boolean expression that raises a RejectError exception if it evaluates to False.

return_callableboolean, default=False

By default, returns the source code of the function. If this option is set to True, an executable function is returned.

Returns

lbls_atomset[str]

The name of the primary value to be extracted for the SQL query.

funclist[str] or callable

The function that consumes a line from the SQL query, and returns the dictionary of extracted values.

Examples

>>> from mendevi.database.meta import merge_extractors
>>> print("\n".join(merge_extractors({"rate", "enc_scenario"}, select="'yeti' in hostname")[1]))
def line_extractor(raw: dict[str]) -> dict[str]:
    """Get the labels: enc_scenario, rate, reject."""
    hostname = extract.extract_hostname(raw)
    reject = not ('yeti' in hostname)
    if reject:
        raise RejectError("this line must be filtered")
    encode_cmd = extract.extract_encode_cmd(raw)
    size = extract.extract_video_size(raw)
    video_name = extract.extract_video_name(raw)
    video_duration = extract.extract_video_duration(raw)
    enc_scenario = f"cmd: {encode_cmd}, video_name: {video_name}, hostname: {hostname}"
    rate = None if size is None or video_duration is None else 8.0 * size / video_duration
    return {
        'enc_scenario': enc_scenario,
        'rate': rate,
        'reject': reject,
    }