Skip to content

Scorer

autogen.beta.eval.scorer.Scorer #

Scorer(fn, *, key=None)

Wraps a user function as a callable scoring unit.

Most users will not construct this directly — they write a function and decorate it with :func:scorer. Prebuilt scorer factories (e.g. tool_called(name)) construct :class:Scorer instances programmatically to supply a meaningful key independent of any closure's function name.

PARAMETER DESCRIPTION
fn

The scoring function. May be sync or async. May declare any subset of the injectable parameters (inputs, outputs, reference_outputs, trace, task); anything else raises TypeError at construction time.

TYPE: ScorerFn

key

Stable identifier for the feedback this scorer produces. Defaults to fn.__name__. Pass-rate and stats lookups on :class:~autogen.beta.eval.RunResult use this key.

TYPE: str | None DEFAULT: None

Source code in autogen/beta/eval/scorer.py 
 95
 96
 97
 98
 99
100
101
102
103
104
def __init__(
    self,
    fn: ScorerFn,
    *,
    key: str | None = None,
) -> None:
    self._fn = fn
    self._key = key if key is not None else getattr(fn, "__name__", "scorer")
    self._params = _validate_signature(fn, self._key)
    self._is_async = inspect.iscoroutinefunction(fn)

key property #

key

The feedback key this scorer emits (defaults to the wrapped function's name).