Skip to content

scorer

autogen.beta.eval.scorer.scorer #

scorer(fn)

Decorate a function as a scorer.

Example::

@scorer
def called_get_weather(trace):
    return len(trace.events_of(ToolCallEvent, name="get_weather")) == 1

The decorated object is a :class:Scorer instance. Pass it directly to :func:autogen.beta.eval.run_agent via the scorers= argument.

Source code in autogen/beta/eval/scorer.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
def scorer(fn: ScorerFn) -> Scorer:
    """Decorate a function as a scorer.

    Example::

        @scorer
        def called_get_weather(trace):
            return len(trace.events_of(ToolCallEvent, name="get_weather")) == 1

    The decorated object is a :class:`Scorer` instance. Pass it directly
    to :func:`autogen.beta.eval.run_agent` via the ``scorers=`` argument.
    """
    return Scorer(fn)