Skip to content

PairwiseComparator

autogen.beta.eval.pairwise.PairwiseComparator #

Bases: Protocol

Decides A vs B for one task. LLM / human / custom implementations interchange.

key instance-attribute #

key

Result key this comparator reports under (its column in the result).

compare async #

compare(*, task, trace_a, trace_b, reference_outputs)
Source code in autogen/beta/eval/pairwise.py 
78
79
80
81
82
83
84
85
async def compare(
    self,
    *,
    task: Task,
    trace_a: Trace,
    trace_b: Trace,
    reference_outputs: dict[str, Any] | None,
) -> PairwiseOutcome: ...