​
eval_function_completions

eval_function_completions(
    responses: list[str],
    definition: str,
    test: str | None = None,
    entry_point: str | None = None,
    assertions: str | Callable[[str], tuple[str, float]] | None = None,
    timeout: float | None = 3,
    use_docker: bool | None = True
) -> dict

(openai<1) Select a response from a list of responses for the function completion task (using generated assertions), and/or evaluate if the task is successful using a gold test.

Parameters:
NameDescription
responsesThe list of responses.

Type: list[str]
definitionThe input definition.

Type: str
testThe test code.

Type: str | None

Default: None
entry_pointThe name of the function.

Type: str | None

Default: None
assertionsThe assertion code which serves as a filter of the responses, or an assertion generator.

When provided, only the responses that pass the assertions will be considered for the actual test (if provided).

Type: str | Callable[[str], tuple[str, float]] | None

Default: None
timeoutThe timeout for executing the code.

Type: float | None

Default: 3
use_dockerType: bool | None

Default: True
Returns:
TypeDescription
dictdict: The success metrics.

decide_use_dockerexecute_code