Loading...
Loading...
Design MLflow scorers and run GenAI evaluation on an agent, dataset, precomputed outputs, or traces. Use for quality criteria, judges, MemAlign, datasets, evaluation runs, and results analysis.
npx skill4agent add mlflow/skills mlflow-evaluationmlflow.genai.datasets@scorermake_judgemlflow.genai.evaluate()MLFLOW_TRACKING_URIMLFLOW_EXPERIMENT_ID.envmlflow-onboardinguv runmlflow-trace-instrumentationinputsoutputspredict_fnmlflow scorers list -x "$MLFLOW_EXPERIMENT_ID"required_columnsCorrectnessRETRIEVERTOOL"yes""no""pass""fail"# MLflow calls predict_fn(**record["inputs"]). Nested values stay nested.
record = {"inputs": {"query": "Where is order 42?", "context": {"order_id": 42}}}
def predict(query, context):
return agent(query=query, context=context) # bind project config here as neededinputsexpectationsexpectations.expected_responseFalse"no"mlflow.genai.evaluate(data=selected_data, predict_fn=predict, scorers=registered_scorers)predict_fnmlflow-agent-improvementmlflow-evaluation