DeepEval
Workflow, limitations and sourcesPython evaluation framework for test cases, metrics and agent trajectories.
deepeval.com/docs/getting-startedAI coding assistants now handle whole tasks rather than single lines: reading a repository, planning a change, editing several files and running tests. Evaluate them on how well they understand your codebase, how they handle review, and whether they can be constrained to your conventions.
Python evaluation framework for test cases, metrics and agent trajectories.
deepeval.com/docs/getting-startedLLM vulnerability scanner using probes and detectors to surface failure modes.
github.com/NVIDIA/garakDesktop and terminal agent for coding and multi-step workflows.
github.com/aaif-goose/gooseFramework for model evaluations composed from datasets, solvers and scorers.
inspect.aisi.org.uk/AI coding agent with terminal, desktop and IDE interfaces.
opencode.ai/docs/Evaluation library for AI applications, including retrieval-augmented generation.
docs.ragas.io/en/stable/