Here’s a list of evals tools and frameworks I’ve been profiling:

• OpenAI Evals: Open-source, from OpenAI.
• LangSmith: Observability and Evals.
• PromptPex: More focused on prompt testing.
• ChainForge: Prompt robustness testing visual UI.
• PromptLayer: Full end-to-end AI testing and monitoring.
• Garak: Security-focused, from NVIDIA.

ChainForge example flow. In this example ChainForge evaluates model robustness to prompt injection attacks.