ME News reports that on April 17 (UTC+8), according to monitoring by Beating, LangSmith, the observability tool from LangChain, an AI agent development platform, has released two updates: an evaluator template library and reusable evaluators. Assessing whether an AI agent is “usable” is one of the most time-consuming tasks in current development. Agents may invoke the correct tools but produce incorrect answer formats, perform well in single-turn conversations but crash in multi-turn ones, or deliver seemingly reasonable final answers while retrieving wrong documents during intermediate steps. Developers must set checkpoints across multiple levels—single steps, full trajectories, multi-turn dialogues, and specific tool calls—each requiring custom evaluators developed through writing prompts, calibrating against real data, and iterative tuning, often taking weeks from scratch. LangSmith now offers over 30 ready-to-use templates spanning five categories: Security & Protection (prompt injection detection, PII leakage checks, bias and toxicity), Answer Quality (correctness, usefulness, tone), Execution Trajectory (whether the agent took the correct steps), User Behavior Analysis (language distribution, satisfaction signals), and Multimodal (review of audio and image outputs). Each template includes pre-tuned LLM evaluation prompts and rule-based code evaluators that can be used directly or customized, and are applicable for both live monitoring and offline experimentation. Reusable evaluators address organizational management challenges: the new Evaluators tab centrally displays all evaluators within a workspace, allowing one-click attachment to new projects. Updates to prompts take effect globally across all projects, eliminating the need to maintain duplicate copies in each individual project. These templates are now open-sourced alongside openevals v0.2.0, which adds support for multimodal evaluation. (Source: BlockBeats)
LangSmith Launches 30+ Evaluation Templates for AI Agent Quality Testing
KuCoinFlashShare
Summary
AI and crypto news broke on April 17 (UTC+8) as LangChain’s LangSmith launched over 30 evaluation templates for AI agent testing. The update includes an evaluator template library and reusable evaluators across five categories: safety, response quality, execution trajectory, user behavior analysis, and multimodal. These templates support both online monitoring and offline experiments, featuring optimized LLM prompts and rule-based code. The new Evaluators tab enables centralized management. The templates are open-sourced with openevals v0.2.0, now adding multimodal support. New token listings and AI tools continue to shape the market.
Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information.
Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.