Tag: #langsmith
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 5 posts
5 posts are tagged #langsmith; the most recent was published on 2026-05-16.
Often tagged together: #english 3 #langfuse 3 #2026-03 2 #arize 2 #deep-dive 2
Most read in the last 90 days:
- Agent Evaluation Systems in 2026 — Inspect AI vs Promptfoo vs Phoenix vs LangSmith vs OpenAI Evals (You're Measuring the Agent, Not the Model)
- Comparing LLM Production Monitoring Platforms: A Practical Operations Guide for LangSmith, LangFuse, and Arize Phoenix
- LLM Evaluation & Observability: Eval Harness, LLM-as-Judge, Tracing, Regression Prevention (2025)
LLM Observability & Prompt Tools 2026 — Helicone / LangSmith / Langfuse / Braintrust / Athina / Comet Opik / Portkey Deep Dive ♪ Listenable
The 2026 LLM ops map. Helicone (YC) · LangSmith (LangChain) · Langfuse (open source, Series A) · W&B Weave · Arize Phoenix · Braintrust · Athina · Comet Opik (released March 2025) · Vellum · PromptHub · Portkey AI Gatewa
2026-05-16 · 22 min read #llm-observability#prompt-engineering#helicone#langsmith#langchainAgent Evaluation Systems in 2026 — Inspect AI vs Promptfoo vs Phoenix vs LangSmith vs OpenAI Evals (You're Measuring the Agent, Not the Model) ♪ Listenable
LLM evals measure the model. Agent evals measure whether the model plus the harness plus the tools actually carry a task to completion. They are different problems. This is a map of the 2026 landscape — Inspect AI from U
2026-05-14 · 20 min read #agent-evaluation#inspect-ai#promptfoo#phoenix#langsmithLLM Evaluation & Observability: Eval Harness, LLM-as-Judge, Tracing, Regression Prevention (2025)
Claiming a new model "seems better" is not science. The 1 failure mode of 2025 LLM products is "not measuring". Eval harness design, LLM-as-judge pitfalls and calibration, 3-layer observability (Trace/Span/Metric), produ
2026-04-15 · 11 min read #llm-eval#observability#llm-judge#distributed-tracing#langsmithComparing LLM Production Monitoring Platforms: A Practical Operations Guide for LangSmith, LangFuse, and Arize Phoenix ♪ Listenable
A comprehensive comparison guide of three LLM production monitoring platforms (LangSmith, LangFuse, Arize Phoenix). Covers trace collection, prompt version management, evaluation pipelines, cost monitoring, quality dashb
2026-03-09 · 23 min read #ai-platform#llm-monitoring#langsmith#langfuse#arizeChatbot Performance Monitoring and Conversation Quality Analysis: From Metric Design to A/B Test Automation ♪ Listenable
A practical guide to chatbot performance monitoring and conversation quality analysis. Covers core metric design, tracing with LangSmith/Langfuse, automated quality evaluation pipelines, and A/B testing framework with co
2026-03-08 · 30 min read #chatbot#monitoring#analytics#ab-testing#langsmith