Skip to main content
usedby
Search tools, companies

How Pylon uses Braintrust

~10 → 0 mins

Engineering time to debug an issue

As published on braintrust.dev. Captured by usedby on Oct 6, 2026.

What happened

Pylon, an agentic B2B support platform, uses Braintrust to test every AI prompt through playgrounds with curated datasets, enforced as a CI requirement. It also uses Braintrust for production observability, building eval datasets from production edge cases, LLM-as-a-judge scoring, and automated root cause analysis of on-call bugs via the Braintrust MCP and CLI.

Summary written by usedby from the source page, in English. The figures are those of Braintrust and Pylon, not ours.

  • 100%“100% enforced, playgrounds by default”

Previously an on-call engineer had to spend ten minutes actively digging into an issue. Now they run this command, set it and forget it, go grab a coffee, and come back to the root cause totally figured out: a timeline, turn by turn, of what was happening and what went wrong.

From the page. braintrust.dev, captured Oct 6, 2026

This discipline is what Braintrust was built for, which is not just logging and observability, but improving quality by bringing it into our everyday development process.

Fred Zhao, Software Engineer, AI Team, Pylon. Source, captured Oct 6, 2026

What the story claims, and what we checked

We compared the story with its live page on Oct 6, 2026.

  • The figure: ~10 → 0 minsCheckedThe figure is on the page; its label is our wording.
  • The passage quoted aboveCheckedCopied word for word from the page, near the name of Pylon.
  • The numbers in our summaryCheckedEach one is printed on the page.
  • Pylon uses BraintrustCheckedConfirmed line. Latest check across sources: Oct 6, 2026.
  • The result itselfNot checkedWe quote it; we did not measure it.

Same company, same tool or same industry.

80%+Score threshold for internal releaseBox uses Braintrust. Another customer of Braintrust>0.9Eval system macro F1 scoreNavan uses Braintrust. Another customer of Braintrust30-min“Continuous signal, 30-min fix cycles”Rex uses Braintrust. Another customer of Braintrust