The use case: how PagerDuty deploys Arize AI
PagerDuty uses Arize AX to trace each step of its customer-facing AI agents in PagerDuty Advance and to run LLM-as-a-judge evals on tool choice, parameters, and response relevance. It also ties Slack user feedback to the underlying traces and uses online monitors to alert on-call engineers when output quality drops.
In the customer’s words
The biggest benefit we’ve seen is that Arize AX has opened up the black box of what an agent is,Other AI tools used by PagerDuty
Other public relationships recorded on usedby.
Explore PagerDuty’s AI stackArize AI users in DevTools & Infrastructure
Companies in the same sector, with public evidence.
See all Arize AI customersGo straight to the source
Based on Arize AI’s published case study. We check the public evidence; the results are reported by the vendor.
arize.com- Last checked
- Nov 16, 2026
Open source capture What the story claims, and what we checked
What we compared with the page.
- The passage quoted aboveChecked
Copied word for word from the page, near the name of PagerDuty.
- PagerDuty uses Arize AIChecked
Confirmed line.
- The result itselfNot checked
We quote it; we did not measure it.
PagerDuty can use those evaluations with Arize monitors and its own incident response platform. If an output-quality metric crosses a threshold, the workflow can create a PagerDuty incident, notify the agent team’s on-call engineer, and create a Jira ticket so the issue has clear ownership.






