Does Phoenix Offer Red Teaming or Vulnerability Scanning?

Hey everyone! Does phoenix provide red teaming or vulnerability scanner functionality out of the box or is that something we need to build on top?

· Sorted by Oldest

Hey everyone! Does phoenix provide red teaming or vulnerability scanner functionality out of the box or is that something we need to build on top?

· Sorted by Oldest

Mikyo
·
Hey Christen J. - Phoenix itself doesn't natively perform red teaming at inference time but Arize does partner with Guardrails for inference time protections (which can be traced and captured in Phoenix or Arize). It's not a project I'm super familiar with TBH but happy to intro you to the people running it. https://docs.arize.com/arize/llm-monitoring-and-guardrails/guardrails
Christen J.
·
Humm interesting I wonder if I could integrate https://github.com/leondz/garak with Phoenix so I can trace everything during red teaming process

Mikyo
·
Hey Christen J. - Phoenix itself doesn't natively perform red teaming at inference time but Arize does partner with Guardrails for inference time protections (which can be traced and captured in Phoenix or Arize). It's not a project I'm super familiar with TBH but happy to intro you to the people running it. https://docs.arize.com/arize/llm-monitoring-and-guardrails/guardrails
Christen J.
·
Humm interesting I wonder if I could integrate https://github.com/leondz/garak with Phoenix so I can trace everything during red teaming process