TestMyPrompt Review

It's an open-source LLM vulnerability scanner that tests against over 210 adversarial attacks, including prompt injection, jailbreaks, and data extraction, delivered as a single Go binary.

While TestMyPrompt is a paid hosted service, Augustus offers a powerful, comprehensive open-source solution that you run yourself, requiring more setup but providing full control over your testing environment.

It's an open-source Python library designed for input and output filtering, offering prompt injection detection, PII redaction, and toxicity filtering that runs locally.

Unlike TestMyPrompt's hosted scanning service, LLM Guard is a library you integrate directly into your application, providing real-time, local protection and detection without sending data externally.

It's a CLI and library for evaluating and red-teaming LLM applications, offering automated evaluations, side-by-side model comparisons, and vulnerability scanning with CI/CD integration.

Promptfoo provides a broader framework for LLM testing and evaluation beyond just security scanning, allowing for more comprehensive red teaming and performance comparisons, whereas TestMyPrompt focuses specifically on prompt vulnerabilities.

It's a hosted LLM scanner that detects prompt injection, AI jailbreaks, PII leakage, and OWASP LLM Top 10 vulnerabilities, offering a clear playbook for detection.

FilterPrompt offers a similar hosted scanning service to TestMyPrompt, but provides a free full scan to get started and focuses on a broader range of OWASP LLM Top 10 vulnerabilities.

An automated prompt injection scanner for custom LLM applications that supports both white-box (with system prompts) and black-box (HTTP endpoint) testing modes.

Similar to TestMyPrompt, promptmap2 is a dedicated prompt injection scanner, but it's open-source and provides flexible testing modes (white-box and black-box) for custom LLM applications, offering more control for self-hosted environments.