AppSecNews

AI Security

AI and LLM Security

Test and defend models, prompts, agents and the infrastructure around them.

45 tools profiled

How it differs Tests and guards models, LLM applications and agents against prompt injection, jailbreaks and data leakage. Scanning an AI app's own code is still SAST or SCA.

License
Subcategory
Deployment
Languages
Integrations
Maturity
Signals
Clear

18 tools match

Active Library Clear all

  • Adversarial Robustness Toolbox (ART)

    Linux Foundation AI & Data

    AI Security

    Python library implementing adversarial attacks and defenses against machine learning models, covering evasion, poisoning, extraction and inference.

    Open source
    Established

  • Arize AI

    Arize AI

    AI Security

    LLM and ML observability platform that captures application traces and runs evaluations over them, with an open source tracing and evaluation component.

    Open source and commercial
    Established

  • Arthur AI

    Arthur

    AI Security

    Model monitoring and guardrail platform that evaluates LLM inputs and outputs inline for injection, sensitive data and unsupported claims.

    Open source and commercial
    Growing

  • DeepTeam

    Confident AI

    AI Security

    Open source Python framework that generates adversarial prompts against an LLM application and scores the responses for vulnerabilities such as bias, PII leakage and excessive agency.

    Open source
    Growing

  • Future AGI

    Future AGI

    AI Security

    Evaluation and observability platform for LLM applications, combining offline scoring of model output with runtime checks on inputs and responses.

    Freemium
    Emerging

  • FuzzyAI

    CyberArk

    AI Security

    Open source fuzzer that applies a catalog of published jailbreak and prompt injection techniques against local or hosted language model endpoints.

    Open source
    Growing

  • Galileo AI

    Galileo

    AI Security

    Evaluation and observability platform for LLM and agent applications, with purpose built scoring models and an inline guardrail path.

    Commercial
    Growing

  • Garak

    NVIDIA

    AI Security

    Command line LLM vulnerability scanner that fires a library of attack probes at a model endpoint and scores the responses with matched detectors.

    Open source
    Growing

  • Giskard

    Giskard

    AI Security

    Python testing library that scans models and LLM applications for security and quality failures, then turns findings into a reusable test suite.

    Freemium
    Growing

  • Adversarial Robustness Toolbox (ART)

    Linux Foundation AI & Data

    Python library implementing adversarial attacks and defenses against machine learning models, covering evasion, poisoning, extraction and inference.

    Open source Established
    AI Security

  • Arize AI

    Arize AI

    LLM and ML observability platform that captures application traces and runs evaluations over them, with an open source tracing and evaluation component.

    Open source and commercial Established
    AI Security

  • Arthur AI

    Arthur

    Model monitoring and guardrail platform that evaluates LLM inputs and outputs inline for injection, sensitive data and unsupported claims.

    Open source and commercial Growing
    AI Security

  • DeepTeam

    Confident AI

    Open source Python framework that generates adversarial prompts against an LLM application and scores the responses for vulnerabilities such as bias, PII leakage and excessive agency.

    Open source Growing
    AI Security

  • Future AGI

    Future AGI

    Evaluation and observability platform for LLM applications, combining offline scoring of model output with runtime checks on inputs and responses.

    Freemium Emerging
    AI Security

  • FuzzyAI

    CyberArk

    Open source fuzzer that applies a catalog of published jailbreak and prompt injection techniques against local or hosted language model endpoints.

    Open source Growing
    AI Security

  • Galileo AI

    Galileo

    Evaluation and observability platform for LLM and agent applications, with purpose built scoring models and an inline guardrail path.

    Commercial Growing
    AI Security

  • Garak

    NVIDIA

    Command line LLM vulnerability scanner that fires a library of attack probes at a model endpoint and scores the responses with matched detectors.

    Open source Growing
    AI Security

  • Giskard

    Giskard

    Python testing library that scans models and LLM applications for security and quality failures, then turns findings into a reusable test suite.

    Freemium Growing
    AI Security

  • Guardrails AI

    Guardrails AI

    AI Security

    Python framework that wraps LLM calls in composable validators and decides what to do when a prompt or a response fails one.

    Open source and commercial
    Growing

  • LLM Guard

    Protect AI

    AI Security

    Python library of composable input and output scanners that sanitize prompts and validate model responses entirely within your own environment.

    Open source
    Growing

  • NVIDIA NeMo Guardrails

    NVIDIA

    AI Security

    Toolkit for defining rails on an LLM conversation using a dedicated modeling language, controlling input, output, topic, retrieval and tool execution.

    Open source
    Growing

  • OpenAI Guardrails

    OpenAI

    AI Security

    Open-source Python package that runs a configured pipeline of safety checks on prompts and model responses, tripping on policy violations.

    Open source
    Emerging

  • Promptfoo

    Promptfoo

    AI Security

    Config driven test and red team harness for LLM applications, running assertions and generated adversarial probes against prompts, models and agents.

    Open source and commercial
    Growing

  • Protecto

    Protecto

    AI Security

    Data privacy layer for AI pipelines that identifies sensitive fields in text and substitutes tokens so models never see the underlying values.

    Commercial
    Emerging

  • PyRIT

    Microsoft

    AI Security

    Python framework from Microsoft for automating adversarial probing of generative AI systems, with composable attack, transformation and scoring parts.

    Open source
    Growing

  • Rebuff

    Protect AI

    AI Security

    Open source prompt injection detector that layers heuristics, a classifier prompt, a vector store of known attacks and canary tokens.

    Open source
    Emerging

  • WhyLabs

    WhyLabs

    AI Security

    Observability platform for ML and LLM systems built on lightweight statistical profiles, with drift monitoring and text quality and safety metrics.

    Open source and commercial
    Established

  • Guardrails AI

    Guardrails AI

    Python framework that wraps LLM calls in composable validators and decides what to do when a prompt or a response fails one.

    Open source and commercial Growing
    AI Security

  • LLM Guard

    Protect AI

    Python library of composable input and output scanners that sanitize prompts and validate model responses entirely within your own environment.

    Open source Growing
    AI Security

  • NVIDIA NeMo Guardrails

    NVIDIA

    Toolkit for defining rails on an LLM conversation using a dedicated modeling language, controlling input, output, topic, retrieval and tool execution.

    Open source Growing
    AI Security

  • OpenAI Guardrails

    OpenAI

    Open-source Python package that runs a configured pipeline of safety checks on prompts and model responses, tripping on policy violations.

    Open source Emerging
    AI Security

  • Promptfoo

    Promptfoo

    Config driven test and red team harness for LLM applications, running assertions and generated adversarial probes against prompts, models and agents.

    Open source and commercial Growing
    AI Security

  • Protecto

    Protecto

    Data privacy layer for AI pipelines that identifies sensitive fields in text and substitutes tokens so models never see the underlying values.

    Commercial Emerging
    AI Security

  • PyRIT

    Microsoft

    Python framework from Microsoft for automating adversarial probing of generative AI systems, with composable attack, transformation and scoring parts.

    Open source Growing
    AI Security

  • Rebuff

    Protect AI

    Open source prompt injection detector that layers heuristics, a classifier prompt, a vector store of known attacks and canary tokens.

    Open source Emerging
    AI Security

  • WhyLabs

    WhyLabs

    Observability platform for ML and LLM systems built on lightweight statistical profiles, with drift monitoring and text quality and safety metrics.

    Open source and commercial Established
    AI Security
Tick up to 4 tools above.