LLM Security GitHub Repositories

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

Open-source AI penetration testing tool to find and fix your app’s vulnerabilities.

Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

the LLM vulnerability scanner

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

🐢 Open-Source Evaluation & Testing library for LLM Agents

A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

Sandbox any AI agent in seconds - zero setup, zero latency.

A secure low code deception runtime framework, leveraging AI for System Virtualization.

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.

ADR secures enterprise AI agents through observability, security benchmarking, and threat detection. Deployed at Uber.

OWASP Top 10 for Large Language Model Apps (Part of the GenAI Security Project)

A security scanner for your LLM agentic workflows

AI-first security scanner. NEW in v2026.7: Claude Code compromise detection — vet .claude/ hooks, permissions & skills before you clone — plus an always-on AI attack-signature scanner and native Rust & PHP rules. Also: medusa scan --git to vet any repo, medusa secrets scan for leaked API keys. 40,000+ patterns, zero setup.

An easy-to-use Python framework to generate adversarial jailbreak prompts.

CLI security scanner built for the agentic era. Detects CI/CD misconfigs, agent permission risks, MCP tool injection, hardcoded secrets, and DMCA-flagged AI dependencies.

Open-source AI agent firewall for MCP security and agent egress. Scans mediated HTTP, MCP, A2A, and WebSocket traffic for exfiltration, SSRF, and prompt injection, and emits mediator-signed action receipts: verifiable audit evidence from outside the agent.

Local security audit for AI API relays and LLM proxies: detects prompt injection, model substitution, tool-call rewriting, SSE anomalies, error leakage, and Web3 wallet risks.

Runtime security for AI apps and agents: prompt injection detection, tool-call authorization, sensitive-data redaction, bot protection, and rate limiting. Drop it into your JS/TS code.

Give each AI agent its own isolated machine with root, Docker, and systemd. Active defense detects and stops threats automatically.

Papers and resources related to the security and privacy of LLMs 🤖

Open-source adversary emulation for AI agents and MCP servers.

Open-source runtime AI agent security tool - monitors and controls AI agents, catching malicious tool use, prompt injection, and policy drift in real time, before the agent acts.

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

This repository provides a benchmark for prompt injection attacks and defenses in LLMs

A curated list of MLSecOps tools and resources for securing machine learning and AI systems - adversarial ML defense, LLM security, AI red teaming, model scanning, supply-chain protection, and MLOps pipeline security.

Privacy proxy for your OpenAI requests

Open detection-rule standard for AI agent security threats — like Sigma, but for AI agents. Executable rules across 10 categories; merged into Microsoft AGT, Cisco AI Defense, MISP, OWASP, FINOS & SigmaHQ. MIT-licensed.