A plugin integrated with a GenAI model may contain vulnerabilities such as poor input validation or improper access control. An adversary may exploit these flaws by crafting a prompt that causes the model to pass a malicious payload to the plugin, potentially leading to system compromise, data exfiltration or privilege escalation.
AI/ML / Gen AI / Threats / DEV
Insecure Plugin
CCC.GenAI.TH07
Related Capabilities
| ID | Title | Description |
|---|---|---|
| CCC.GenAI.CP25 | Plugin Integrations | Ability for the model to use tools to complete a model interaction. For example web search, python code execution or external maths engine. |
Related Controls
| ID | Title | Description |
|---|---|---|
| CCC.GenAI.CN06 | Least Privilege for Plugins | Restricts the permissions of any external tools the GenAI system can call to limit the potential damage if an agent is coerced to perform unintended actions or vulnerabilities in the tools are exploited. |
External Mappings
| Framework | ID | Remarks |
|---|---|---|
| SAIF | IIC | Insecure Integrated Component |
| OWASP-LLM-TOP10 | LLLM07 | Insecure Plugin Design |
| MITRE-ATLAS | AML.T0053 | LLM Plugin Compromise |