CVE-2024-12720

Executive Summary

CVE-2024-12720 is a Regular Expression Denial of Service (ReDoS) vulnerability in the Huggingface Transformers library, specifically within the tokenization_nougat_fast.py file. The flaw resides in the post_process_single() function, where a regular expression with exponential time complexity can be triggered by specially crafted input. This leads to excessive CPU consumption and potential application downtime, constituting a Denial of Service (DoS) condition. The vulnerability affects version v4.46.3, which is the latest at the time of analysis. No patch is currently available, and no public proof-of-concept (PoC) or exploit code has been released. The EPSS score is 0.0%, indicating low exploitation likelihood in the wild, but the risk remains for environments processing untrusted input through the affected tokenizer.

Supply Chain & Package Security

This vulnerability affects the PyPI package transformers. Organizations using this library in production or development environments should assess their exposure. Since the vulnerable function is part of the tokenization pipeline, any application that processes user-supplied text through the Nougat tokenizer is at risk. Supply chain security measures should include monitoring for updates from Huggingface and applying patches as soon as they become available.

CVSS Breakdown & Threat Level

• CVSS Score: 5.3 (Medium)
• Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L
• Attack Vector: Network (AV:N) – The vulnerability can be exploited remotely.
• Attack Complexity: Low (AC:L) – No special conditions required.
• Privileges Required: None (PR:N) – No authentication needed.
• User Interaction: None (UI:N) – No user action required.
• Scope: Unchanged (S:U) – The impact is confined to the vulnerable component.
• Confidentiality Impact: None (C:N) – No data exposure.
• Integrity Impact: None (I:N) – No data modification.
• Availability Impact: Low (A:L) – Degraded performance or downtime.

Threat Level: Medium. While the CVSS score is moderate, the potential for CPU exhaustion and service disruption in AI/ML pipelines makes this a significant concern for high-availability environments.

Deep Dive & Mechanics

The vulnerability is a classic ReDoS (Regular Expression Denial of Service) caused by a regex pattern with exponential backtracking. In tokenization_nougat_fast.py, the post_process_single() function uses a regex to clean or transform tokenized output. The specific regex pattern contains nested quantifiers or alternations that, when matched against a carefully crafted input string, cause the regex engine to explore an exponential number of paths.

For example, a pattern like (a+)+b can cause catastrophic backtracking when the input is "aaaaac" (where 'c' fails to match 'b'). The engine tries all combinations of a+ groups before failing. In this case, the vulnerable regex likely has similar characteristics, leading to high CPU usage and potential denial of service.

The attack vector is simple: an attacker sends a specially crafted string to an application using the Nougat tokenizer. The string triggers the exponential backtracking, consuming CPU resources and potentially causing the application to become unresponsive.

Vulnerable vs. Patched Code

Since no patch is available, we can only describe the vulnerable code pattern. The vulnerable function post_process_single() contains a regex that is susceptible to ReDoS. A patched version would replace the vulnerable regex with a safer alternative, such as using atomic groups, possessive quantifiers, or a non-regex approach.

Vulnerable Code (Conceptual):

import re

def post_process_single(text):
    # Vulnerable regex with exponential backtracking
    pattern = r'(some_pattern)+'  # Example pattern
    return re.sub(pattern, replacement, text)

Patched Code (Conceptual):

import re

def post_process_single(text):
    # Safer regex using atomic groups or possessive quantifiers
    pattern = r'(?>some_pattern)+'  # Atomic group prevents backtracking
    return re.sub(pattern, replacement, text)

Exploitation & Lab Setup

Red Team

Objective: Trigger a Denial of Service by sending a crafted input to the Nougat tokenizer.

Setup:

1. Install the vulnerable version: pip install transformers==4.46.3
2. Create a Python script that imports the Nougat tokenizer and sends a malicious input.

Exploit Script:

from transformers import NougatTokenizerFast

tokenizer = NougatTokenizerFast.from_pretrained("facebook/nougat-base")
# Crafted input to trigger ReDoS (example pattern)
malicious_input = "a" * 100 + "!"  # Adjust based on actual regex
try:
    tokenizer(malicious_input)
except Exception as e:
    print(f"Error: {e}")

Note: The exact malicious input depends on the vulnerable regex pattern. Without public PoC, this is a conceptual example.

Blue Team

Detection:

• Monitor CPU usage spikes in processes using the Transformers library.
• Log and alert on unusually long tokenization times.
• Use the detection rules provided below.

Mitigation:

• Implement input validation to reject suspicious patterns (e.g., long strings of repeated characters).
• Apply rate limiting to tokenization endpoints.
• Use a Web Application Firewall (WAF) to filter potential ReDoS payloads.

Indicators of Compromise (IOCs)

• File: tokenization_nougat_fast.py (vulnerable file)
• Process: Python processes with high CPU usage during tokenization
• Network: No specific network IOCs; attack is application-layer
• Logs: Error logs showing re.error or timeout exceptions in tokenization

Detection Engineering

YARA Rule

rule CVE_2024_12720_ReDoS_Transformers {
    meta:
        description = "Detects vulnerable regex pattern in tokenization_nougat_fast.py"
        author = "Valters IT"
        date = "2025-04-10"
        cve = "CVE-2024-12720"
    strings:
        $vuln_func = "post_process_single"
        $vuln_regex = /\([^)]+\)\+/  // Simplified pattern for detection
    condition:
        $vuln_func and $vuln_regex
}

Sigma Rule

title: CVE-2024-12720 ReDoS in Huggingface Transformers
id: 12345678-1234-1234-1234-123456789012
status: experimental
description: Detects potential ReDoS exploitation attempts against Nougat tokenizer
author: Valters IT
logsource:
    category: application
    product: python
detection:
    selection:
        EventID: 100  # Example event ID for high CPU
        ProcessName: 'python*'
        CommandLine: '*transformers*'
    condition: selection
falsepositives:
    - Legitimate tokenization of large inputs
level: medium

Suricata Rule

alert http any any -> any any (msg:"CVE-2024-12720 ReDoS Attempt"; flow:to_server; content:"POST"; http_method; content:"/tokenize"; http_uri; pcre:"/(a+)+b/"; sid:1000001; rev:1;)

Remediation & Mitigation

1. Immediate: Disable or restrict access to the Nougat tokenizer if not required.
2. Short-term: Implement input validation to reject strings with excessive repetition or patterns that could trigger ReDoS.
3. Long-term: Monitor Huggingface for a patch and update to a fixed version once available. Consider using a regex timeout wrapper (e.g., regex library with timeout) to limit execution time.
4. Workaround: Replace the vulnerable regex with a safer implementation, such as using re.match with atomic groups or switching to a non-regex parser.

Valters IT – Securing AI/ML pipelines against supply chain and application-layer threats.