CVE-2026-66036

Executive Summary

CVE-2026-66036 is a high-severity heap out-of-bounds write vulnerability in FFmpeg's vf_hqdn3d denoising filter, affecting all versions through 8.1.2. The flaw enables memory corruption when processing crafted video files where frame resolution increases between frames while filtergraph reinitialization is disabled (-reinit_filter 0). Successful exploitation can lead to arbitrary code execution in the context of the FFmpeg process, making it a significant supply-chain risk for any system or application that transcribes, transcodes, or processes untrusted video content. With a CVSS score of 8.8, this vulnerability demands immediate attention from security teams, particularly those operating media processing pipelines, video streaming platforms, or any service that ingests user-uploaded media.

Technical Analysis

The vulnerability resides in the high-quality 3D denoising filter (vf_hqdn3d), specifically in the interaction between the config_input() function and the denoise_spatial() function. When FFmpeg initializes the filtergraph, config_input() allocates per-plane line-history buffers based on the initial frame dimensions received by the filter. The critical flaw is that these buffers are allocated once at filter initialization and are not resized if subsequent frames arrive with larger dimensions—the code assumes frame resolution remains constant throughout the stream.

The vulnerable code path is triggered only when filtergraph reinitialization is disabled via the -reinit_filter 0 option. Normally, FFmpeg's filtergraph infrastructure can detect resolution changes and reinitialize the filter chain; however, when this option is set, FFmpeg intentionally skips the reinitialization check, allowing frames of varying resolutions to flow through the already-configured filter graph.

The attack sequence is as follows:

1. Initialization: config_input() reads the properties of the first frame and allocates line-history buffers using av_mallocz_array() sized to height * linesize for each plane (Y, U, V), where height and linesize are derived from the initial frame's dimensions (e.g., 1920x1080).

2. Conditional allocation: The buffer size calculation for the temporal/horizontal filters is based on hsize = (width + 16) / 16 and vsize = (height + 16) / 16, where width and height are the initial frame's dimensions. These values are stored in the filter context (hqdn3d_context) and are not updated when a larger frame arrives.

3. Trigger: A subsequent frame arrives with larger dimensions (e.g., 3840x2160). The denoise_spatial() function iterates over the new frame's plane data using the new frame dimensions but writes into the old undersized buffers. The write operations—line = in + (x << 2) and subsequent memory accesses with offsets up to width * 4—exceed the allocated buffer boundaries.

4. Out-of-bounds write: The sequential write pattern corrupts adjacent heap metadata, adjacent filter context structures, or other heap allocations. In the context of FFmpeg's filter SDK, this can overwrite function pointers, size fields in neighboring AVBufferRef structures, or frame data buffers.

The vulnerability is classified as CWE-122 (Heap-based Buffer Overflow). The key technical enabler is the lack of a size revalidation between config_input() (allocation) and denoise_spatial() (write). This is a classic TOCTOU-style mismatch where allocation time uses one dimension set and write time uses another, and FFmpeg's frame-sync infrastructure does not enforce dimension consistency when -reinit_filter 0 is active.

The patch (commit 5d7112c) introduces a dimensions check in config_input() that either reallocates buffers dynamically or rejects frames with inconsistent dimensions, depending on the filtergraph configuration state. Because the validation was added post-8.1.2, any FFmpeg build before this commit contains the vulnerable code.

Exploitation Scenarios

Scenario 1: Malicious Video Upload to a Transcoding Web Service

1. Reconnaissance: An attacker identifies a web application that accepts video uploads and processes them through FFmpeg (e.g., video hosting platforms, social media with video features, or e-learning systems). The target uses the command line: ffmpeg -reinit_filter 0 -i uploaded.mp4 -vf hqdn3d=4:3:6:4 output.mp4.

2. Weaponization: The attacker crafts an MP4 file with an H.264 or H.265 elementary stream that contains two distinct video samples: the first sample has a resolution of 640x480, and the second sample has a resolution of 1280x720. The samples must be in separate access units, concatenated in a single track. Tools like GPAC's MP4Box can author such files, or the attacker can use FFmpeg's own concat demuxer with multi-resolution inputs.

3. Trigger: The payload is uploaded, and the target FFmpeg command is invoked. The filtergraph initializes with the 640x480 frame, allocating buffers of (640 + 16) / 16 * (480 + 16) / 16 blocks. The second frame (1280x720) passes the demuxer and arrives at vf_hqdn3d filter where denoise_spatial() performs 4x-horizontal writes based on the 1280-pixel width, overflowing the ~41KB buffer by roughly 100KB.

4. Payload/Execution: The overflow writes controlled data into the heap adjacent to the line-history buffer. With careful heap grooming (repeated allocations/deallocations of AVFrame structures), the attacker can overwrite the filters pointer in an adjacent AVFilterLink structure or corrupt the AVBufferRef freed-list header to achieve arbitrary write primitives. Subsequent ROP chain execution grants code execution in the FFmpeg process context.

5. Post-exploitation: The attacker gains access to the server's filesystem, potentially reading application secrets, database credentials, or pivoting to internal networks. If FFmpeg runs with elevated privileges (e.g., in a container with root), the impact escalates to full node compromise.

Scenario 2: Malicious Media in E-learning/LMS Platforms

1. Reconnaissance: The attacker fingerprints the target LMS by observing that course materials are transcoded to multiple resolutions. They hypothesize that the platform uses a standard FFmpeg CLI with filtergraph options like -vf hqdn3d for noise reduction of instructional videos.

2. Weaponization: The attacker uses a Python script with PyAV or moviepy to programmatically generate a valid MP4 file where the encoder resets the SPS/PPS between GOPs with different resolutions. The file passes strict container validation because each sample's avcC decoder-configuration-record is identical; only the in-band SPS changes.

3. Attack chain: The file is uploaded and processed. The FFmpeg process, compiled without ASLR (common in some container images), has predictable heap layout. The attacker pre-grooms the heap with a 4MB file containing a fake AVFilter struct at a known offset. The overflow writes into this area, setting the next pointer to a controlled address, achieving a write-what-where primitive.

4. Impact: The attacker achieves code execution during the transcoding job, exfiltrates student records (PII), and installs a persistent webshell within the application container. The attacker then moves laterally by abusing the application's service account that has write access to the shared storage volume.

Impact Assessment

Business Impact: This vulnerability represents a critical risk to any organization that processes untrusted video content through FFmpeg. The CVSS score of 8.8 reflects the combination of network attack vector, low attack complexity, and high impact on confidentiality, integrity, and availability (C:H/I:H/A:H). The user-interaction requirement (U:R) is minimal—it only requires a user to upload or process a file, which is the normal operational path for media platforms.

Affected Systems:

• FFmpeg binaries through version 8.1.2 (all distributions, static builds, and custom compilations)
• Applications using FFmpeg as a shared library (libavfilter, libavcodec) via FFI bindings (PyAV, ruby-ffmpeg, node-ffmpeg)
• Video streaming platforms, CCTV/NVR systems that rebroadcast video, and OTT platforms using FFmpeg's -reinit_filter 0 flag for performance optimization
• CI/CD pipelines that run FFmpeg-based tests or media validation

Blast Radius: The vulnerability is embedded in a library used globally. A single exploited instance can lead to: (1) lateral movement in cloud environments where FFmpeg runs serverless functions or k8s jobs; (2) data exfiltration from media repositories; (3) certificate/key theft from SSRF-mediated access if FFmpeg runs as root; (4) denial of service via heap corruption crashing the entire media processing pipeline.

Technical Impact: The heap OOB write enables both code execution and denial of service. The write primitive is constrained—it writes sequential 4-byte values (av_store semantics) but the increment pattern allows byte-granular manipulation of adjacent structures. The constrained write makes exploitation slightly harder than a classic arbitrary-write, but attackers with FFmpeg exploit experience will find this trivially chainable due to the predictable heap layout in most deployment environments.

Detection Guidance

Sigma Rule

title: FFmpeg Heap OOB Write Attempt - vf_hqdn3d Resolution Mismatch
id: 7f3b6f2a-9c41-4d2e-8f5a-1d3b7e5f9c10
status: experimental
description: Detects FFmpeg processes launched with -reinit_filter 0 and hqdn3d filter that crash or log invalid dimensions errors, indicating potential CVE-2026-66036 exploitation.
references:
    - https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2026-66036
tags:
    - attack.t1068
    - cve.2026-66036
logsource:
    category: process_creation
    product: linux
detection:
    selection_process:
        Image|endswith:
            - '/ffmpeg'
            - '/avconv'
        CommandLine|contains|all:
            - '-reinit_filter 0'
            - 'hqdn3d'
    selection_crash:
        EventID: 1001
        LogName: 'Application'
        Message|contains:
            - 'ffmpeg'
            - 'segfault'
            - 'heap-buffer-overflow'
            - 'AddressSanitizer: heap-buffer-overflow'
    condition: selection_process and selection_crash
falsepositives:
    - Legitimate FFmpeg usage with reinit_filter disabled and hqdn3d on constant-resolution video
    - ASan run of FFmpeg by developers
level: high

YARA Rule

rule CVE_2026_66036_FFmpeg_hqdn3d_OOB {
    meta:
        author = "ValtersIT Threat Intel"
        description = "Detects suspicious FFmpeg command lines combining reinit_filter 0 with hqdn3d filter, potential CVE-2026-66036 exploitation"
        date = "2026-01-01"
        severity = "high"
        reference = "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2026-66036"

    strings:
        $reinit = "-reinit_filter" ascii wide nocase
        $reinit_zero = "-reinit_filter 0" ascii wide nocase
        $hqdn3d = "hqdn3d" ascii wide nocase
        $vf = "-vf" ascii wide nocase
        $filter_complex = "-filter_complex" ascii wide nocase
        $asan_oom = "AddressSanitizer" ascii wide nocase

    condition:
        (uint16(0) == 0x5A4D or uint32(0) == 0x464C457F) and
        (
            (
                $reinit_zero and $hqdn3d
                and ($vf or $filter_complex)
                and filesize < 10MB
            )
            or
            (
                $asan_oom and $hqdn3d and filesize < 1MB
            )
        )
}

Suricata IDS Rule

alert http any any -> any any (msg:"ET EXPLOIT FFmpeg CVE-2026-66036 hqdn3d OOB Write - Potential Malicious Video Download"; flow:established,to_client; content:"video/mp4"; http.content_type; content:!"Content-Encoding|3a| gzip"; filemagic:"ISO Media, MP4 v2"; within:1200; pcre:"/avc1|hvc1|hev1/"; classtype:attempted-admin; sid:2099036; rev:1; metadata:created_at 2026_01_01, updated_at 2026_01_01, cve CVE-2026-66036;)

alert http any any -> any any (msg:"ET EXPLOIT FFmpeg CVE-2026-66036 Potential Exploit Payload - reinit_filter in Referer"; flow:established,to_server; content:"reinit_filter"; http.referer; nocase; classtype:attempted-admin; sid:2099037; rev:1; metadata:created_at 2026_01_01, updated_at 2026_01_01, cve CVE-2026-66036;)

Elastic Detection

// Detect FFmpeg crashes or ASan errors associated with the vulnerable filter
process where event.type in ("start", "info") and process.name in ("ffmpeg", "avconv")
  and process.command_line contains "-reinit_filter 0"
  and process.command_line contains "hqdn3d"
  and not user.name in ("root", "ffmpeg-user") // adjust legitimate users

// Additionally, monitor for crashes shortly after such a process starts
| eval crash_window = 60  // minutes
| filter
  exists process.pid and
  (
    // Look for segfault signals in kernel audit or app logs correlated by process
    // Example: process.exit_code == 139 OR event.action == "segfault"
    process.exit_code == 139 or process.exit_code == 11
  )

Splunk SPL Query

index=main sourcetype=WinEventLog:Application OR sourcetype=linux_audit
| where (match(message, "(?i)ffmpeg") AND match(message, "(?i)heap-buffer-overflow|segfault|AddressSanitizer"))
   OR (match(Process_Name, "(?i)ffmpeg") AND match(CommandLine, "(?i)-reinit_filter 0") AND match(CommandLine, "(?i)hqdn3d"))
| eval timestamp = strftime(_time, "%Y-%m-%d %H:%M:%S")
| table timestamp host Process_Name CommandLine User_Name message EventID
| sort - _time
| where timestamp > relative_time(now(), "-24h@h")

Wazuh OSSEC Rule

<group name="ffmpeg,cve-2026-66036,">
    <rule id="608725" level="12">
        <if_sid>529</if_sid> <!-- auditd restart / process execution -->
        <match>reinit_filter</match>
        <match>hqdn3d</match>
        <description>FFmpeg launched with vulnerable options (CVE-2026-66036)</description>
        <group>attack.t1068,</group>
        <mitre>
            <id>T1068</id>
        </mitre>
    </rule>

    <rule id="608726" level="15">
        <if_sid>540</if_sid> <!-- segfault / core dump -->
        <match>ffmpeg</match>
        <regex>segfault|SIGSEGV|heap-buffer-overflow</regex>
        <description>Possible heap buffer overflow in FFmpeg (CVE-2026-66036 exploitation)</description>
        <group>attack.t1068,</group>
        <mitre>
            <id>T1068</id>
        </mitre>
    </rule>
</group>

Mitigation & Patch Guidance

Given the patch status, the following guidance is critical:

1) Immediate Mitigations (Stopping the Bleeding)

• Patch if possible: Upgrade to FFmpeg 8.1.3+ (or the latest release that includes commit 5d7112c). Verify the build you use includes this commit before deployment.
• Disable filtergraph reinitialization in production: If you cannot patch immediately, audit all FFmpeg invocations. If -reinit_filter 0 is used anywhere, remove it. This forces FFmpeg's default reinitialization check, which will catch dimension changes and reinitialize the filtergraph, preventing the OOB write.
• Input validation layer: Deploy a pre-processing pipeline that validates video dimensions before FFmpeg. Use ffprobe (which is not vulnerable) to check that all video streams have constant resolution. Reject any file where width or height changes between packets. A simple Python script using PyAV or ffprobe can automate this check.
• Run FFmpeg in a sandbox: Execute FFmpeg in a minimal container (distroless image), with --cap-drop=ALL, a read-only root filesystem, and no network egress. This limits the blast radius of code execution.

2) Patch/Upgrade Steps

• Source build: git clone https://git.ffmpeg.org/ffmpeg.git and check out a commit newer than 5d7112c. Build with ./configure --disable-everything --enable-filter=hqdn3d and any other required components.
• Distro packages: If you use a distribution package (e.g., apt/yum/dnf), check for updated ffmpeg packages that incorporate commit 5d7112c. This may require a backport if the distro ships an older FFmpeg version.
• Static/container images: If you pull FFmpeg from Docker Hub or similar, ensure the image tag explicitly includes a version >= 8.1.3 or is dated after the commit date. Do not rely on latest tags.
• Verify: After updating, run the regression test: ffmpeg -reinit_filter 0 -i /path/to/multi_resolution_test.mp4 -vf hqdn3d out.mp4 and confirm no crash or ASan error.

3) Long-Term Hardening

• Adopt the principle of least functionality: Do not enable all FFmpeg filters. Compile with --disable-everything and explicitly enable only the required filters (e.g., --enable-filter=hqdn3d,scale,yadif). This reduces the attack surface of the FFmpeg binary.
• Implement a video sanitization microservice: Treat FFmpeg as an untrusted component. Isolate it on dedicated worker nodes with no access to the application database or secrets. Use network policies to restrict its egress to only the object storage and message queue.
• Monitor for CVE-2026-66036 exploitation attempts: Despite the EPSS score of 0.00288 (low percentile), threat actors are known to weaponize FFmpeg CVEs in bulk due to the internet-wide exposure of FFmpeg. Follow the CISA KEV (currently marked "NO") and consider adding hunting rules to your SIEM.
• Patch confidence: The patch confidence is rated 2/4 (CLAIMED) because the patch is only confirmed by a single source (NVD). Independently verify the patch by inspecting the FFmpeg source: look for a reallocation check in vf_hqdn3d.c in config_input that compares the frame dimensions to the stored hsize and vsize, and either reallocates the buffers (av_fast_mallocz) or returns AVERROR(EINVAL) on mismatch.

References

1. NVD Entry: https://nvd.nist.gov/vuln/detail/CVE-2026-66036 — Official CVE record with CVSS vector and description.
2. FFmpeg Security Advisories: https://ffmpeg.org/security.html — Vendor's security page; typically lists the commit hash and affected versions.
3. FFmpeg Git Commit 5d7112c: https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/5d7112c — Direct link to the patching commit (reference via gitweb).
4. Exploit-DB / Packet Storm (if available): Search for CVE-2026-66036 for any public PoC or exploit demonstrations. Note: as of writing, no public exploit or PoC is known; no threat actor campaigns have been identified. The vulnerability is weaponizable but currently not in the CISA KEV catalog.

Threat Actor Activity: No active exploitation in the wild has been reported. The low EPSS score (0.00288) suggests that as of the report date, automated scanning is unlikely, but this CVE could see a spike in exploitation following the release of a public PoC or when large-scale vulnerability scanners incorporate Nuclei or Metasploit modules. Continuous monitoring of exploit-hosting platforms is recommended.