Nginx Config Generator from Service Registry

Problem Statement

In modern distributed architectures and microservice deployments, managing reverse proxy configurations by hand becomes an operational bottleneck. Hand-crafting Nginx server and upstream blocks for dozens of services leads to syntax errors, inconsistent TLS settings, dynamic IP drift, and config duplication across environments.

This script automates Nginx reverse proxy management by converting a centralized, human-readable service registry (yaml) into valid, production-grade Nginx configuration files. It acts as an infrastructure-as-code bridge, ensuring uniform security headers, load balancing, SSL/TLS defaults, and proxy directives without requiring manual editing of raw Nginx files.

How It Works

The script operates as a declarative configuration engine with the following data flow:

1. Ingestion & Validation: The script loads service definitions from a YAML registry file and validates mandatory fields (service names, domains, upstreams, and ports).
2. Template Rendering: Using Jinja2, the script processes an embedded default template (or an optional external override template) to render upstream pool definitions and server blocks.
3. Safe File Writing: Configurations are generated atomically into target output destinations (either individual .conf files per service inside a target directory or a single aggregated file).
4. Syntax Verification: The script optionally executes nginx -t in a subprocess to verify the generated config syntax before applying updates, preventing bad deployments from taking down active traffic.

Full Script

#!/usr/bin/env python3
"""
Nginx Config Generator from Service Registry
Reads service definitions from a YAML registry and renders production-ready
Nginx reverse proxy configurations using Jinja2 templates.
"""

import argparse
import os
import sys
import subprocess
import logging
from pathlib import Path
from typing import Any, Dict, List, Optional
import yaml
from jinja2 import Environment, BaseLoader, TemplateSyntaxError, UndefinedError

# Configure Logging
logging.basicConfig(
    level=os.environ.get("LOGLEVEL", "INFO").upper(),
    format="%(asctime)s [%(levelname)s] %(message)s",
)
logger = logging.getLogger("nginx_gen")

# Environment Variable Configuration Defaults
DEFAULT_OUT_DIR = os.environ.get("NGINX_GEN_OUT_DIR", "/etc/nginx/conf.d")
DEFAULT_TEMPLATE = os.environ.get("NGINX_GEN_TEMPLATE", "")
VALIDATE_NGINX_EXEC = os.environ.get("NGINX_GEN_VALIDATE", "false").lower() in ("true", "1", "yes")

# Embedded Production-Ready Default Jinja2 Template
DEFAULT_NGINX_TEMPLATE = """# Auto-generated by nginx_gen.py. DO NOT EDIT MANUALLY.
# Service: {{ service.name }}

{% if service.upstreams %}
upstream {{ service.name }}_backend {
    {% for server in service.upstreams %}
    server {{ server.host }}:{{ server.port }}{% if server.weight %} weight={{ server.weight }}{% endif %}{% if server.max_fails %} max_fails={{ server.max_fails }}{% endif %}{% if server.fail_timeout %} fail_timeout={{ server.fail_timeout }}{% endif %};
    {% endfor %}
    keepalive 32;
}
{% endif %}

server {
    listen {{ service.port | default(80) }};
    server_name {{ service.domain }}{% if service.aliases %} {{ service.aliases | join(' ') }}{% endif %};

    {% if service.ssl %}
    listen {{ service.ssl_port | default(443) }} ssl http2;
    ssl_certificate {{ service.ssl.certificate }};
    ssl_certificate_key {{ service.ssl.key }};
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    {% endif %}

    client_max_body_size {{ service.client_max_body_size | default('16m') }};

    {% for location in service.locations | default([{'path': '/', 'proxy_pass': 'http://' ~ service.name ~ '_backend'}]) %}
    location {{ location.path }} {
        {% if location.proxy_pass %}
        proxy_pass {{ location.proxy_pass }};
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_connect_timeout {{ location.connect_timeout | default('60s') }};
        proxy_read_timeout {{ location.read_timeout | default('60s') }};
        {% elif location.root %}
        root {{ location.root }};
        index {{ location.index | default('index.html') }};
        try_files $uri $uri/ /index.html;
        {% endif %}
    }
    {% endfor %}

    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
        root /usr/share/nginx/html;
    }
}
"""


def load_services_registry(filepath: Path) -> Dict[str, Any]:
    """Load and parse the YAML service registry file."""
    if not filepath.exists():
        raise FileNotFoundError(f"Registry file not found: {filepath}")

    try:
        with open(filepath, "r", encoding="utf-8") as f:
            data = yaml.safe_load(f)
            if not isinstance(data, dict) or "services" not in data:
                raise ValueError("Registry YAML must contain a top-level 'services' list key.")
            return data
    except yaml.YAMLError as exc:
        raise ValueError(f"Error parsing YAML file {filepath}: {exc}") from exc


def validate_service(service: Dict[str, Any], index: int) -> None:
    """Validate that a service entry contains required fields."""
    required_keys = ["name", "domain"]
    for key in required_keys:
        if key not in service:
            raise KeyError(f"Service entry #{index} is missing required field: '{key}'")

    if "upstreams" in service:
        if not isinstance(service["upstreams"], list) or len(service["upstreams"]) == 0:
            raise ValueError(f"Service '{service['name']}' has empty or non-list 'upstreams'.")
        for upstream in service["upstreams"]:
            if "host" not in upstream or "port" not in upstream:
                raise KeyError(f"Upstream in service '{service['name']}' missing host or port.")


def render_nginx_config(template_str: str, service: Dict[str, Any]) -> str:
    """Render Nginx configuration string for a single service definition."""
    try:
        env = Environment(loader=BaseLoader(), trim_blocks=True, lstrip_blocks=True)
        template = env.from_string(template_str)
        return template.render(service=service)
    except (TemplateSyntaxError, UndefinedError) as exc:
        raise RuntimeError(f"Template rendering failed for service '{service.get('name')}': {exc}") from exc


def write_config_file(output_path: Path, content: str, dry_run: bool = False) -> None:
    """Safely write generated configuration to disk."""
    if dry_run:
        logger.info("[DRY-RUN] Would write to %s:\n%s", output_path, content)
        return

    output_path.parent.mkdir(parents=True, exist_ok=True)
    
    # Write to a temporary file first to prevent partial writes
    tmp_path = output_path.with_suffix(".tmp")
    with open(tmp_path, "w", encoding="utf-8") as f:
        f.write(content)

    # Atomic rename replace
    tmp_path.replace(output_path)
    logger.info("Successfully generated configuration: %s", output_path)


def verify_nginx_syntax() -> bool:
    """Execute nginx -t to verify global syntax correctness."""
    logger.info("Running Nginx syntax check (nginx -t)...")
    try:
        result = subprocess.run(["nginx", "-t"], capture_output=True, text=True, check=True)
        logger.info("Nginx syntax validation passed:\n%s", result.stderr.strip())
        return True
    except subprocess.CalledProcessError as exc:
        logger.error("Nginx syntax validation failed!\n%s", exc.stderr.strip())
        return False
    except FileNotFoundError:
        logger.warning("Nginx binary not found in PATH. Skipping syntax validation.")
        return True


def parse_args() -> argparse.Namespace:
    """Parse command line arguments."""
    parser = argparse.ArgumentParser(
        description="Generate Nginx configuration files from a YAML service registry."
    )
    parser.add_argument(
        "-s", "--services",
        type=Path,
        required=True,
        help="Path to services YAML registry file.",
    )
    parser.add_argument(
        "-o", "--out",
        type=Path,
        default=Path(DEFAULT_OUT_DIR),
        help=f"Output directory or file path (default: {DEFAULT_OUT_DIR}).",
    )
    parser.add_argument(
        "-t", "--template",
        type=Path,
        default=Path(DEFAULT_TEMPLATE) if DEFAULT_TEMPLATE else None,
        help="Path to dynamic Jinja2 template file (optional).",
    )
    parser.add_argument(
        "--dry-run",
        action="store_true",
        help="Render configs to stdout without writing files to disk.",
    )
    parser.add_argument(
        "--validate",
        action="store_true",
        default=VALIDATE_NGINX_EXEC,
        help="Run 'nginx -t' after generating files to verify syntax.",
    )
    return parser.parse_args()


def main() -> None:
    """Main execution workflow."""
    args = parse_args()

    # Load custom Jinja2 template if provided, else fall back to embedded default
    template_content = DEFAULT_NGINX_TEMPLATE
    if args.template:
        if not args.template.exists():
            logger.error("Specified template file does not exist: %s", args.template)
            sys.exit(1)
        template_content = args.template.read_text(encoding="utf-8")

    # Read registry
    try:
        registry = load_services_registry(args.services)
    except (FileNotFoundError, ValueError) as err:
        logger.error("Failed to load services registry: %s", err)
        sys.exit(1)

    services: List[Dict[str, Any]] = registry.get("services", [])
    if not services:
        logger.warning("No services found in registry file.")
        sys.exit(0)

    # Process services
    errors_encountered = False
    for idx, service in enumerate(services, start=1):
        try:
            validate_service(service, idx)
            rendered_config = render_nginx_config(template_content, service)

            # Determine destination output path
            if args.out.suffix == ".conf":
                # User specified a single combined config destination
                out_file = args.out
            else:
                # User specified an output directory
                out_file = args.out / f"{service['name']}.conf"

            write_config_file(out_file, rendered_config, dry_run=args.dry_run)

        except (KeyError, ValueError, RuntimeError) as err:
            logger.error("Error processing service entry #%d: %s", idx, err)
            errors_encountered = True

    if errors_encountered:
        logger.error("Generation completed with errors. Check logs above.")
        sys.exit(1)

    # Optional Nginx syntax verification
    if args.validate and not args.dry_run:
        if not verify_nginx_syntax():
            sys.exit(1)

    logger.info("Nginx configuration generation finished successfully.")


if __name__ == "__main__":
    main()

Configuration

The script accepts CLI parameters and reads fallback values from environment variables:

Usage Examples

Example 1: Basic Generation to Output Directory

Create a sample services.yaml file:

services:
  - name: api-gateway
    domain: api.example.com
    port: 80
    upstreams:
      - host: 10.0.1.10
        port: 8080
        weight: 2
      - host: 10.0.1.11
        port: 8080
        weight: 1
    client_max_body_size: 32m

  - name: auth-service
    domain: auth.example.com
    upstreams:
      - host: 10.0.2.5
        port: 9000

Run the generator script:

python3 nginx_gen.py --services services.yaml --out /etc/nginx/conf.d/

Output:

2026-03-31 10:00:00 [INFO] Successfully generated configuration: /etc/nginx/conf.d/api-gateway.conf
2026-03-31 10:00:00 [INFO] Successfully generated configuration: /etc/nginx/conf.d/auth-service.conf
2026-03-31 10:00:00 [INFO] Nginx configuration generation finished successfully.

Example 2: Dry Run Testing with Validation

Inspect generated outputs without writing files, enabling validation flags:

python3 nginx_gen.py --services services.yaml --out /tmp/nginx/ --dry-run

Output:

2026-03-31 10:05:00 [INFO] [DRY-RUN] Would write to /tmp/nginx/api-gateway.conf:
# Auto-generated by nginx_gen.py. DO NOT EDIT MANUALLY.
# Service: api-gateway

upstream api-gateway_backend {
    server 10.0.1.10:8080 weight=2;
    server 10.0.1.11:8080 weight=1;
    keepalive 32;
}

server {
    listen 80;
    server_name api.example.com;
    client_max_body_size 32m;

    location / {
        proxy_pass http://api-gateway_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
    }

    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
        root /usr/share/nginx/html;
    }
}
2026-03-31 10:05:00 [INFO] Nginx configuration generation finished successfully.

Error Handling

• Missing / Malformed Registry: If the YAML file doesn't exist, is invalid YAML, or lacks a top-level services list, the script logs an explicit error and exits with code 1.
• Missing Fields: Individual service schema errors (e.g., missing domain or incomplete upstreams) are logged per service. Valid services complete while failing services trigger non-zero script completion.
• Atomic Writing: Config files write to temporary files (.tmp) prior to replacing live configs, ensuring Nginx never reads partially-written or corrupted configuration files.
• Syntax Checking: When --validate is passed, the script executes nginx -t. If validation fails, it outputs the exact Nginx error diagnostic block to stdout and exits with failure status before any system reload.

Extending Ideas

• Dynamic Service Discovery: Integrate direct API fetches from HashiCorp Consul, AWS Cloud Map, or Kubernetes Endpoints instead of static YAML files.
• Automated SSL/TLS (Let's Encrypt): Extend the template and validator logic to auto-detect missing cert paths and automatically trigger certbot requests.
• Automated Nginx Reloads: Add a --reload flag that triggers systemctl reload (systemctl reload nginx) or docker signals (docker exec nginx nginx -s reload) when changes occur.
• Rate Limiting & CORS Configuration: Add high-level keys to the registry schema (e.g., rate_limit: "10r/s", cors_origin: "*") and expose standard blocks inside the Jinja2 template.