Ansible Playbook Runner via ansible-runner

Ansible Playbook Runner via ansible-runner

Problem Statement

Running Ansible playbooks from CI/CD pipelines or automation platforms usually means shelling out to ansible-playbook and parsing its stdout, which is fragile. You lose structured events, exit codes get swallowed, and capturing per-task results requires ugly regex. When you need to run a playbook programmatically — say, from a Flask endpoint, a Celery worker, or a Python-based orchestrator — you want a Pythonic interface that returns structured results, streams events, and handles failures cleanly.

ansible-runner solves this by providing a stable Python API that wraps Ansible execution, captures events as JSON, and exposes artifacts on disk. This script is a thin, production-ready CLI wrapper around ansible-runner that accepts a playbook, inventory, and extra vars, runs them, streams task events in real time, and exits with a meaningful status code so it can be dropped into any pipeline.

How It Works

The script uses ansible_runner.run() with a private_data_dir (a temporary working directory) that contains the project tree. It copies the playbook and inventory into that directory, then invokes Ansible via the runner. Two callbacks are registered: event_handler for streaming live events to stdout, and status_handler for tracking play recap status. The runner returns a Runner object whose .status is "successful", "failed", "timeout", or "canceled" — the script maps this to a POSIX exit code.

Key design decisions: (1) use a tempfile.TemporaryDirectory so each run is isolated and cleaned up automatically; (2) honor ANSIBLE_* environment variables so users can tune behavior without editing code; (3) stream events to stderr so stdout stays clean for piping; (4) accept repeatable --extra-vars flags and merge them into a single dict for Ansible.

Full Script

#!/usr/bin/env python3
"""
Ansible Playbook Runner via ansible-runner.

Runs an Ansible playbook through ansible-runner, streams task events to stderr,
and exits with a code that reflects the play recap status.

Usage:
    python3 ansible_run.py --playbook deploy.yml --inventory hosts.ini \
        --extra-vars version=2.1.0
"""

from __future__ import annotations

import argparse
import json
import logging
import os
import shutil
import sys
import tempfile
from pathlib import Path
from typing import Any

import ansible_runner

# ---------------------------------------------------------------------------
# Configuration (environment variables with sensible defaults)
# ---------------------------------------------------------------------------
LOG_LEVEL: str = os.environ.get("ANSIBLE_RUN_LOG_LEVEL", "INFO").upper()
DEFAULT_TIMEOUT: int = int(os.environ.get("ANSIBLE_RUN_TIMEOUT", "1800"))  # seconds
DEFAULT_VERBOSITY: int = int(os.environ.get("ANSIBLE_RUN_VERBOSITY", "1"))
KEEP_ARTIFACTS: bool = os.environ.get("ANSIBLE_RUN_KEEP_ARTIFACTS", "0") == "1"
ARTIFACT_DIR: str | None = os.environ.get("ANSIBLE_RUN_ARTIFACT_DIR")  # None => temp

logging.basicConfig(
    level=getattr(logging, LOG_LEVEL, logging.INFO),
    format="%(asctime)s [%(levelname)s] %(message)s",
    stream=sys.stderr,
)
log = logging.getLogger("ansible-run")


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def parse_extra_vars(pairs: list[str]) -> dict[str, Any]:
    """Parse KEY=VALUE strings into a dict; values are JSON-decoded when possible."""
    result: dict[str, Any] = {}
    for pair in pairs:
        if "=" not in pair:
            raise argparse.ArgumentTypeError(
                f"--extra-vars expects KEY=VALUE, got: {pair!r}"
            )
        key, raw = pair.split("=", 1)
        try:
            result[key] = json.loads(raw)  # handles ints, bools, lists, JSON objects
        except json.JSONDecodeError:
            result[key] = raw  # fall back to plain string
    return result


def event_handler(event: dict[str, Any]) -> bool:
    """Called by ansible-runner for every event. Return True to keep running."""
    etype = event.get("event")
    if etype == "runner_on_ok":
        host = event.get("event_data", {}).get("host", "?")
        task = event.get("event_data", {}).get("task", "?")
        log.info("OK   %-20s %s", host, task)
    elif etype == "runner_on_failed":
        host = event.get("event_data", {}).get("host", "?")
        task = event.get("event_data", {}).get("task", "?")
        log.error("FAIL %-20s %s", host, task)
    elif etype == "runner_on_unreachable":
        host = event.get("event_data", {}).get("host", "?")
        log.error("UNREACHABLE %s", host)
    elif etype == "playbook_on_play_start":
        log.info("=== PLAY: %s ===", event.get("event_data", {}).get("name", "?"))
    return True  # never cancel from the handler


def status_handler(status: dict[str, Any], runner: Any) -> bool:
    """Called on status transitions (e.g. running -> successful)."""
    log.debug("status=%s", status.get("status"))
    return True


def build_private_data_dir(
    playbook: Path,
    inventory: Path | None,
    extra_vars: dict[str, Any],
) -> tuple[str, str]:
    """
    Prepare an isolated working directory with the project tree that
    ansible-runner expects: project/, inventory/, env/.
    Returns (private_data_dir, playbook_basename).
    """
    if ARTIFACT_DIR:
        Path(ARTIFACT_DIR).mkdir(parents=True, exist_ok=True)
        pdd = tempfile.mkdtemp(prefix="ansible-run-", dir=ARTIFACT_DIR)
    else:
        pdd = tempfile.mkdtemp(prefix="ansible-run-")

    project_dir = Path(pdd) / "project"
    project_dir.mkdir()

    # Copy playbook (and its sibling files, e.g. roles/, files/) into project/
    shutil.copy2(playbook, project_dir / playbook.name)

    # Copy inventory if provided
    if inventory is not None:
        inv_dir = Path(pdd) / "inventory"
        inv_dir.mkdir()
        shutil.copy2(inventory, inv_dir / inventory.name)

    # Persist extra vars so ansible-runner injects them
    if extra_vars:
        env_dir = Path(pdd) / "env"
        env_dir.mkdir()
        (env_dir / "extravars").write_text(json.dumps(extra_vars))

    return pdd, playbook.name


# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def run_playbook(args: argparse.Namespace) -> int:
    playbook = Path(args.playbook).resolve()
    if not playbook.is_file():
        log.error("playbook not found: %s", playbook)
        return 2

    inventory = Path(args.inventory).resolve() if args.inventory else None
    if inventory is not None and not inventory.is_file():
        log.error("inventory not found: %s", inventory)
        return 2

    extra_vars = parse_extra_vars(args.extra_vars)

    pdd, playbook_name = build_private_data_dir(playbook, inventory, extra_vars)
    log.info("private_data_dir=%s", pdd)

    try:
        runner = ansible_runner.run(
            private_data_dir=pdd,
            playbook=playbook_name,
            inventory=str(inventory) if inventory else None,
            extravars=extra_vars or None,
            verbosity=args.verbosity,
            timeout=args.timeout,
            quiet=True,                 # we render events ourselves
            event_handler=event_handler,
            status_handler=status_handler,
        )
    except ansible_runner.exceptions.AnsibleRunnerException as exc:
        log.exception("ansible-runner failed to start: %s", exc)
        return 3
    except Exception as exc:  # noqa: BLE001 - last-resort safety net
        log.exception("unexpected error: %s", exc)
        return 4

    log.info("final status: %s (rc=%s)", runner.status, runner.rc)

    # Map runner status to a POSIX exit code
    status_map = {
        "successful": 0,
        "failed": 1,
        "timeout": 124,
        "canceled": 130,
    }
    return status_map.get(runner.status, 5)


def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Run an Ansible playbook via ansible-runner.",
    )
    parser.add_argument("--playbook", "-p", required=True,
                        help="Path to the playbook YAML file.")
    parser.add_argument("--inventory", "-i", default=None,
                        help="Path to an inventory file (optional).")
    parser.add_argument("--extra-vars", "-e", action="append", default=[],
                        metavar="KEY=VALUE",
                        help="Extra vars; repeatable. Values parsed as JSON when possible.")
    parser.add_argument("--timeout", type=int, default=DEFAULT_TIMEOUT,
                        help=f"Overall timeout in seconds (default: {DEFAULT_TIMEOUT}).")
    parser.add_argument("--verbosity", "-v", type=int, default=DEFAULT_VERBOSITY,
                        help=f"Ansible verbosity 0-5 (default: {DEFAULT_VERBOSITY}).")
    return parser.parse_args(argv)


def main(argv: list[str] | None = None) -> int:
    args = parse_args(argv)
    log.info("running playbook=%s inventory=%s", args.playbook, args.inventory)

    try:
        return run_playbook(args)
    except KeyboardInterrupt:
        log.warning("interrupted by user")
        return 130
    finally:
        # Clean up temp dir unless the user asked to keep artifacts
        if not KEEP_ARTIFACTS:
            for entry in Path(tempfile.gettempdir()).glob("ansible-run-*"):
                # Only remove our own temp dirs
                if entry.is_dir() and entry.name.startswith("ansible-run-"):
                    shutil.rmtree(entry, ignore_errors=True)


if __name__ == "__main__":
    sys.exit(main())

Configuration

Standard Ansible environment variables (ANSIBLE_HOST_KEY_CHECKING, ANSIBLE_SSH_ARGS, ANSIBLE_CONFIG, etc.) are also honored because ansible-runner invokes the real Ansible engine.

Usage Examples

1. Basic run against an inventory file:

$ python3 ansible_run.py --playbook deploy.yml --inventory hosts.ini \
    --extra-vars version=2.1.0
2025-01-15 10:22:01 [INFO] running playbook=deploy.yml inventory=hosts.ini
2025-01-15 10:22:01 [INFO] private_data_dir=/tmp/ansible-run-a3f9c1
2025-01-15 10:22:02 [INFO] === PLAY: Deploy application ===
2025-01-15 10:22:03 [INFO] OK   web-01               Install package
2025-01-15 10:22:04 [INFO] OK   web-02               Install package
2025-01-15 10:22:05 [INFO] final status: successful (rc=0)
$ echo $?
0

2. Multiple extra vars with typed values (JSON-parsed):

$ python3 ansible_run.py -p rolling_restart.yml -i prod.ini \
    -e batch_size=2 -e maintenance=true -e 'tags=["api","web"]'

batch_size becomes an int, maintenance a bool, and tags a list — no manual quoting gymnastics.

3. Keep artifacts for debugging a failed run:

$ ANSIBLE_RUN_KEEP_ARTIFACTS=1 ANSIBLE_RUN_LOG_LEVEL=DEBUG \
    python3 ansible_run.py -p deploy.yml -i hosts.ini -e version=2.1.0
2025-01-15 10:31:00 [ERROR] FAIL web-02               Restart service
2025-01-15 10:31:01 [INFO] final status: failed (rc=2)
$ echo $?
1
$ ls /tmp/ansible-run-*/artifacts/   # stdout, stderr, job_events/ available

Error Handling

• Missing playbook or inventory → logged as ERROR, exit code 2. No partial Ansible invocation.
• ansible-runner fails to start (e.g. ansible binary not on PATH, malformed playbook) → AnsibleRunnerException caught, exit code 3.
• Playbook task failure or unreachable host → runner.status == "failed", exit code 1. The event_handler already logged which host/task failed.
• Timeout exceeded → runner.status == "timeout", exit code 124 (matches timeout(1) convention).
• User hits Ctrl-C → KeyboardInterrupt caught, exit code 130 (matches shell SIGINT convention).
• Unexpected exceptions → caught by the final except Exception, exit code 4, full traceback logged so nothing is silently swallowed.
• Cleanup always runs in finally; temp dirs are removed unless ANSIBLE_RUN_KEEP_ARTIFACTS=1.

Extending Ideas

• Return structured JSON results — replace the event handler's log calls with an in-memory list of {host, task, status, duration} dicts and dump to stdout at the end, so callers can pipe into jq.
• Support multiple inventories or an inventory directory — accept --inventory as action="append" and copy each into private_data_dir/inventory/; Ansible will merge them.
• Add a --check / --diff dry-run mode — pass cmdline="--check --diff" to ansible_runner.run() and log the pending changes without touching hosts.
• Vault password file support — add --vault-password-file and pass it via envvars={"ANSIBLE_VAULT_PASSWORD_FILE": path} so encrypted vars decrypt automatically.
• Prometheus metrics — wrap run_playbook with prometheus_client.Counter / Histogram keyed by playbook name and final status for observability in CI dashboards.