Git Blame Statistics Analyzer

Git Blame Statistics Analyzer

Problem Statement

When maintaining a codebase, it's often useful to understand who wrote what — not just for attribution, but for spotting knowledge silos, identifying hotspots that need refactoring, and understanding contributor patterns over time. While git log tells you who committed changes, it doesn't tell you who actually wrote the lines still present in the code today. git blame answers that question, but running it per-file and aggregating the results by hand is tedious and error-prone.

This script automates that workflow. It walks every tracked Python (or configurable) file in a repository, runs blame over each one, and produces a report showing line ownership per author, per file, and per time window. It's useful for onboarding docs, code review process improvements, and identifying files that are effectively "owned" by a single person.

How It Works

The script uses GitPython to open the repository and enumerate tracked files via repo.git.ls_files(). For each file, it invokes repo.blame() (which wraps git blame --line-porcelain) and collects the commit metadata attached to each line. Because blame is expensive, we filter files by extension and skip binary or generated paths, and we cache commit metadata by SHA so we don't repeatedly hit the object store.

The results are aggregated into three views: author totals (lines owned per author across the repo), file breakdown (top authors per file, useful for spotting single-owner files), and recency weighting based on the --since window. The output is a simple aligned text report by default, with an optional JSON mode for piping into other tools. All the knobs — file extensions, ignore patterns, since-window, output format — are exposed via CLI flags and environment variables so the script works both interactively and inside CI.

Full Script

#!/usr/bin/env python3
"""
Git Blame Statistics Analyzer

Walk every tracked source file in a Git repository, run `git blame` on each,
and produce a report of line ownership by author. Useful for spotting
knowledge silos, understanding contributor patterns, and building
onboarding documentation.

Requires: gitpython (pip install gitpython)
"""

from __future__ import annotations

import argparse
import json
import os
import re
import sys
from collections import Counter, defaultdict
from dataclasses import dataclass, field
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Iterable

try:
    from git import Repo, InvalidGitRepositoryError, GitCommandError
except ImportError:
    sys.stderr.write("Error: gitpython is required. Install with: pip install gitpython\n")
    sys.exit(1)


# ---------------------------------------------------------------------------
# Configuration (overridable via environment variables)
# ---------------------------------------------------------------------------

DEFAULT_EXTENSIONS = os.environ.get(
    "BLAME_EXTENSIONS", ".py,.js,.ts,.go,.rs,.java,.rb,.sh"
).split(",")

DEFAULT_IGNORE_DIRS = os.environ.get(
    "BLAME_IGNORE_DIRS", ".git,node_modules,venv,.venv,dist,build,__pycache__"
).split(",")

DEFAULT_SINCE = os.environ.get("BLAME_SINCE", "0d")  # 0d = all history
MAX_FILE_SIZE_BYTES = int(os.environ.get("BLAME_MAX_FILE_SIZE", 500_000))


# ---------------------------------------------------------------------------
# Data structures
# ---------------------------------------------------------------------------

@dataclass
class AuthorStats:
    """Aggregated line ownership for a single author."""
    name: str
    email: str
    lines: int = 0
    files_touched: set[str] = field(default_factory=set)
    oldest_commit: datetime | None = None
    newest_commit: datetime | None = None

    def record(self, commit_dt: datetime, filepath: str) -> None:
        self.lines += 1
        self.files_touched.add(filepath)
        if self.oldest_commit is None or commit_dt < self.oldest_commit:
            self.oldest_commit = commit_dt
        if self.newest_commit is None or commit_dt > self.newest_commit:
            self.newest_commit = commit_dt


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------

_SINCE_RE = re.compile(r"^(\d+)\s*([dwmy])?$", re.IGNORECASE)


def parse_since(value: str) -> datetime | None:
    """Parse '90d', '6m', '2w', '1y' into a UTC cutoff datetime.

    Returns None for '0d' or empty input, meaning 'no cutoff'.
    """
    value = (value or "").strip()
    if not value or value == "0d":
        return None

    m = _SINCE_RE.match(value)
    if not m:
        raise ValueError(f"Invalid --since value: {value!r} (expected e.g. 90d, 6m, 1y)")

    n = int(m.group(1))
    unit = (m.group(2) or "d").lower()
    days = {"d": 1, "w": 7, "m": 30, "y": 365}[unit] * n
    return datetime.now(timezone.utc) - timedelta(days=days)


def should_skip(path: str, extensions: Iterable[str], ignore_dirs: Iterable[str]) -> bool:
    """Decide whether a tracked file should be excluded from the analysis."""
    parts = Path(path).parts
    if any(ig and ig in parts for ig in ignore_dirs):
        return True
    if extensions and Path(path).suffix not in extensions:
        return True
    return False


def format_table(rows: list[tuple[str, ...]], headers: tuple[str, ...]) -> str:
    """Render a simple aligned text table."""
    widths = [len(h) for h in headers]
    for row in rows:
        for i, cell in enumerate(row):
            widths[i] = max(widths[i], len(cell))

    def fmt(row: Iterable[str]) -> str:
        return "  ".join(cell.ljust(w) for cell, w in zip(row, widths))

    sep = "  ".join("-" * w for w in widths)
    lines = [fmt(headers), sep]
    lines.extend(fmt(r) for r in rows)
    return "\n".join(lines)


# ---------------------------------------------------------------------------
# Core analysis
# ---------------------------------------------------------------------------

def analyze_repo(
    repo_path: Path,
    since: datetime | None,
    extensions: list[str],
    ignore_dirs: list[str],
) -> tuple[dict[str, AuthorStats], dict[str, Counter[str]], int]:
    """Run blame over every relevant file and aggregate stats.

    Returns:
        (author_stats, file_author_counts, total_lines)
    """
    try:
        repo = Repo(repo_path)
    except InvalidGitRepositoryError as exc:
        raise SystemExit(f"Error: {repo_path} is not a Git repository") from exc

    if repo.bare:
        raise SystemExit("Error: bare repositories are not supported")

    try:
        tracked = repo.git.ls_files().splitlines()
    except GitCommandError as exc:
        raise SystemExit(f"Error: could not list tracked files: {exc}") from exc

    author_stats: dict[str, AuthorStats] = {}
    file_author_counts: dict[str, Counter[str]] = {}
    total_lines = 0

    # Cache commit SHA -> (author_name, author_email, datetime) so we don't
    # repeatedly look up the same commit object across thousands of lines.
    commit_cache: dict[str, tuple[str, str, datetime]] = {}

    for relpath in tracked:
        if should_skip(relpath, extensions, ignore_dirs):
            continue

        abs_path = repo_path / relpath
        try:
            if abs_path.stat().st_size > MAX_FILE_SIZE_BYTES:
                continue
        except OSError:
            continue

        try:
            # blame returns list[list[Commit, list[str]]] — one entry per
            # contiguous run of lines from the same commit.
            blamed = repo.blame("HEAD", relpath)
        except GitCommandError:
            # File may be untracked at HEAD, deleted, or unreadable. Skip it.
            continue

        file_counter: Counter[str] = Counter()

        for commit, lines in blamed:
            sha = commit.hexsha
            if sha not in commit_cache:
                dt = datetime.fromtimestamp(commit.committed_date, tz=timezone.utc)
                commit_cache[sha] = (commit.author.name, commit.author.email, dt)

            name, email, dt = commit_cache[sha]

            if since is not None and dt < since:
                continue

            key = f"{name} <{email}>"
            stats = author_stats.setdefault(key, AuthorStats(name=name, email=email))

            for _ in lines:
                stats.record(dt, relpath)
                file_counter[key] += 1
                total_lines += 1

        if file_counter:
            file_author_counts[relpath] = file_counter

    return author_stats, file_author_counts, total_lines


# ---------------------------------------------------------------------------
# Reporting
# ---------------------------------------------------------------------------

def render_text_report(
    author_stats: dict[str, AuthorStats],
    file_author_counts: dict[str, Counter[str]],
    total_lines: int,
    top_n: int,
) -> str:
    out: list[str] = []
    out.append(f"Total lines analyzed: {total_lines}")
    out.append(f"Contributors: {len(author_stats)}")
    out.append("")

    # --- Author totals ---
    out.append("== Author Totals ==")
    rows = []
    for key, s in sorted(author_stats.items(), key=lambda kv: kv[1].lines, reverse=True):
        pct = (s.lines / total_lines * 100) if total_lines else 0.0
        newest = s.newest_commit.strftime("%Y-%m-%d") if s.newest_commit else "-"
        rows.append((s.name, str(s.lines), f"{pct:5.1f}%", str(len(s.files_touched)), newest))
    out.append(format_table(rows, ("Author", "Lines", "Share", "Files", "Last seen")))
    out.append("")

    # --- Per-file breakdown: show top authors per file ---
    out.append(f"== Top {top_n} Authors per File (files with >1 contributor) ==")
    multi_owner = {f: c for f, c in file_author_counts.items() if len(c) > 1}
    if not multi_owner:
        out.append("(no files with multiple contributors in the selected window)")
    else:
        for filepath in sorted(multi_owner):
            counter = multi_owner[filepath]
            file_total = sum(counter.values())
            summary = ", ".join(
                f"{name.split(' <')[0]} ({cnt})"
                for name, cnt in counter.most_common(top_n)
            )
            out.append(f"  {filepath} [{file_total} lines]: {summary}")

    return "\n".join(out)


def render_json_report(
    author_stats: dict[str, AuthorStats],
    file_author_counts: dict[str, Counter[str]],
    total_lines: int,
) -> str:
    payload = {
        "total_lines": total_lines,
        "contributors": [
            {
                "name": s.name,
                "email": s.email,
                "lines": s.lines,
                "files_touched": sorted(s.files_touched),
                "oldest_commit": s.oldest_commit.isoformat() if s.oldest_commit else None,
                "newest_commit": s.newest_commit.isoformat() if s.newest_commit else None,
            }
            for s in sorted(author_stats.values(), key=lambda x: x.lines, reverse=True)
        ],
        "files": {
            path: dict(counter) for path, counter in file_author_counts.items()
        },
    }
    return json.dumps(payload, indent=2)


# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------

def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(
        prog="git_blame_stats.py",
        description="Analyze git blame output to report line ownership by author.",
    )
    p.add_argument("--repo", default=".", help="Path to the Git repository (default: .)")
    p.add_argument(
        "--since",
        default=DEFAULT_SINCE,
        help="Only count lines whose commit is newer than this (e.g. 90d, 6m, 1y). Default: %(default)s",
    )
    p.add_argument(
        "--extensions",
        default=",".join(DEFAULT_EXTENSIONS),
        help="Comma-separated file extensions to include (default: %(default)s)",
    )
    p.add_argument(
        "--ignore-dirs",
        default=",".join(DEFAULT_IGNORE_DIRS),
        help="Comma-separated directory names to skip (default: %(default)s)",
    )
    p.add_argument(
        "--top",
        type=int,
        default=3,
        help="How many authors to show per file (default: %(default)s)",
    )
    p.add_argument(
        "--author-report",
        action="store_true",
        help="Print the author totals report (default behavior).",
    )
    p.add_argument(
        "--json",
        action="store_true",
        help="Emit JSON instead of a text report.",
    )
    return p


def main(argv: list[str] | None = None) -> int:
    args = build_parser().parse_args(argv)

    repo_path = Path(args.repo).resolve()
    if not repo_path.exists():
        sys.stderr.write(f"Error: path does not exist: {repo_path}\n")
        return 2

    try:
        since = parse_since(args.since)
    except ValueError as exc:
        sys.stderr.write(f"Error: {exc}\n")
        return 2

    extensions = [e.strip() for e in args.extensions.split(",") if e.strip()]
    ignore_dirs = [d.strip() for d in args.ignore_dirs.split(",") if d.strip()]

    author_stats, file_author_counts, total_lines = analyze_repo(
        repo_path=repo_path,
        since=since,
        extensions=extensions,
        ignore_dirs=ignore_dirs,
    )

    if args.json:
        print(render_json_report(author_stats, file_author_counts, total_lines))
    else:
        print(render_text_report(author_stats, file_author_counts, total_lines, args.top))

    return 0


if __name__ == "__main__":
    sys.exit(main())

Configuration

All configuration is available both as CLI flags and environment variables, so it can be set once in CI and overridden interactively.

CLI flags override the environment: --extensions, --ignore-dirs, --since, --top, --json, --repo.

Usage Examples

1. Full author report on the current repo (all history):

$ python3 git_blame_stats.py --repo . --author-report
Total lines analyzed: 4821
Contributors: 7

== Author Totals ==
Author             Lines  Share  Files  Last seen
Alice Example       2134  44.3%     62  2024-11-02
Bob Example         1387  28.8%     41  2024-10-18
Carol Example        712  14.8%     23  2024-09-30
...

2. Last 90 days only, JSON output for CI ingestion:

$ python3 git_blame_stats.py --repo . --since 90d --json | jq '.contributors[0]'
{
  "name": "Alice Example",
  "email": "alice@example.com",
  "lines": 412,
  "files_touched": ["src/api.py", "src/models.py", ...],
  "oldest_commit": "2024-08-15T10:22:01+00:00",
  "newest_commit": "2024-11-02T08:11:44+00:00"
}

3. Python files only, top 5 authors per file:

$ python3 git_blame_stats.py --extensions .py --top 5
== Top 5 Authors per File (files with >1 contributor) ==
  src/api.py [312 lines]: Alice (204), Bob (88), Carol (20)
  src/models.py [188 lines]: Bob (140), Alice (48)

Error Handling

• Not a Git repo / bare repo — Repo() raises InvalidGitRepositoryError, which is caught and converted into a friendly SystemExit message.
• git ls-files fails — wrapped in GitCommandError handling, exits with a clear message rather than a traceback.
• Per-file blame failures — a file may be tracked but missing at HEAD (staged deletion, submodule weirdness). These are caught and the file is skipped silently so one bad file doesn't abort the run.
• Oversized files — skipped based on BLAME_MAX_FILE_SIZE to avoid pathological blame runs on generated bundles.
• Invalid --since — parse_since raises ValueError, which main converts to exit code 2 with a helpful message.
• Missing gitpython — detected at import time and reported with the install hint.
• Unreadable file stats — OSError on stat() is caught and the file is skipped.

Exit codes: 0 success, 2 bad arguments or missing path.

Extending Ideas

• Add a churn-vs-ownership heatmap — combine blame line counts with git log --numstat to flag files where one author owns most lines and the file changes frequently. Those are the highest-risk knowledge silos.
• Track "bus factor" per file — compute the minimum number of authors that cover >50% of a file's lines and sort files by that number ascending. Files with a bus factor of 1 are candidates for pairing or documentation.
• Cache results to disk — blame is slow on large repos. Persist commit_cache and per-file counters to a JSON file keyed by HEAD SHA, and only re-blame files whose blob SHA changed.
• Slack/webhook output — add a --notify <webhook> flag that posts the top-3 authors per file for recently modified files, so reviewers know who to ping.