Parallelize Test Shards with Buildkite Matrix and RSpec Bisect

This configuration uses Buildkite's matrix expansion to fan out a single test step into N shards, each running a disjoint subset of the suite. The matrix directive generates one job per combination of the shard index, and Buildkite schedules them concurrently across the agent pool. Inside each job, rspec --bisect or a partitioner splits the spec files deterministically so no two shards run the same example.

Use this when a test suite exceeds ten minutes and agent capacity is available. It is standard for large Rails and Node monorepos where the suite grows faster than single-machine CPU. The matrix approach keeps the pipeline YAML DRY while still giving per-shard retries and logs.

Security is mostly unchanged, but sharding multiplies secret exposure surface, so scope test credentials to read-only and short-lived tokens. Performance scales near-linearly until agent queue contention dominates; monitor queue wait time alongside shard duration. Common pitfalls include non-deterministic partitioning that causes flaky failures on different shards, and forgetting to merge coverage reports from all shards before uploading, which silently lowers reported coverage.

> System.Execute()

steps:
  - label: "RSpec shard %n"
    command: |
      bundle exec rspec --format progress \
        --seed $$((BUILDKITE_PARALLEL_JOB + 1)) \
        $$(bundle exec rspec --dry-run --format json | \
          jq -r --argjson i $$BUILDKITE_PARALLEL_JOB \
          '.examples[].id' | awk -v i=$$BUILDKITE_PARALLEL_JOB -v n=$$BUILDKITE_PARALLEL_JOB_COUNT 'NR % n == i')
    parallelism: 8
    retry:
      automatic:
        limit: 1

> Architecture & Security Context

Verified for Valters IT Hub. When deploying this configuration in a production homelab or enterprise environment, ensure that you monitor system resources and implement strict access controls. Administrator privileges are required.

ID: VALTERSIT-2B7A40880B9E