Simple Key-Value Config Store with File Backend

Problem Statement

Managing local developer configuration, active environment settings, or CLI state often leads to fragmented environment variables, hardcoded scripts, or messy custom configuration parsing logic. System administrators and DevOps engineers frequently need a lightweight mechanism to quickly read, set, list, or delete key-value options from standard shell scripts without installing external dependencies like Redis or SQLite.

This script solves the local persistence problem by providing a simple, file-backed key-value configuration store accessible directly through a command-line interface. It allows users to quickly inspect and manipulate key-value pairs across shell sessions, utility scripts, and automation pipelines using standard library Python.

How It Works

The architecture relies on a structured JSON backend for persistence and Python's standard argparse module for command routing. Keys and values are stored inside a JSON document, defaulting to ~/.config/kv_store/config.json. The location can be overridden using the KV_STORE_FILE environment variable or the --file command-line argument.

To prevent data corruption during write operations (for example, if a script is interrupted or experiences a power loss mid-write), the store utilizes an atomic write design pattern. New state is written to a temporary file in the same target directory before being atomically renamed over the existing store file using pathlib.Path.replace().

                  +-------------------------+
                  |  CLI Arguments / Flags  |
                  +------------+------------+
                               |
                               v
                  +-------------------------+
                  |   KeyValueStore Engine  |
                  +------------+------------+
                               |
            +------------------+------------------+
            | Read                                | Write (Atomic)
            v                                     v
   +-----------------+                    +---------------+
   |   config.json   |                    |  config.tmp   |
   +-----------------+                    +-------+-------+
                                                  | Rename
                                                  v
                                          +---------------+
                                          |  config.json  |
                                          +---------------+

Full Script

#!/usr/bin/env python3
"""
Simple Key-Value Config Store with File Backend

A production-ready utility to manage simple persistent key-value configuration
data using JSON storage and atomic file updates.
"""

import argparse
import json
import os
import sys
from pathlib import Path
from typing import Optional


# Default store location: ~/.config/kv_store/config.json or KV_STORE_FILE env var
DEFAULT_STORE_PATH = os.environ.get(
    "KV_STORE_FILE",
    str(Path.home() / ".config" / "kv_store" / "config.json"),
)


class KeyValueStore:
    """Manages reading and writing key-value configuration pairs to disk."""

    def __init__(self, filepath: str) -> None:
        self.filepath = Path(filepath).expanduser().resolve()
        self._ensure_storage_exists()

    def _ensure_storage_exists(self) -> None:
        """Create storage directory and initial empty store file if missing."""
        if not self.filepath.exists():
            try:
                self.filepath.parent.mkdir(parents=True, exist_ok=True)
                self._atomic_write({})
            except OSError as e:
                print(f"Error initializing store directory: {e}", file=sys.stderr)
                sys.exit(1)

    def _read_data(self) -> dict[str, str]:
        """Read and parse the JSON configuration store from disk."""
        try:
            with open(self.filepath, "r", encoding="utf-8") as f:
                data = json.load(f)
                if not isinstance(data, dict):
                    raise ValueError("Store file root must be a JSON object.")
                return data
        except (json.JSONDecodeError, ValueError) as e:
            print(f"Error parsing store file '{self.filepath}': {e}", file=sys.stderr)
            sys.exit(1)
        except OSError as e:
            print(f"Error reading store file '{self.filepath}': {e}", file=sys.stderr)
            sys.exit(1)

    def _atomic_write(self, data: dict[str, str]) -> None:
        """Atomically write data to disk using a temporary file."""
        temp_file = self.filepath.with_suffix(".tmp")
        try:
            with open(temp_file, "w", encoding="utf-8") as f:
                json.dump(data, f, indent=2, sort_keys=True)
                f.write("\n")
            # Replace current store with newly written temporary file atomically
            temp_file.replace(self.filepath)
        except OSError as e:
            if temp_file.exists():
                temp_file.unlink()
            print(f"Error writing store file '{self.filepath}': {e}", file=sys.stderr)
            sys.exit(1)

    def get(self, key: str) -> Optional[str]:
        """Retrieve a value by key."""
        data = self._read_data()
        return data.get(key)

    def set(self, key: str, value: str) -> None:
        """Set or update a key-value pair."""
        data = self._read_data()
        data[key] = value
        self._atomic_write(data)

    def delete(self, key: str) -> bool:
        """Delete a key-value pair if it exists. Returns True if deleted."""
        data = self._read_data()
        if key in data:
            del data[key]
            self._atomic_write(data)
            return True
        return False

    def list_all(self) -> dict[str, str]:
        """Return all key-value entries."""
        return self._read_data()

    def clear(self) -> None:
        """Remove all stored keys."""
        self._atomic_write({})


def build_parser() -> argparse.ArgumentParser:
    """Construct and configure command-line argument subcommands."""
    parser = argparse.ArgumentParser(
        description="Simple Key-Value Config Store with File Backend",
        formatter_class=argparse.RawTextHelpFormatter,
    )
    parser.add_argument(
        "--file",
        type=str,
        default=DEFAULT_STORE_PATH,
        help=f"Path to JSON file store (default: {DEFAULT_STORE_PATH})",
    )

    subparsers = parser.add_subparsers(dest="command", required=True)

    # Subcommand: set <key> <value>
    set_parser = subparsers.add_parser("set", help="Set a configuration key")
    set_parser.add_argument("key", help="Key name")
    set_parser.add_argument("value", help="String value to assign")

    # Subcommand: get <key>
    get_parser = subparsers.add_parser("get", help="Get a configuration value")
    get_parser.add_argument("key", help="Key name")

    # Subcommand: delete <key>
    del_parser = subparsers.add_parser("delete", help="Delete a key")
    del_parser.add_argument("key", help="Key name")

    # Subcommand: list
    subparsers.add_parser("list", help="List all stored configuration pairs")

    # Subcommand: clear
    subparsers.add_parser("clear", help="Clear all stored keys")

    return parser


def main() -> None:
    parser = build_parser()
    args = parser.parse_args()

    store = KeyValueStore(filepath=args.file)

    if args.command == "set":
        store.set(args.key, args.value)
        print(f"OK: Set '{args.key}'")

    elif args.command == "get":
        val = store.get(args.key)
        if val is not None:
            print(val)
        else:
            print(f"Error: Key '{args.key}' not found.", file=sys.stderr)
            sys.exit(1)

    elif args.command == "delete":
        if store.delete(args.key):
            print(f"OK: Deleted '{args.key}'")
        else:
            print(f"Error: Key '{args.key}' not found.", file=sys.stderr)
            sys.exit(1)

    elif args.command == "list":
        items = store.list_all()
        if not items:
            print("Store is empty.")
        else:
            for k, v in items.items():
                print(f"{k}={v}")

    elif args.command == "clear":
        store.clear()
        print("OK: Store cleared.")


if __name__ == "__main__":
    main()

Configuration

The behavior of the store can be controlled using either environment variables or runtime flags:

Usage Examples

1. Setting and Reading Configuration Values

$ python3 kv_store.py set API_URL https://api.example.com
OK: Set 'API_URL'

$ python3 kv_store.py set TIMEOUT_SECONDS 30
OK: Set 'TIMEOUT_SECONDS'

$ python3 kv_store.py get API_URL
https://api.example.com

2. Listing and Deleting Entries

$ python3 kv_store.py list
API_URL=https://api.example.com
TIMEOUT_SECONDS=30

$ python3 kv_store.py delete TIMEOUT_SECONDS
OK: Deleted 'TIMEOUT_SECONDS'

$ python3 kv_store.py list
API_URL=https://api.example.com

3. Using Custom Storage Path

$ python3 kv_store.py --file ./project_config.json set ENVIRONMENT staging
OK: Set 'ENVIRONMENT'

$ python3 kv_store.py --file ./project_config.json get ENVIRONMENT
staging

Error Handling

The script handles failure modes gracefully without displaying unhandled Python stack traces:

• Missing Key Errors: Querying (get) or deleting (delete) a key that does not exist prints a clear error message to stderr and exits with code 1.
• Malformed File/JSON: If the store file is manually corrupted or altered to an invalid JSON structure, _read_data() catches json.JSONDecodeError and ValueError, alerts the user on stderr, and exits safely.
• Permission/Disk Errors: Directory creation or permission errors trigger an explicit OSError catch block.
• Interrupt Safety: Writes occur on a separate .tmp file prior to replacement. If a write fails midway, the original configuration file remains untouched, and the temporary file is cleaned up automatically.

Extending Ideas

1. Type Auto-Casting / JSON Values: Modify set to parse values as native JSON primitives (numbers, booleans, objects) when applicable rather than coercing all inputs to strings.
2. Encryption at Rest: Integrate the standard library base64 module combined with user-provided secret keys or standard system utilities to obfuscate sensitive values like tokens and credentials.
3. Namespace / Profile Support: Add a --profile flag to support logical namespaces within the same backend JSON file (e.g., dev, staging, prod).
4. Key TTL (Time to Live): Store entries alongside an expiration epoch timestamp to support transient settings that automatically expire.