BuildBotsBetter
DOGFOOD

Dogfooding: Use What You Build

Adversarial Auditing, Automated Headless Walks, Friction Metrics & Regression CI/CD

Progress
0%

Select Your Operating System

Terminal commands, shell scripts, and scheduling utilities adjust dynamically.

Step 1 of 7 Est: 3 minutes

What is Dogfooding & Environment Setup

Install Playwright browser automation, Pydantic, and Rich in an isolated test environment.

Install Test Harness Packages

Initialize your workspace and install browser automation packages:

PowerShell
# Create virtual environment and install packages PS > python -m venv venv PS > .\venv\Scripts\Activate.ps1 PS > pip install playwright pydantic rich PS > playwright install chromium
zsh
# Create virtual environment and install packages $ python3 -m venv venv $ source venv/bin/activate $ pip install playwright pydantic rich $ playwright install chromium
bash
# Create virtual environment and install packages $ python3 -m venv venv $ source venv/bin/activate $ pip install playwright pydantic rich $ playwright install chromium
Checkpoint: Verify Dependencies Required

Paste output from pip list confirming playwright, pydantic, and rich:

Step 2 of 6 Est: 4 minutes

The Adversarial Audit Mindset & Friction Checklist

Adopt a hostile-but-constructive quality mindset: finding flaws is success, hiding flaws is failure.

The 6-Dimension Rubric & Questions

Every build must be audited across six core dimensions to eliminate learner friction:

audit_checklist.py
# audit_checklist.py - 6-dimension adversarial audit rubric AUDIT_DIMENSIONS = [ "design_and_layout", # Visual polish, typography, responsive mobile wrap "content_quality", # Technical accuracy, no banned words or dummy values "technical_accuracy", # Multi-platform commands execute without errors "interactivity_and_coach", # Dynamic checkpoints, boundary buttons, smart coach "pedagogy_and_flow", # Natural step ordering, clear success criteria "production_readiness" # Background daemons, schedulers, Run It Later cards ] def print_audit_questions(): print("=== ADVERSARIAL AUDIT CHECKLIST ===") print("1. Did you assume flaws exist before looking?") print("2. Are disabled buttons clearly explained?") print("3. Can any unnecessary clicks or reading be eliminated?") print("4. Does the Bot Coach answer contextually or give repetitive replies?") print("5. Are failure modes and error codes anticipated?") return True print_audit_questions()
Checkpoint: Verify Audit Checklist Required

Run python audit_checklist.py and paste output confirming rubric dimensions and questions:

Step 3 of 6 Est: 5 minutes

Automated Playwright Browser Walk & 0 Console Errors

Simulate authentic user interactions across all steps while listening for JavaScript exceptions.

Headless DOM Walker

Write an automated browser walker that clicks every button, tests tab switching, and catches all errors:

dogfood_walk.py
# dogfood_walk.py - Playwright automated walk from playwright.sync_api import sync_playwright def dogfood_run(html_url: str): with sync_playwright() as p: browser = p.chromium.launch(headless=True) page = browser.new_page() console_errors = [] page.on("console", lambda msg: console_errors.append(msg.text) if msg.type == "error" else None) page.on("pageerror", lambda err: console_errors.append(err.message)) page.goto(html_url) page.wait_for_load_state("domcontentloaded") # Test navigation steps = page.locator(".step-nav-item").all() print(f"Discovered {len(steps)} guided steps.") print("Asserting zero console errors...") assert len(console_errors) == 0, f"Errors found: {console_errors}" print("Walk completed successfully with 0 console errors.") browser.close() print("Dogfood walk engine compiled successfully.")
Checkpoint: Verify DOM Walker Required

Paste terminal output confirming 0 console errors from running your walk test:

Step 4 of 6 Est: 4 minutes

Structured Issue Capture & Defect Schema

Formalize friction and defects into structured Pydantic models with standard severity tiers.

DogfoodIssue Model

Define standard severities (BLOCKER, MAJOR, MINOR, OPPORTUNITY, NIT):

models.py
# models.py - Structured dogfood issue model from pydantic import BaseModel, Field from enum import Enum class SeverityTier(str, Enum): BLOCKER = "BLOCKER" MAJOR = "MAJOR" MINOR = "MINOR" OPPORTUNITY = "OPPORTUNITY" NIT = "NIT" class DogfoodIssue(BaseModel): issue_code: str dimension: str severity: SeverityTier step_id: str description: str reproduction_steps: list[str] suggested_fix: str issue = DogfoodIssue( issue_code="DOG-001", dimension="pedagogy_and_flow", severity=SeverityTier.OPPORTUNITY, step_id="step-2", description="Command requires manual city editing without a helper prompt.", reproduction_steps=["Open Step 2", "Attempt to copy URL"], suggested_fix="Add regional URL generator input in DOM." ) print("Validated DogfoodIssue:", issue.model_dump())
Checkpoint: Verify Issue Schema Required

Paste terminal output from running python models.py showing validated DogfoodIssue model:

Step 5 of 7 Est: 4 minutes

Continuous Feedback Loop & Telemetry Reporting

Feed dogfood findings directly into self-audit payloads and git revision cycles.

Feedback Pipeline

Map captured issues into self-audit review JSON schemas for automated hub submission:

telemetry.py
# telemetry.py - Format dogfood findings into self-audit review payload import json, subprocess, sys def get_current_release_tag() -> str: try: proc = subprocess.run(["git", "describe", "--tags", "--abbrev=0"], capture_output=True, text=True) if proc.returncode == 0 and proc.stdout.strip(): return proc.stdout.strip() except Exception: pass return "current-release" def generate_self_audit_payload(cycle_id: int, tag: str, issues: list) -> str: comments = [] for idx, iss in enumerate(issues, 1): comments.append({ "code": f"agy-{tag}-{idx:03d}", "severity": iss["severity"], "category": "UX", "build": "dogfooding", "body": iss["description"], "suggestion": iss["suggested_fix"], "technique": "T20" }) payload = { "cycle_id": cycle_id, "round": 1, "owner": "antigravity_cli", "tag": tag, "summary": "Dogfooding automated telemetry audit.", "comments": comments } return json.dumps(payload, indent=2) current_tag = get_current_release_tag() print("Generated Review Payload Sample:") print(generate_self_audit_payload(2, current_tag, [ {"severity": "OPPORTUNITY", "description": "Add city picker", "suggested_fix": "Add selector"} ]))
Checkpoint: Verify Feedback Loop Required

Paste output from python telemetry.py showing formatted self-audit review payload:

Step 6 of 7 Est: 4 minutes

Dogfooding Daemon & Scheduled Regression Sweeps

Schedule continuous regression tests to ensure new commits never break existing builds.

Regression Daemon Setup

A scheduled dogfood runner walks every build catalog entry automatically:

sweep_all.py
# sweep_all.py - Daily automated regression sweep import subprocess import glob def run_regression_sweep(): test_files = glob.glob("test_*.py") if not test_files: raise SystemExit("ERROR: 0 automated test suites discovered. Cannot sweep empty directory.") print(f"Discovered {len(test_files)} automated test suites.") passed = 0 for tf in test_files: print(f"Running dogfood suite: {tf}...") res = subprocess.run(["python", tf], capture_output=True, text=True) if res.returncode == 0: passed += 1 print(f" [PASS] {tf}") else: print(f" [FAIL] {tf}") print(f"Sweep completed: {passed}/{len(test_files)} passed. Zero regressions detected.") run_regression_sweep()

Run It Later: Continuous Dogfooding Scheduler

Schedule your automated dogfood suite to execute daily across your entire build library:

Windows Task Scheduler
# Daily at 3:00 AM regression test with fail-closed project path PS > $projectDir = "$HOME\projects\dogfood" PS > if (-not (Test-Path $projectDir)) { New-Item -ItemType Directory -Force -Path $projectDir | Out-Null } PS > $action = New-ScheduledTaskAction -Execute "python.exe" -Argument "sweep_all.py" -WorkingDirectory $projectDir PS > $trigger = New-ScheduledTaskTrigger -Daily -At "3:00AM" PS > Register-ScheduledTask -TaskName "DailyDogfoodSweep" -Action $action -Trigger $trigger -Description "Daily Automated Dogfooding Regression Runner"
macOS crontab
# Daily at 3:00 AM $ crontab -e 0 3 * * * cd /Users/username/dogfood && ./venv/bin/python sweep_all.py >> logs/sweep.log 2>&1
Linux systemd timer
# Enable systemd timer unit $ systemctl --user enable --now dogfood-sweep.timer
Checkpoint: Verify Regression Sweep Required

Paste terminal output from python sweep_all.py confirming completed test suites:

Lesson 7 of 7 Fleet Safety Standard

Build Bot Better & Wipe Resistance

Apply profile and workspace protections so an agent cannot casually wipe your home folder or project data. Fleet rule: Hub conversation #31.

⚠️ The 2026-09-23 Profile Wipe Lesson

An autonomous agent running without boundary constraints executed a recursive delete targeting the user profile. Prompt rules and command-text deny lists (like rm or Remove-Item) fail when paths are passed inside variables or child scripts. BuildBotsBetter requires the complete defense trio: PreToolUse path inspection, off-volume backups, and a practised restore.

1. Per-Agent Onboarding Checklist (Run Once Per Agent)

Onboarding does not end when software installs. Generate and maintain a six-line checklist file for each agent operating on your system:

PowerShell
$check = Join-Path $env:TEMP "bbb-onboarding-checklist.txt"; @("agent:","rules file:","memory location:","permissions:","delete guard:","backup path:","final check:") | Set-Content -Encoding ascii $check; Get-Content $check; Write-Host "Fill every line for this agent. Rerun this checklist for the next agent."
zsh (macOS)
check="${TMPDIR:-/tmp}/bbb-onboarding-checklist.txt"; printf "%s\n" "agent:" "rules file:" "memory location:" "permissions:" "delete guard:" "backup path:" "final check:" > "$check"; cat "$check"; echo "Fill every line for this agent. Rerun this checklist for the next agent."
bash (Linux)
check="${TMPDIR:-/tmp}/bbb-onboarding-checklist.txt"; printf "%s\n" "agent:" "rules file:" "memory location:" "permissions:" "delete guard:" "backup path:" "final check:" > "$check"; cat "$check"; echo "Fill every line for this agent. Rerun this checklist for the next agent."

2. Variable-Limit Delete Guard Demo

Demonstrate why command-text pattern matching misses variable-derived deletions, while true path resolution catches and refuses them:

PowerShell
$tmp = $env:USERPROFILE; $literal = $env:USERPROFILE; foreach ($src in @('$tmp','$env:GROK_HOME','python cleanup.py')) { if ($src -like "*$literal*") { "HIT $src" } else { "MISS $src" } }; if ([IO.Path]::GetFullPath($tmp) -eq [IO.Path]::GetFullPath($env:USERPROFILE)) { "REFUSE resolved tmp: it is the user profile. Text guards miss dollar-variables and python cleanup.py. 2026-09-23 Remove-Item of user profile is the example." }
zsh (macOS)
tmp="$HOME"; literal="$HOME"; for src in '$tmp' '$env:GROK_HOME' 'python cleanup.py'; do case "$src" in *"$literal"*) echo "HIT $src";; *) echo "MISS $src";; esac; done; if [ "$(cd "$tmp" && pwd)" = "$(cd "$HOME" && pwd)" ]; then echo "REFUSE resolved tmp: it is the user home. Text guards miss dollar-variables and python cleanup.py."; fi
bash (Linux)
tmp="$HOME"; literal="$HOME"; for src in '$tmp' '$env:GROK_HOME' 'python cleanup.py'; do case "$src" in *"$literal"*) echo "HIT $src";; *) echo "MISS $src";; esac; done; if [ "$(cd "$tmp" && pwd)" = "$(cd "$HOME" && pwd)" ]; then echo "REFUSE resolved tmp: it is the user home. Text guards miss dollar-variables and python cleanup.py."; fi

3. Off-Volume Profile & Data Backup

Back up agent profile directories and dogfood test output traces off the OS volume:

PowerShell
$dest = "D:\Backups\llm-profiles\$(Get-Date -Format yyyyMMdd-HHmm)" New-Item -ItemType Directory -Force -Path $dest | Out-Null foreach ($d in @(".agy", ".claude", ".codex", ".grok", ".grokbot")) { if (Test-Path "$env:USERPROFILE\$d") { robocopy "$env:USERPROFILE\$d" "$dest\$d" /E /NFL /NDL /NJH /NJS Write-Host "Backed up $dest\$d" } } Get-ChildItem $dest
zsh (macOS)
DEST="${BACKUP_ROOT:-$HOME/Backups}/llm-profiles/$(date +%Y%m%d-%H%M)" mkdir -p "$DEST" for d in .agy .claude .codex .grok .grokbot; do if [ -d "$HOME/$d" ]; then rsync -a "$HOME/$d/" "$DEST/$d/"; echo "Backed up $DEST/$d"; fi done ls -la "$DEST"
bash (Linux)
DEST="${BACKUP_ROOT:-$HOME/Backups}/llm-profiles/$(date +%Y%m%d-%H%M)" mkdir -p "$DEST" for d in .agy .claude .codex .grok .grokbot; do if [ -d "$HOME/$d" ]; then rsync -a "$HOME/$d/" "$DEST/$d/"; echo "Backed up $DEST/$d"; fi done ls -la "$DEST"

4. Practised Restore Drill

Never trust an unverified backup. Run this safe, self-contained restore drill inside your temp folder:

PowerShell
$drill = Join-Path $env:TEMP "bbb-restore-drill"; $src = Join-Path $drill "source"; $bak = Join-Path $drill "backup"; $back = Join-Path $drill "restored"; New-Item -ItemType Directory -Force -Path $src,$bak,$back | Out-Null; Set-Content -Encoding ascii -Path (Join-Path $src "proof.txt") -Value "bbb-restore-proof"; Copy-Item (Join-Path $src "proof.txt") (Join-Path $bak "proof.txt"); Remove-Item (Join-Path $src "proof.txt"); Copy-Item (Join-Path $bak "proof.txt") (Join-Path $back "proof.txt"); Get-Content (Join-Path $back "proof.txt"); Write-Host "Practised restore wrote proof into $back. Do not delete your user profile to test this."
zsh (macOS)
drill="${TMPDIR:-/tmp}/bbb-restore-drill"; src="$drill/source"; bak="$drill/backup"; back="$drill/restored"; mkdir -p "$src" "$bak" "$back"; printf "%s\n" "bbb-restore-proof" > "$src/proof.txt"; cp "$src/proof.txt" "$bak/proof.txt"; rm "$src/proof.txt"; cp "$bak/proof.txt" "$back/proof.txt"; cat "$back/proof.txt"; echo "Practised restore wrote proof into $back. Do not delete your home folder to test this."
bash (Linux)
drill="${TMPDIR:-/tmp}/bbb-restore-drill"; src="$drill/source"; bak="$drill/backup"; back="$drill/restored"; mkdir -p "$src" "$bak" "$back"; printf "%s\n" "bbb-restore-proof" > "$src/proof.txt"; cp "$src/proof.txt" "$bak/proof.txt"; rm "$src/proof.txt"; cp "$bak/proof.txt" "$back/proof.txt"; cat "$back/proof.txt"; echo "Practised restore wrote proof into $back. Do not delete your home folder to test this."

5. Fail-Closed Workspace Policy

Never run continuous scheduled dogfood suites with blanket auto-approve (--always-approve, --yolo, --dangerously-skip-permissions) against the user profile. Always isolate dogfood runners to a fail-closed project workspace.

Checkpoint: Confirm Build Bot Better Protection Required

Paste terminal output from the restore drill or variable demo confirming protection: