Quality Gates

Overview

AIOX implements a three-layer quality gate system that validates code at multiple checkpoints before it reaches production. Each layer has distinct responsibilities, automation levels, and enforcement mechanisms.

Design Philosophy: Shift quality left - catch issues as early as possible when they’re cheapest to fix.

Layer 1: Pre-Commit Validation

Purpose

Prevent broken code from ever being committed to the repository.

Enforcement

Local Git Hooks (.aiox-core/hooks/pre-commit) Automated: Yes (100%) Agent Responsible: @dev (Dex)

Validation Checks

Linting (ESLint)

Command: npm run lintPurpose: Enforce coding standards and catch common errorsChecks:

Code style consistency
Unused variables
Potential bugs (e.g., missing await)
Import order
Absolute vs relative imports (Constitution: Absolute Imports)

Configuration: .eslintrc.jsSeverity: BLOCK (must pass)

# Example output on failure:
❌ ESLint failed:
   src/auth/oauth.ts
     45:12  error  'token' is assigned but never used  no-unused-vars
     67:5   error  Missing return statement           consistent-return

Type Checking (TypeScript)

Command: npm run typecheckPurpose: Validate type safety across the codebaseChecks:

Type errors
Missing type definitions
Incorrect function signatures
Null/undefined handling

Configuration: tsconfig.jsonSeverity: BLOCK (must pass)

# Example output on failure:
❌ Type check failed:
   src/auth/oauth.ts:67:15 - error TS2322: Type 'string | undefined' is not assignable to type 'string'.

Unit & Integration Tests

Command: npm testPurpose: Verify functionality and prevent regressionsChecks:

All tests pass
Coverage >= previous level (no regression)
No flaky tests (consistent results)

Framework: Jest, Vitest, or project-specificSeverity: BLOCK (must pass)

# Example output on failure:
❌ Tests failed:
   FAIL  src/auth/oauth.test.ts
     ● OAuthProvider › exchangeToken › should handle expired codes
       Expected status 400, received 500

Build Verification

Command: npm run buildPurpose: Ensure code compiles and builds successfullyChecks:

TypeScript compilation
Asset bundling
Tree-shaking optimization
No build warnings

Severity: BLOCK (must pass)

# Example output on failure:
❌ Build failed:
   ERROR in src/auth/oauth.ts
   Module not found: Error: Can't resolve '@/utils/crypto'

Metrics Collection

Layer 1 runs are tracked in .aiox/data/quality-metrics.json:

{
  "layers": {
    "layer1": {
      "passRate": 0.95,
      "avgTimeMs": 12000,
      "totalRuns": 247,
      "lastRun": "2026-03-05T14:30:00Z"
    }
  }
}

Bypass (Emergency Only)

# NOT RECOMMENDED - Only for critical hotfixes
git commit --no-verify -m "hotfix: emergency patch"

Bypassing Layer 1 requires human approval and must be documented in the commit message with justification.

Layer 2: PR Automation

Purpose

Automated review and validation in the CI/CD pipeline before human review.

Enforcement

GitHub Actions (.github/workflows/) CodeRabbit AI Review Automated: Yes (100%) Agent Responsible: @devops (Felix) + Quinn (@qa)

Validation Checks

CI/CD Pipeline
CodeRabbit Review
Quinn Review

GitHub Actions Workflow:

name: Quality Gate
on: [pull_request]

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
      
      # Re-run all Layer 1 checks in CI environment
      - run: npm run lint
      - run: npm run typecheck
      - run: npm test
      - run: npm run build
      
      # Additional CI-only checks
      - run: npm run test:e2e
      - run: npm run security:scan
      - run: npm run lighthouse  # If UI changes

Multi-Environment Testing:

Node 18, 20, 22
Ubuntu, macOS, Windows
Different dependency versions

Automated AI Code Review:CodeRabbit analyzes:

Code quality and best practices
Security vulnerabilities
Performance issues
Test coverage gaps
Documentation completeness

Severity Levels:

🔴 CRITICAL: Must fix before merge
🟠 HIGH: Should fix (human can waive)
🟡 MEDIUM: Nice to have
🟢 LOW: Informational

Example Finding:

finding:
  severity: HIGH
  category: security
  file: src/auth/oauth.ts
  line: 67
  description: "Token not validated for expiration before use"
  suggestion: "Add expiration check: if (token.exp < Date.now()) throw new TokenExpiredError()"

QA Agent Review Task:Quinn (@qa) performs:

Functional verification against story criteria
NFR compliance check
Test quality assessment
Documentation review

Verdict Format:

verdict: PASS | CONCERNS | FAIL | WAIVED
confidence: 0.0-1.0
findings:
  - type: critical | major | minor
    description: "Issue description"
    severity: HIGH | MEDIUM | LOW

See Development Cycle: QA Review for details.

Auto-Catch Rate

Layer 2 tracks how many issues are caught automatically vs. requiring human intervention:

{
  "layers": {
    "layer2": {
      "passRate": 0.88,
      "avgTimeMs": 180000,
      "totalRuns": 156,
      "autoCatchRate": 0.73,
      "coderabbit": {
        "active": true,
        "findingsCount": 342,
        "severityBreakdown": {
          "critical": 12,
          "high": 45,
          "medium": 128,
          "low": 157
        }
      },
      "quinn": {
        "findingsCount": 89,
        "topCategories": [
          "test-coverage",
          "documentation",
          "error-handling"
        ]
      }
    }
  }
}

Auto-Catch Rate = Issues detected by automation / Total issues (automation + human) Target: ≥ 70% (current: 73%)

Layer 3: Human Review

Purpose

Final architectural validation and business logic review by human experts.

Enforcement

GitHub PR Review (required approvals) Automated: No Responsible: Human reviewers (architects, senior developers)

Review Criteria

Architectural Alignment

Reviewers: @architect (Aria) + human architectsChecks:

Design aligns with system architecture
No architectural anti-patterns introduced
Dependency directions correct
Module boundaries respected
Technical debt documented if introduced

Questions to Ask:

Does this change fit our architectural vision?
Are there simpler alternatives?
What are the long-term maintenance implications?

Business Logic Validation

Reviewers: Product team + domain expertsChecks:

Implementation matches business requirements
Edge cases align with business rules
User experience considerations addressed
Regulatory/compliance requirements met

Questions to Ask:

Does this solve the actual user problem?
Are there business scenarios not covered?
What happens if this fails in production?

Security Review

Reviewers: Security team (for sensitive changes)Checks:

No hardcoded secrets or credentials
Input validation on all user data
Authentication/authorization correct
Data encryption where required
Audit logging for sensitive operations

Triggered for:

Authentication/authorization changes
Database schema changes
External API integrations
Payment processing

Knowledge Transfer

Reviewers: Team members who will maintain the codeChecks:

Code is understandable to team
Documentation explains “why” not just “what”
Complex logic has explanatory comments
Runbook updated if operational changes

Questions to Ask:

Can someone else debug this at 2am?
Is the reasoning behind decisions documented?
Are there gotchas that need explanation?

Approval Workflow

Required Approvals: Configurable (default: 1 for standard changes, 2 for architectural) Review SLA:

Standard changes: 24 hours
Urgent hotfixes: 4 hours
Architectural changes: 48 hours

Metrics Collection

{
  "layers": {
    "layer3": {
      "passRate": 0.92,
      "avgTimeMs": 86400000,
      "totalRuns": 134,
      "lastRun": "2026-03-05T09:15:00Z"
    }
  }
}

Quality Metrics Dashboard

Overall Quality Trends

{
  "trends": {
    "passRates": [
      { "date": "2026-03-01", "value": 0.89 },
      { "date": "2026-03-02", "value": 0.91 },
      { "date": "2026-03-03", "value": 0.90 },
      { "date": "2026-03-04", "value": 0.93 },
      { "date": "2026-03-05", "value": 0.92 }
    ],
    "autoCatchRate": [
      { "date": "2026-03-01", "value": 0.68 },
      { "date": "2026-03-05", "value": 0.73 }
    ]
  }
}

Viewing Metrics

# Show current quality metrics
aiox quality-metrics --summary

# Detailed breakdown by layer
aiox quality-metrics --layer=1

# Trend analysis
aiox quality-metrics --trends --days=30

# Export to CSV
aiox quality-metrics --export=csv > metrics.csv

Quality Gate Schema

All metrics conform to the schema defined in .aiox-core/quality/schemas/quality-metrics.schema.json:

Schema Overview

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Quality Metrics",
  "type": "object",
  "required": ["version", "lastUpdated", "layers", "trends", "history"],
  "properties": {
    "version": { "const": "1.0" },
    "lastUpdated": { "type": "string", "format": "date-time" },
    "retentionDays": { "type": "integer", "default": 30 },
    "layers": {
      "layer1": { "$ref": "#/definitions/layerMetrics" },
      "layer2": { "$ref": "#/definitions/layer2Metrics" },
      "layer3": { "$ref": "#/definitions/layerMetrics" }
    },
    "trends": {
      "autoCatchRate": { "type": "array" },
      "passRates": { "type": "array" }
    },
    "history": { "type": "array" }
  }
}

Metrics Collector API

Programmatic access to quality metrics:

const { MetricsCollector } = require('.aiox-core/quality/metrics-collector');

const collector = new MetricsCollector({
  dataFile: '.aiox/data/quality-metrics.json',
  retentionDays: 30
});

// Record a Layer 1 run
await collector.recordPreCommit({
  passed: true,
  durationMs: 12000,
  findingsCount: 0
});

// Record a Layer 2 PR review
await collector.recordPRReview({
  passed: true,
  durationMs: 180000,
  findingsCount: 3,
  coderabbit: {
    findingsCount: 2,
    severityBreakdown: { critical: 0, high: 0, medium: 1, low: 1 }
  },
  quinn: {
    findingsCount: 1,
    topCategories: ['documentation']
  }
});

// Get current metrics
const metrics = await collector.getMetrics();
console.log(metrics.layers.layer1.passRate);

// Cleanup old history
const removed = await collector.cleanup();
console.log(`Removed ${removed} old records`);

See .aiox-core/quality/metrics-collector.js for full API documentation.

Constitution Enforcement

Quality gates enforce Constitution principles:

Principle	Layer	Enforcement
CLI First	L1, L2	WARN if UI created before CLI functional
Agent Authority	All	BLOCK if @dev tries to push (only @devops)
Story-Driven	L1	BLOCK if no valid story associated
No Invention	L2 (Quinn)	BLOCK if spec contains invented features
Quality First	All	BLOCK on any gate failure
Absolute Imports	L1 (ESLint)	ERROR on relative imports

Best Practices

Run Locally Before Push

Always run quality checks locally before pushing:

# Manual pre-push check
npm run lint && npm run typecheck && npm test && npm run build

# Or use the pre-push task
*pre-push-quality-gate

This catches issues before they trigger CI failures.

Incremental Fixes

Don’t accumulate quality debt:

Fix linting issues as you code (use IDE integration)
Write tests alongside implementation (TDD)
Address CodeRabbit findings immediately
Don’t defer minor issues to “later”

Meaningful Test Coverage

Quality over quantity:Good:

test('OAuth token exchange handles expired codes', async () => {
  const expiredCode = 'expired-code-123';
  await expect(oauth.exchangeToken(expiredCode))
    .rejects.toThrow(TokenExpiredError);
});

Bad (False Coverage):

test('OAuth token exchange works', async () => {
  const result = await oauth.exchangeToken('code');
  expect(result).toBeDefined(); // Meaningless assertion
});

Review Your Own PRs

Before requesting review:

Review your own diff on GitHub
Check for:
- Debug statements left in
- Commented-out code
- TODO comments
- Accidental file inclusions
Add PR description with context
Link to story: “Closes #123”

Troubleshooting

Issue: Layer 1 hook not runningCause: Git hooks not installedResolution:

# Reinstall hooks
npm run prepare

# Verify hook exists
ls -la .git/hooks/pre-commit

Issue: CI passes but local tests failCause: Dependency version mismatch or environment differencesResolution:

# Sync dependencies
rm -rf node_modules package-lock.json
npm install

# Clear test cache
npm run test -- --clearCache

# Check Node version matches CI
node --version  # Should match .github/workflows/

Issue: CodeRabbit findings overwhelmingResolution:

Address CRITICAL and HIGH severity first
Group similar findings and fix in batch
Request human review to waive non-critical items
Consider adding CodeRabbit config to tune sensitivity

Metrics Retention

History is retained for 30 days by default (configurable):

// Automatic cleanup on each recordRun()
// Removes entries older than retentionDays

// Manual cleanup
const collector = new MetricsCollector({ retentionDays: 30 });
const removed = await collector.cleanup();
console.log(`Cleaned up ${removed} old records`);

Next Steps

Development Cycle

See how quality gates integrate into the development workflow

Constitution

Review the principles enforced by quality gates

Agent: QA

Learn about Quinn’s review methodology

Metrics Collector

API documentation for programmatic metrics access

⌘I

Get Started

Core Concepts

Agents

CLI Reference

Workflows

Advanced

Overview

Layer 1: Pre-Commit Validation

Purpose

Enforcement

Validation Checks

Metrics Collection

Bypass (Emergency Only)

Layer 2: PR Automation

Purpose

Enforcement

Validation Checks

Auto-Catch Rate

Layer 3: Human Review

Purpose

Enforcement

Review Criteria

Approval Workflow

Metrics Collection

Quality Metrics Dashboard

Overall Quality Trends

Viewing Metrics

Quality Gate Schema

Metrics Collector API

Constitution Enforcement

Best Practices

Troubleshooting

Metrics Retention

Next Steps

Development Cycle

Constitution

Agent: QA

Metrics Collector

​Overview

​Layer 1: Pre-Commit Validation

​Purpose

​Enforcement

​Validation Checks

​Metrics Collection

​Bypass (Emergency Only)

​Layer 2: PR Automation

​Purpose

​Enforcement

​Validation Checks

​Auto-Catch Rate

​Layer 3: Human Review

​Purpose

​Enforcement

​Review Criteria

​Approval Workflow

​Metrics Collection

​Quality Metrics Dashboard

​Overall Quality Trends

​Viewing Metrics

​Quality Gate Schema

​Metrics Collector API

​Constitution Enforcement

​Best Practices

​Troubleshooting

​Metrics Retention

​Next Steps

Development Cycle

Constitution

Agent: QA

Metrics Collector

Overview

Layer 1: Pre-Commit Validation

Purpose

Enforcement

Validation Checks

Metrics Collection

Bypass (Emergency Only)

Layer 2: PR Automation

Purpose

Enforcement

Validation Checks

Auto-Catch Rate

Layer 3: Human Review

Purpose

Enforcement

Review Criteria

Approval Workflow

Metrics Collection

Quality Metrics Dashboard

Overall Quality Trends

Viewing Metrics

Quality Gate Schema

Metrics Collector API

Constitution Enforcement

Best Practices

Troubleshooting

Metrics Retention

Next Steps