How to Gate CI/CD on Agent Readiness: A Practical Glintbase CLI Tutorial

Software teams have spent decades learning to block deployments on failing tests.

A unit test fails — the PR cannot merge. An integration test fails — the deployment stops. A type check fails — the build is rejected.

The principle is simple: if you can measure a quality property, you can gate on it.

In 2026, we can measure how agent-ready your documentation is. And that means you can gate on it.

This tutorial walks through setting up the Glintbase CLI as a CI/CD quality gate — a concrete, working pipeline configuration that checks your documentation's Agent Readiness Score on every release and blocks the deployment if the score falls below your threshold.

Why Documentation Quality Belongs in CI/CD

The objection we hear most often is: "Documentation is qualitative. You cannot automate quality checks."

This is true of prose quality. It is not true of machine operability.

The properties that determine whether an AI agent can successfully traverse your documentation — code sample validity, entry point clarity, structural coherence, prerequisite explicitness — are all measurable. They are not a matter of opinion. Either a code sample executes against the current SDK or it does not. Either an llms.txt exists or it does not. Either a prerequisite is stated or it is missing.

These are binary and continuous facts that a scanner can verify.

The problem is not that documentation quality cannot be measured. The problem is that nobody was running the measurements systematically, on a schedule, tied to a deployment pipeline.

That is what Glintbase CI/CD integration solves.

What the Gate Checks

The Glintbase scanner runs the following checks against your documentation URL:

1. llms.txt Presence and Validity Verifies that /llms.txt exists and is parseable as a structured index file. Checks for required fields: product description, canonical URL, key documentation links.

2. Code Sample Validity Extracts all fenced code blocks from documentation pages and executes them in a sandboxed environment against the declared SDK version. Returns a pass rate (0–100%).

3. OpenAPI Specification Sync If an OpenAPI spec URL is configured, compares the spec against the live API — checking that documented endpoints exist, that parameter schemas match, and that response schemas are current.

4. Link Integrity Crawls internal documentation links and checks for 404s, redirect chains deeper than 2 hops, and circular references.

5. Prerequisite Coverage Analyzes getting-started guides and tutorial pages for the presence of prerequisite sections. Flags guides that require credentials or dependencies without declaring them explicitly.

6. ARS Composite Score Produces the weighted composite Agent Readiness Score from all sub-dimension measurements.

Setting Up the CLI

Installation

npm install -g @glintbase/cli

Verify installation:

glintbase --version
# glintbase-cli/0.9.2 linux-x64 node-v22.4.0

Authentication

glintbase auth login
# Opens browser for OAuth flow
# Paste the token from your Glintbase dashboard:
# Enter your API token: glint_sk_...
# ✓ Authenticated as victor@glintbase.dev

Run a Local Scan

Before setting up CI/CD, run a baseline scan against your documentation to understand your starting score:

glintbase scan https://docs.yourplatform.dev

Output:

🔍 Scanning https://docs.yourplatform.dev ...
   ✓ llms.txt found and valid
   ✓ OpenAPI spec found at /openapi.json
   ✗ Code sample validity: 61/100 (37 of 61 samples pass)
   ✓ Link integrity: 98.2% (4 broken links found)
   ✗ Prerequisite coverage: 44/100 (6 guides missing prerequisite sections)
   ✓ Entry point clarity: 78/100
   ✓ Structural coherence: 71/100

   Agent Readiness Score: 58 / 100
   
   Grade: C+  |  Percentile: 47th (ARS 2026 cohort)
   
   Full report: https://app.glintbase.dev/scans/sc_abc123
   Recommendations: https://app.glintbase.dev/scans/sc_abc123/recommendations

Configuring the CI/CD Gate

glintbase.config.json

Create a configuration file at the root of your repository:

{
  "scan": {
    "url": "https://docs.yourplatform.dev",
    "timeout": 300,
    "checks": {
      "llms_txt": true,
      "code_samples": true,
      "openapi_sync": true,
      "link_integrity": true,
      "prerequisites": true
    }
  },
  "gate": {
    "min_ars_score": 65,
    "min_code_sample_validity": 80,
    "block_on_broken_llms_txt": true,
    "warn_on_broken_links": true,
    "fail_on_broken_links": false
  },
  "notifications": {
    "slack_webhook": "${GLINTBASE_SLACK_WEBHOOK}",
    "on_score_drop": true,
    "on_gate_failure": true
  }
}

Key configuration options:

• min_ars_score: The minimum acceptable composite ARS. The pipeline fails if the score falls below this number. We recommend starting at 50 and increasing by 5 points each quarter.
• min_code_sample_validity: Minimum percentage of code samples that must pass execution. 80% is a reasonable starting target.
• block_on_broken_llms_txt: Hard failure if llms.txt is missing or malformed.
• warn_on_broken_links / fail_on_broken_links: Start with warn, graduate to fail once your baseline is clean.

GitHub Actions Integration

Create .github/workflows/agent-readiness.yml:

name: Agent Readiness Gate

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    # Also run daily at 9am UTC to catch drift from external API changes
    - cron: '0 9 * * *'

jobs:
  agent-readiness-check:
    name: Check Agent Readiness Score
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Install Glintbase CLI
        run: npm install -g @glintbase/cli@latest

      - name: Authenticate
        run: glintbase auth login --token ${{ secrets.GLINTBASE_API_TOKEN }}

      - name: Run Agent Readiness Scan
        id: ars_scan
        run: |
          glintbase scan \
            --config glintbase.config.json \
            --output json \
            --output-file ars-results.json
        continue-on-error: true

      - name: Upload ARS Report
        uses: actions/upload-artifact@v4
        with:
          name: agent-readiness-report
          path: ars-results.json

      - name: Post PR Comment
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const results = JSON.parse(fs.readFileSync('ars-results.json', 'utf8'));
            const emoji = results.ars_score >= 65 ? '✅' : '❌';
            const body = `## ${emoji} Agent Readiness Score: ${results.ars_score}/100
            
            | Dimension | Score |
            |---|---|
            | Entry Point Clarity | ${results.dimensions.entry_point_clarity}/100 |
            | Code Sample Validity | ${results.dimensions.code_sample_validity}% passing |
            | Structural Coherence | ${results.dimensions.structural_coherence}/100 |
            | Prerequisite Coverage | ${results.dimensions.prerequisite_coverage}/100 |
            
            [View full report](${results.report_url})`;
            
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body
            });

      - name: Enforce Gate
        run: glintbase gate --results ars-results.json --config glintbase.config.json

GitLab CI Integration

# .gitlab-ci.yml

agent-readiness:
  stage: quality
  image: node:22-slim
  script:
    - npm install -g @glintbase/cli@latest
    - glintbase auth login --token $GLINTBASE_API_TOKEN
    - glintbase scan --config glintbase.config.json --output json --output-file ars-results.json
    - glintbase gate --results ars-results.json --config glintbase.config.json
  artifacts:
    reports:
      dotenv: ars-results.json
    paths:
      - ars-results.json
    expire_in: 30 days
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "main"'

Reading the Gate Output

When the gate runs, it produces a clear pass/fail signal with actionable context:

╔══════════════════════════════════════════════════╗
║  Glintbase Agent Readiness Gate — FAILED         ║
╚══════════════════════════════════════════════════╝

Score:     52 / 100  (threshold: 65)
Status:    ❌ GATE FAILED — score below minimum threshold

Blocking Issues:
  ✗ Code sample validity: 61%  (minimum: 80%)
    → 24 of 61 samples fail execution against current SDK
    → Most failures: /docs/authentication (8 failures)
    → Fix guide: https://app.glintbase.dev/scans/sc_abc123/fixes

  ✗ ARS Score: 52  (minimum: 65)
    → Weighted composite below threshold

Warnings (non-blocking):
  ⚠ 4 broken internal links found
  ⚠ 3 guides missing prerequisite sections

Full report: https://app.glintbase.dev/scans/sc_abc123

Run locally: glintbase scan https://docs.yourplatform.dev --fix

Recommended Rollout Strategy

Do not start with a gate threshold that your platform currently fails.

Week 1–2: Run the scanner in report-only mode (no gate). Establish your baseline ARS and understand where your failures are.

Week 3–4: Set min_ars_score to your current score minus 5 points. This immediately blocks any regression below your baseline without requiring you to fix existing issues first.

Monthly: Review the scan report. Fix the highest-impact issues. Raise the threshold by 5 points.

Quarterly target: Reach 70+ ARS by the end of Q4. This puts you in the top 20th percentile of the ARS 2026 cohort.

The Daily Drift Check

The GitHub Actions configuration above includes a scheduled cron run at 9am UTC daily.

This matters because documentation can drift without anyone on your team touching it.

External API dependencies change. Third-party services update their authentication flows. SDK methods are deprecated upstream. None of these changes trigger a commit to your repository — but they may break code samples or invalidate documented prerequisites.

The daily drift check catches these regressions at the source, before a developer encounters them through an AI agent integration failure.

It is, in essence, a continuous documentation health monitor.

What This Changes

When agent readiness becomes a first-class CI/CD concern, something shifts in how documentation is maintained.

Documentation starts being discussed in engineering standups. Broken code samples get filed as bugs, not backlog notes. The llms.txt gets updated when the product changes. Prerequisites get added to guides before merge.

The cultural shift is the real outcome. The gate is the mechanism that produces it.

Join the Glintbase waitlist and be first to access the CLI and API when we open the beta → glintbase.dev