Skip to main content

​
flightline.yaml

The flightline.yaml file sits at the root of your repository and defines how Flightline interacts with your codebase. It is used to configure scanning paths, risk thresholds, and custom evaluation rubrics.

​
Initializing Configuration

You can automatically generate a starting configuration by running the scaffold command: 
flightline scaffold
This command scans your codebase for AI operations and proposes a “flight plan” based on what it finds.

​
Configuration Structure

A typical flightline.yaml includes the following sections: 
# flightline.yaml
project_name: "my-ai-service"

monitoring:
  # The risk tier at which a failure blocks your CI pipeline
  fail_on: "critical"
  
  # Directories to scan for AI operations during discovery
  scan_paths:
    - "./src"
    - "./app"
    
  # Patterns to ignore during scanning
  ignore:
    - "**/node_modules/**"
    - "**/tests/**"

operations:
  - id: "op_12345"
    name: "generate_summary"
    path: "app/services/summarizer.py"
    risk_tier: "high"
    # custom_rubric: "./rubrics/summary_quality.md"

​
Section Details

​
monitoring

This section controls how Flightline behaves in your CI/CD pipeline.
  • fail_on: Can be set to critical, high, medium, or low. Any failure at or above this tier will cause flightline check to return a non-zero exit code.
  • scan_paths: A list of directories that flightline discover will look into.
  • ignore: Standard glob patterns for files or directories that should be skipped.

​
operations

This section maps specific AI call sites in your code to their metadata. These are usually populated automatically by the scaffold or discover commands.
  • id: A unique identifier for the AI operation.
  • risk_tier: The assigned risk level for this operation.
  • custom_rubric: Path to a Markdown file defining specific qualitative standards for this operation.

​
Custom Rubrics

For qualitative Tier 2 checks, you can provide a Markdown file that defines the grading criteria. This allows you to tailor the Intelligence Layer’s reasoning to your specific brand voice or business logic. 
# Summary Quality Rubric

1. The summary must be under 3 sentences.
2. It must never mention the customer's internal ID.
3. The tone should be professional and concise.

Ship-Readiness

See how these settings impact your ship-readiness scores.