Skip to main content

​
Your First Run

This guide walks through the core Flightline workflow: initialize a project, generate synthetic test data, and run evaluations.
Flightline is in early access. The exact CLI output may differ from what’s shown here. Contact the team for the latest documentation.

​
Prerequisites

  • Flightline CLI installed
  • A Pydantic schema or sample JSON file to test against

​
Step 1: Initialize Your Project

Navigate to your project directory and run the init wizard: 
flightline init
Flightline scans your project and guides you through setup: 
$ flightline init

> Scanning project... found 3 Pydantic models.
> Which do you want to test? [LoanApp, UserProfile, Transaction]
> Selected: LoanApp
This creates a flightline.yaml configuration file in your project root.

​
Step 2: Generate Synthetic Data

With your schema configured, generate edge-case test scenarios: 
flightline generate
Flightline analyzes your schema and creates synthetic records: 
$ flightline generate

> Analyzing LoanApp schema...
> Identified 5 Key Dimensions: Credit, Income, LoanType, Assets, DTI.
> Generating 20 edge-case scenarios...
> [OK] Created 20 synthetic records in /tests/data/

​
Step 3: Run Evaluations

Execute your prompt against the generated test data: 
flightline run
Flightline runs your LLM prompt against each synthetic record and validates the output: 
$ flightline run

> Running 'Financial Summary' prompt against 20 records...
> ❌ FAILURE: Scenario #4 (Negative Income).
> Expected: "Applicant rejected."
> Received: "Applicant approved with $0 income."

​
What’s Next?

The Cartographer

Learn how Flightline maps your schema’s edge cases.

The Fabricator

Understand synthetic data generation.