Getting Started with Flow Test Engine

Flow Test Engine is a language-agnostic runner for API suites written in YAML. You can bolt it onto any repository—regardless of the tech stack—to manage HTTP flows, capture variables, and publish reports.

1. Requirements

• Node.js 16 or newer (runtime used by Flow Test Engine)
• npm 8 or newer (ships with Node.js)
• Docker Desktop (optional, only if you want the bundled httpbin mock server)

📌 Install Node.js even if your application is written in another language—Flow Test Engine runs beside your project and never touches your runtime code.

2. Quick Start with npm (local install)

Install and run Flow Test Engine locally:

# Install the engine
npm install -g flow-test-engine

# Initialize configuration and sample suites in the current directory
flow-test init

# Execute all discovered suites using the generated config
flow-test

Alternatively, you can use without global install:

# Initialize configuration with npx
npx flow-test-engine init

# Execute with npx
npx flow-test-engine

The wizard creates:

• flow-test.config.yml with discovery rules, retries, and reporting options.
• tests/ containing starter YAML suites you can edit or replace.
• results/ where execution artifacts are stored (results/latest.json).

3. Make Flow Test Part of Your Repository

For long-term use, keep Flow Test assets in their own workspace so teammates and CI can run them consistently.

mkdir flow-tests
cd flow-tests
npm init -y
npm install flow-test-engine
npx flow-test init

This produces a flow-tests/package.json similar to:

{
  "name": "flow-tests",
  "private": true,
  "scripts": {
    "test": "flow-test",
    "test:verbose": "flow-test --verbose",
    "test:dry-run": "flow-test --dry-run",
    "test:critical": "flow-test --priority critical"
  },
  "dependencies": {
    "flow-test-engine": "^1.1.1"
  }
}

A minimal suite (flow-tests/tests/my-first-test.yaml) might look like:

suite_name: "Payment API Smoke"
node_id: "payment-smoke"
base_url: "https://sandbox.my-api.com"

metadata:
  priority: "critical"
  tags: ["smoke", "payment"]
  description: "Basic smoke test for payment API"

variables:
  payment_id: "12345"
  api_token: "{{$env.API_TOKEN}}"

steps:
  - name: "Get payment"
    request:
      method: GET
      url: "/payments/{{payment_id}}"
      headers:
        Authorization: "Bearer {{api_token}}"
    assert:
      status_code: 200
      body:
        id:
          equals: "{{payment_id}}"
        status:
          in: ["CREATED", "CONFIRMED"]

Run it from anywhere in your repo:

npm --prefix flow-tests run test
# or with specific options
npm --prefix flow-tests run test:verbose

4. Everyday CLI Patterns

flow-test --dry-run --detailed              # Discover tests and show plan
flow-test --suite auth,checkout             # Run specific suites
flow-test --priority critical,high          # Focus on high-impact scenarios
flow-test --tag smoke                       # Filter by YAML tags
flow-test --config ./staging.yml            # Point to another config file
flow-test --verbose                         # Show detailed request/response data
flow-test --directory ./api-tests           # Run from specific directory
flow-test --environment staging             # Use staging environment variables

5. Integration Recipes by Ecosystem

The commands below assume you created the dedicated flow-tests/ workspace.

Node.js / TypeScript projects

1. Install the engine next to your source:

   npm install --save-dev flow-test-engine
   npx flow-test init

2. Add scripts to package.json:

   {
     "scripts": {
       "flow-test": "flow-test --config flow-test.config.yml",
       "flow-test:ci": "flow-test --config flow-test.config.yml --report json"
     }
   }

3. Run locally or in CI:

   npm run flow-test
   npm run flow-test:ci

PHP / Laravel (Composer)

1. Keep Flow Test files under flow-tests/ as described earlier.
2. Reference that workspace from composer.json:

   {
     "scripts": {
       "flow-test": "npm --prefix flow-tests run flow-test",
       "flow-test:ci": "npm --prefix flow-tests run flow-test -- --report=json"
     }
   }

3. Execute with Composer:

   composer run flow-test
   composer run flow-test:ci

4. In CI (GitHub Actions example):

   - uses: actions/setup-node@v4
     with:
       node-version: '18'
   - run: npm ci --prefix flow-tests
   - run: composer run flow-test:ci

Java / Spring (Maven)

1. Ensure Node.js is available on the build agent.
2. Add a Maven execution in pom.xml:

   <plugin>
     <groupId>org.codehaus.mojo</groupId>
     <artifactId>exec-maven-plugin</artifactId>
     <version>3.1.0</version>
     <executions>
       <execution>
         <id>flow-tests</id>
         <phase>verify</phase>
         <goals><goal>exec</goal></goals>
         <configuration>
           <workingDirectory>${project.basedir}/flow-tests</workingDirectory>
           <executable>npx</executable>
           <arguments>
             <argument>flow-test</argument>
             <argument>--config</argument>
             <argument>flow-test.config.yml</argument>
           </arguments>
         </configuration>
       </execution>
     </executions>
   </plugin>

3. Run with Maven:

   mvn verify

Python / Django

Add a Makefile (or extend an existing one) so everyone runs the same command:

# Makefile
flow-test:
	@npm --prefix flow-tests run flow-test

flow-test-ci:
	@npm --prefix flow-tests run flow-test -- --report=json

Usage:

make flow-test
make flow-test-ci

In a tox.ini you can also add:

[testenv:flow-tests]
commands = npm --prefix {toxinidir}/flow-tests run flow-test

Dart / Flutter

Create tool/flow_test.dart to bridge into Node.js:

import 'dart:io';

Future<void> main() async {
  final result = await Process.start(
    'npx',
    ['--yes', 'flow-test', '--config', 'flow-tests/flow-test.config.yml'],
    mode: ProcessStartMode.inheritStdio,
  );
  final exitCode = await result.exitCode;
  if (exitCode != 0) {
    exit(exitCode);
  }
}

Add a script to pubspec.yaml (if you use melos or simple make targets):

# pubspec.yaml
scripts:
  flow-test: dart run tool/flow_test.dart

Run:

dart run tool/flow_test.dart
# or with the scripts plugin
dart run flow-test

Pure Terminal / CI Pipelines

If your project has no build tool, keep it simple:

npm ci --prefix flow-tests             # install once in CI
npm --prefix flow-tests run flow-test  # execute suites

Or run ad hoc:

npx --yes flow-test --config flow-tests/flow-test.config.yml

If you rely on Docker for dependent services, start them before running Flow Test. The sample docker-compose.yml in the Flow Test repository spins up httpbin:

docker compose up -d httpbin
npm --prefix flow-tests run flow-test
docker compose down -v

6. Reporting and Dashboards

Every execution writes a JSON artifact at flow-tests/results/latest.json. The Astro dashboard packaged with this repository reads that file directly, so keep it in sync before starting the UI.

1. Copy (or symlink) the latest results into the dashboard data folder:

   mkdir -p report-dashboard/src/data
   cp flow-tests/results/latest.json report-dashboard/src/data/latest.json
   # or keep it synced:
   ln -sf ../flow-tests/results/latest.json report-dashboard/src/data/latest.json

   The dashboard looks for the browser-facing file at report-dashboard/src/data/latest.json, while server-side rendering also checks ../results/latest.json. If neither is present it automatically reads reporting.output_dir from your flow-test.config.* file to locate the latest report. Either path can be kept fresh in CI.
2. Launch the dashboard:

   npm install --prefix report-dashboard        # first run only
   npm run --prefix report-dashboard dev

   Open http://localhost:4321/flow-test/ (Astro’s default port). The UI reloads automatically when src/data/latest.json changes.
   • Prefer to serve it at the root path during local development? Run the command with PUBLIC_BASE_URL=/ npm run --prefix report-dashboard dev and open http://localhost:4321/ instead.
3. Generate a static bundle when you need to publish the report:

   npm run --prefix report-dashboard build

   The build lands in report-dashboard/dist/. Deploy that folder and ship the matching src/data/latest.json (or automate a copy step) so the hosted dashboard can load the latest results.

Example CI step (GitHub Actions):

- run: npm ci --prefix flow-tests
- run: npm --prefix flow-tests run flow-test
- run: npm ci --prefix report-dashboard
- run: cp flow-tests/results/latest.json report-dashboard/src/data/latest.json
- run: npm run --prefix report-dashboard build

7. Next Steps

• Review the YAML syntax reference for all available assertions.
• Explore configuration options to tailor retries, discovery, and environments.
• Check the CLI reference for every flag and filter.
• Learn how to ship HTML dashboards in the reporting guide.

With these patterns you can attach Flow Test Engine to Node, PHP, Java, Python, Dart, or any other project that can execute shell commands.