CLI Reference Guide

This guide covers all command-line options and usage patterns for the Flow Test Engine CLI.

Basic Usage

# Run tests with default configuration
flow-test-engine

# Run specific test file
flow-test-engine my-test.yaml

# Run tests with specific configuration file
flow-test-engine -c my-config.yml

Commands

Main Commands

Dashboard Subcommands

Command Line Options

Configuration

Verbosity Levels

Control the amount of output during execution:

Filtering Options

Run only specific subsets of tests:

Execution Control

Swagger/OpenAPI Import

Generate test files from API specifications:

Postman Collection Import/Export

Exchange suites with Postman collections:

Other Options

Common Usage Patterns

Development and Testing

# Development with TypeScript loader
npm run dev tests/my-test.yaml -- --verbose

# Run specific test suite by name
flow-test-engine --suite "authentication"

# Run only critical priority tests
flow-test-engine --priority critical

# Run multiple priorities
flow-test-engine --priority critical,high

# Test specific functionality by tags
flow-test-engine --tag "payment,checkout"

flow-test-engine --inline-yaml "$(cat tests/sample.yaml)" --inline-base ./tests (or pipe stdin) – observe JSON execution output; dependencies referenced in the YAML are auto-included.

# Test specific nodes
flow-test-engine --node auth-tests,payment-tests

# Dry run to see execution plan
flow-test-engine --dry-run --detailed

# Silent execution (errors only)
flow-test-engine --silent

Dashboard Operations

# Install dashboard dependencies
flow-test-engine dashboard install

# Start development server
flow-test-engine dashboard dev

# Build for production
flow-test-engine dashboard build

# Preview built dashboard
flow-test-engine dashboard preview

# Build and serve
flow-test-engine dashboard serve

Import/Export Operations

# Import OpenAPI spec
flow-test-engine --swagger-import ./api-spec.yaml --swagger-output ./tests/api

# Export to Postman collection
flow-test-engine --postman-export tests/auth-test.yaml --postman-output ./auth.json

# Import Postman collection
flow-test-engine --postman-import collection.json --postman-import-output ./tests/

# Export from results with real data
flow-test-engine --postman-export-from-results results/latest.json

Configuration and Environment

# Use specific config file
flow-test-engine -c ./config/staging.yml

# Override test directory
flow-test-engine --directory ./integration-tests

# Set environment for variable resolution
flow-test-engine --environment production

# Disable log file generation
flow-test-engine --no-log

CI/CD Integration

# Silent execution for CI pipelines
flow-test-engine --silent --priority critical,high

# Run tests for specific environment
flow-test-engine --environment production --config config/prod.yml

# Generate reports without verbose output
flow-test-engine --simple --no-log

Debugging and Troubleshooting

# Maximum verbosity for debugging
flow-test-engine --verbose --suite "failing-test"

# Run single test file
flow-test-engine --directory ./tests --suite "single-test"

# Check configuration without execution
flow-test-engine --dry-run --config debug-config.yml

Import and Generation

# Import OpenAPI spec and generate tests
flow-test-engine --swagger-import openapi.json

# Import with custom output directory
flow-test-engine --swagger-import swagger.yaml --swagger-output ./tests/generated

# Import and run generated tests
flow-test-engine --swagger-import api.json && flow-test-engine --directory ./tests/imported

# Export an existing suite to Postman format
flow-test-engine --postman-export tests/auth-flows-test.yaml --postman-output ./exports/auth-flows.postman_collection.json

# Import a Postman collection as Flow tests
flow-test-engine --postman-import ./postman/create-proposal.postman_collection.json --postman-import-output ./tests/imported-postman

Configuration File Resolution

The CLI looks for configuration files in this order:

1. File specified via --config or as argument
2. flow-test.config.yml
3. flow-test.config.yaml
4. flow-test.yml
5. flow-test.yaml

If no configuration file is found, default settings are used.

Exit Codes

The CLI returns different exit codes based on execution results:

• 0: All tests passed (100% success rate)
• 1: Some tests failed (less than 100% success rate)
• 130: Process interrupted (SIGINT/Ctrl+C)
• 143: Process terminated (SIGTERM)

Environment Variables

The CLI respects these environment variables:

• FLOW_TEST_CONFIG: Default configuration file path
• FLOW_TEST_ENV: Default environment name
• FLOW_TEST_VERBOSE: Set default verbosity (true/false)

Examples by Use Case

API Testing Workflow

# 1. Import API specification
flow-test-engine --swagger-import api-docs.json --swagger-output ./tests/api

# 2. Review generated tests
ls -la ./tests/api/

# 3. Run generated tests
flow-test-engine --directory ./tests/api --verbose

# 4. Run only smoke tests
flow-test-engine --tag smoke --simple

Multi-Environment Testing

# Development environment
flow-test-engine --environment dev --config config/dev.yml --verbose

# Staging environment
flow-test-engine --environment staging --config config/staging.yml --detailed

# Production environment (silent)
flow-test-engine --environment prod --config config/prod.yml --silent

Priority-Based Execution

# Pre-deployment checks
flow-test-engine --priority critical --silent

# Full regression suite
flow-test-engine --priority critical,high,medium --detailed

# Quick smoke test
flow-test-engine --tag smoke --simple

Debugging Failed Tests

# Run failing test with maximum verbosity
flow-test-engine --suite "failing-suite" --verbose

# Check execution plan
flow-test-engine --dry-run --detailed --suite "problematic-test"

# Run with custom config for debugging
flow-test-engine --config debug.yml --verbose --no-log

Advanced Filtering

Complex Filtering Examples

# Multiple priorities
flow-test-engine --priority critical,high

# Multiple suites
flow-test-engine --suite "auth,user-mgmt,payment"

# Multiple tags
flow-test-engine --tag "smoke,api,integration"

# Combine filters (AND logic)
flow-test-engine --priority high --tag api --suite "checkout"

Node-based Filtering

# Run tests by node ID
flow-test-engine --node auth-tests,payment-tests

# Combine with other filters
flow-test-engine --node api-tests --environment staging

Output and Logging

Log Files

By default, the engine generates log files in the results/ directory. Disable with:

flow-test-engine --no-log

Report Generation

Após a execução o engine grava results/latest.json. Use o report-dashboard para visualizar em HTML.

# Iniciar dashboard interativo
npm run report:dashboard:dev

Custom Output Directory

Reports are saved to the directory specified in configuration under reporting.output_dir.

Troubleshooting

Common Issues

1. Configuration not found

   # Specify config explicitly
   flow-test-engine --config ./flow-test.config.yml

2. Tests not discovered

   # Check discovery patterns
   flow-test-engine --dry-run --detailed --directory ./tests

3. Environment variables not resolved

   # Export variables before running
   export API_KEY="your-key"
   flow-test-engine

4. Permission errors

   # Check file permissions
   ls -la flow-test.config.yml

Debug Commands

# Show execution plan
flow-test-engine --dry-run --detailed

# Show discovered tests
flow-test-engine --dry-run --verbose

# Validate configuration
flow-test-engine --config your-config.yml --dry-run

Integration Examples

npm Scripts

Add to package.json:

{
  "scripts": {
    "test": "flow-test-engine",
    "test:smoke": "flow-test-engine --tag smoke --silent",
    "test:critical": "flow-test-engine --priority critical",
    "test:verbose": "flow-test-engine --verbose",
    "test:staging": "flow-test-engine --environment staging --config config/staging.yml",
    "test:import": "flow-test-engine --swagger-import api.json"
  }
}

GitHub Actions

name: API Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      - run: npm install
      - run: npm run test:smoke
      - run: npm run test:critical

Docker Integration

# Run in Docker
docker run --rm -v $(pwd):/app flow-test --config /app/config.yml

# With environment variables
docker run --rm -v $(pwd):/app -e API_KEY=$API_KEY flow-test