Configuration Format Documentation

This document provides comprehensive documentation for the workflow comparison configuration format.

Table of Contents

• Overview
• Configuration File Format
• Top-Level Fields
• Cost Configuration
• Similarity Groups
• Ignore Rules
• Parameter Comparison Rules
• Exemptions
• Connection Rules
• Output Configuration
• Examples

Overview

Configuration files can be written in either YAML or JSON format. YAML is recommended for readability and easier maintenance. The configuration controls:

• Cost weights for different types of graph edits
• Similarity groups to treat similar node types as equivalent
• Ignore rules to exclude certain nodes or parameters
• Parameter comparison rules for flexible matching
• Exemptions for optional nodes
• Output formatting preferences

Configuration File Format

YAML Example

version: "1.0"
name: "my-config"
description: "My custom configuration"

costs:
  nodes:
    insertion: 10.0
    deletion: 10.0
    # ... more costs

similarity_groups:
  triggers:
    - "n8n-nodes-base.webhook"
    - "n8n-nodes-base.manualTrigger"

ignore:
  node_types:
    - "n8n-nodes-base.stickyNote"

parameter_comparison:
  numeric_tolerance:
    - parameter: "options.temperature"
      tolerance: 0.1

output:
  max_edits: 15

JSON Example

{
  "version": "1.0",
  "name": "my-config",
  "description": "My custom configuration",
  "costs": {
    "nodes": {
      "insertion": 10.0,
      "deletion": 10.0
    }
  },
  "similarity_groups": {
    "triggers": [
      "n8n-nodes-base.webhook",
      "n8n-nodes-base.manualTrigger"
    ]
  }
}

Top-Level Fields

version (string, required)

Configuration format version. Currently "1.0".

version: "1.0"

name (string, optional)

A unique identifier for this configuration.

name: "my-custom-config"

description (string, optional)

Human-readable description of what this configuration does.

description: "Strict comparison for production workflows"

Cost Configuration

The costs section defines penalties for different graph edit operations. These costs directly impact the similarity score.

Structure

costs:
  nodes:
    insertion: <float>
    deletion: <float>
    substitution:
      same_type: <float>
      similar_type: <float>
      different_type: <float>
      trigger_mismatch: <float>

  edges:
    insertion: <float>
    deletion: <float>
    substitution: <float>

  parameters:
    mismatch_weight: <float>
    nested_weight: <float>

Node Costs

costs.nodes.insertion (float, default: 10.0)

Cost penalty when a node exists in the ground truth but is missing from the generated workflow.

Use case: Set higher for stricter matching (e.g., 15.0), lower for lenient matching (e.g., 5.0).

costs:
  nodes:
    insertion: 10.0

costs.nodes.deletion (float, default: 10.0)

Cost penalty when a node exists in the generated workflow but not in the ground truth.

Use case: Set higher to penalize extra nodes more severely.

costs:
  nodes:
    deletion: 15.0

costs.nodes.substitution.same_type (float, default: 1.0)

Cost when two nodes have the same type but different parameters.

Use case:

• Low values (0.5-1.0): Allow parameter variations
• High values (2.0-5.0): Require exact parameter matches

costs:
  nodes:
    substitution:
      same_type: 1.0

costs.nodes.substitution.similar_type (float, default: 5.0)

Cost when two nodes are in the same similarity group (see Similarity Groups).

Example: Replacing lmChatOpenAi with lmChatAnthropic (both are LLMs).

costs:
  nodes:
    substitution:
      similar_type: 5.0

costs.nodes.substitution.different_type (float, default: 15.0)

Cost when replacing a node with a completely different type.

Example: Replacing httpRequest with webhook.

costs:
  nodes:
    substitution:
      different_type: 15.0

costs.nodes.substitution.trigger_mismatch (float, default: 50.0)

Special high-cost penalty for trigger node mismatches. Triggers are critical to workflow functionality.

Use case: Keep this high (50.0-100.0) to ensure trigger correctness.

costs:
  nodes:
    substitution:
      trigger_mismatch: 50.0

Edge Costs

costs.edges.insertion (float, default: 5.0)

Cost for a missing connection between nodes.

costs:
  edges:
    insertion: 5.0

costs.edges.deletion (float, default: 5.0)

Cost for an extra connection that shouldn't exist.

costs:
  edges:
    deletion: 5.0

costs.edges.substitution (float, default: 3.0)

Cost for changing the type or properties of a connection.

costs:
  edges:
    substitution: 3.0

Parameter Costs

costs.parameters.mismatch_weight (float, default: 0.5)

Weight multiplier for parameter mismatches within a node.

Formula: parameter_cost = base_cost * mismatch_weight * num_mismatches

costs:
  parameters:
    mismatch_weight: 0.5

costs.parameters.nested_weight (float, default: 0.3)

Weight multiplier for nested/deep parameter differences.

Use case: Set lower to be more forgiving about deep configuration differences.

costs:
  parameters:
    nested_weight: 0.3

Similarity Groups

Similarity groups define sets of node types that should be considered "similar" rather than "different" when substituted. Nodes within the same group incur the similar_type cost instead of different_type.

Structure

similarity_groups:
  <group_name>:
    - "<node_type_1>"
    - "<node_type_2>"
    - "<node_type_3>"

Example

similarity_groups:
  triggers:
    - "n8n-nodes-base.webhook"
    - "n8n-nodes-base.manualTrigger"
    - "n8n-nodes-base.scheduleTrigger"

  ai_llms:
    - "@n8n/n8n-nodes-langchain.lmChatOpenAi"
    - "@n8n/n8n-nodes-langchain.lmChatAnthropic"
    - "@n8n/n8n-nodes-langchain.lmChatOllama"
    - "@n8n/n8n-nodes-langchain.lmChatMistralCloud"

  http_requests:
    - "n8n-nodes-base.httpRequest"
    - "@n8n/n8n-nodes-langchain.toolHttpRequest"

Common Similarity Groups

AI Agents

ai_agents:
  - "n8n-nodes-langchain.agent"
  - "@n8n/n8n-nodes-langchain.agent"
  - "n8n-nodes-langchain.basicAgent"

AI Tools

ai_tools:
  - "@n8n/n8n-nodes-langchain.toolHttpRequest"
  - "@n8n/n8n-nodes-langchain.toolCalculator"
  - "@n8n/n8n-nodes-langchain.toolCode"
  - "@n8n/n8n-nodes-langchain.toolWorkflow"

Ignore Rules

Ignore rules allow you to exclude certain nodes or parameters from comparison. This is useful for:

• UI-only elements that don't affect workflow execution
• Metadata fields like IDs and positions
• Parameters that vary legitimately across implementations

Structure

ignore:
  node_types: [...]
  nodes: [...]
  global_parameters: [...]
  node_type_parameters: {...}
  parameter_paths: [...]

ignore.node_types (list of strings)

Completely ignore nodes of specific types.

Use case: Ignore decorative nodes like sticky notes.

ignore:
  node_types:
    - "n8n-nodes-base.stickyNote"
    - "n8n-nodes-base.comment"

ignore.nodes (list of objects)

Flexible rules for ignoring nodes based on name patterns or other criteria.

Structure:

ignore:
  nodes:
    - pattern: "<regex_pattern>"
      reason: "Why this is ignored"
    - name: "<exact_node_name>"
      reason: "Why this is ignored"
    - node_type: "<node_type>"
      reason: "Why this is ignored"

Example:

ignore:
  nodes:
    - pattern: "^Temp.*"
      reason: "Temporary debugging nodes"
    - name: "Development Only"
      reason: "Used only in development"

ignore.global_parameters (list of strings)

Parameter names to ignore across all node types.

Common use case: Ignore UI-specific metadata.

ignore:
  global_parameters:
    - "position"
    - "id"
    - "notes"
    - "notesInFlow"
    - "color"
    - "disabled"

ignore.node_type_parameters (object)

Parameters to ignore for specific node types.

Structure:

ignore:
  node_type_parameters:
    "<node_type>":
      - "<parameter_path_1>"
      - "<parameter_path_2>"

Example:

ignore:
  node_type_parameters:
    "@n8n/n8n-nodes-langchain.agent":
      - "options.systemMessage"  # Allow different prompts
      - "options.maxIterations"   # Allow iteration variance

    "n8n-nodes-base.httpRequest":
      - "options.timeout"  # Timeout can vary by environment

ignore.parameter_paths (list of strings)

Ignore parameters using path patterns. Supports wildcards:

• * - matches any single path segment
• ** - matches any number of path segments

Example:

ignore:
  parameter_paths:
    - "options.*.timeout"      # Ignore timeout in any option
    - "**.temperature"         # Ignore temperature at any nesting level
    - "options.advanced.**"    # Ignore all advanced options

Parameter Comparison Rules

Parameter comparison rules allow for flexible matching of specific parameters, such as numeric tolerance or semantic similarity.

Structure

parameter_comparison:
  fuzzy_match: [...]
  numeric_tolerance: [...]

Fuzzy Match Rules

For semantic or approximate text matching.

Structure:

parameter_comparison:
  fuzzy_match:
    - parameter: "<parameter_path>"
      type: "semantic"
      threshold: <float>
      cost_if_below: <float>
      options:
        <key>: <value>

Example:

parameter_comparison:
  fuzzy_match:
    - parameter: "options.systemMessage"
      type: "semantic"
      threshold: 0.8
      cost_if_below: 3.0
      options:
        model: "sentence-transformers"

Numeric Tolerance Rules

For numeric parameters that should be "close enough" rather than exact.

Structure:

parameter_comparison:
  numeric_tolerance:
    - parameter: "<parameter_path>"
      tolerance: <float>
      cost_if_exceeded: <float>

Example:

parameter_comparison:
  numeric_tolerance:
    - parameter: "options.temperature"
      tolerance: 0.1
      cost_if_exceeded: 2.0

    - parameter: "options.maxTokens"
      tolerance: 100
      cost_if_exceeded: 1.0

    - parameter: "options.topP"
      tolerance: 0.05
      cost_if_exceeded: 1.5

How it works:

• If |value1 - value2| <= tolerance, parameters are considered equal (no cost)
• If |value1 - value2| > tolerance, cost_if_exceeded is added to the edit cost

Wildcard Support

Parameter paths support wildcards:

parameter_comparison:
  numeric_tolerance:
    - parameter: "options.*.temperature"
      tolerance: 0.1
      cost_if_exceeded: 2.0

This applies to options.llm.temperature, options.model.temperature, etc.

Exemptions

Exemptions reduce penalties for certain nodes that are optional or conditionally required.

Structure

exemptions:
  optional_in_generated: [...]
  optional_in_ground_truth: [...]

exemptions.optional_in_generated (list of objects)

Nodes that can be missing from the generated workflow without full penalty.

Use case: Ground truth has optional nodes that aren't critical.

Structure:

exemptions:
  optional_in_generated:
    - name_pattern: "<regex>"
      penalty: <float>
      reason: "Why this is optional"
    - node_type: "<node_type>"
      penalty: <float>
      when:
        <condition_key>: <condition_value>

Example:

exemptions:
  optional_in_generated:
    - node_type: "@n8n/n8n-nodes-langchain.memoryBufferWindow"
      penalty: 2.0
      reason: "Memory is optional for simple workflows"

    - name_pattern: ".*Debug.*"
      penalty: 1.0
      reason: "Debug nodes are optional in production"

exemptions.optional_in_ground_truth (list of objects)

Nodes that can exist in the generated workflow as extras without full penalty.

Use case: Generated workflow includes helpful but non-essential nodes.

Example:

exemptions:
  optional_in_ground_truth:
    - node_type: "n8n-nodes-base.set"
      penalty: 3.0
      reason: "Set nodes for data transformation are okay to add"

    - node_type: "@n8n/n8n-nodes-langchain.toolCalculator"
      penalty: 2.0
      reason: "Extra tools are acceptable"

Conditional Exemptions

Use the when clause to apply exemptions conditionally:

exemptions:
  optional_in_generated:
    - node_type: "n8n-nodes-base.errorTrigger"
      penalty: 1.0
      when:
        disabled: true
      reason: "Disabled error handlers are optional"

Connection Rules

Rules for handling workflow connections (edges).

Structure

connections:
  ignore_connection_types: [...]
  equivalent_types: [...]

connections.ignore_connection_types (list of strings)

Connection types to completely ignore during comparison.

connections:
  ignore_connection_types:
    - "main"  # Ignore main data flow connections

connections.equivalent_types (list of lists)

Define groups of connection types that should be treated as equivalent.

Example:

connections:
  equivalent_types:
    - ["main", "ai"]
    - ["error", "fallback"]

This means:

• main and ai connections are interchangeable
• error and fallback connections are interchangeable

Output Configuration

Controls how results are formatted and presented.

Structure

output:
  max_edits: <integer>
  group_by: "<grouping_strategy>"
  include_explanations: <boolean>
  include_suggestions: <boolean>

output.max_edits (integer, default: 15)

Maximum number of edit operations to return in the results.

Use case:

• Set higher (e.g., 20-50) for detailed debugging
• Set lower (e.g., 5-10) for quick summaries

output:
  max_edits: 15

output.group_by (string, default: "priority")

How to group edit operations in the output.

Options:

• "priority": Group by priority (critical, major, minor)
• "type": Group by edit type (node, edge, parameter)
• "cost": Order by cost (highest first)

output:
  group_by: "priority"

output.include_explanations (boolean, default: true)

Include detailed explanations for each edit operation.

output:
  include_explanations: true

output.include_suggestions (boolean, default: true)

Include suggestions for how to fix issues.

output:
  include_suggestions: true

Examples

Example 1: Strict Production Configuration

For production workflows where exact matching is critical:

vers