A production-grade, configuration-driven workflow orchestration engine built on GoCodeAlone/modular v1.12.3. Define entire applications in YAML -- from API servers to multi-service chat platforms -- with 90+ module types, dynamic hot-reload, AI-powered generation, and a visual builder UI.
What It Does
The workflow engine turns YAML configuration files into running applications. No code changes needed. The same codebase can operate as:
A RESTful API server with JWT authentication and middleware chains
An event-driven pipeline with Kafka messaging and state machines
A multi-service platform with Docker Compose, reverse proxies, and observability
An AI-assisted workflow builder with drag-and-drop visual editing
JWT Authentication with user registration, login, token generation/validation, role-based claims, and bcrypt password hashing
PII Encryption at Rest using AES-256-GCM with SHA-256 key derivation -- configurable field-level encryption integrated with PersistenceStore and Kafka payloads
Middleware Chain: CORS, rate limiting, request ID propagation, auth enforcement
Dynamic Component Hot-Reload
Load Go components at runtime without restarting the server. The Yaegi interpreter provides:
Sandboxed execution with stdlib-only import validation
File watcher for automatic hot-reload on save
Component registry with full lifecycle management (init, start, stop)
HTTP API: POST/GET/DELETE /api/dynamic/components
AI-Powered Workflow Generation
Hybrid AI integration with two providers:
Anthropic Claude -- direct API with tool use for component generation and validation
GitHub Copilot SDK -- session-based integration for development workflows
Automatic validation loop with compile-test-retry cycle
Natural language to YAML workflow generation
EventBus Integration
Native EventBus bridge adapting MessageBroker to EventBus
For repository layout and where to place examples, fixtures, and generated
scaffolds, see Repository Layout.
Quick Start
Requirements
Go 1.27.2+
Node.js 24+ (for UI development)
Run the Server
# Clone the repository
git clone https://github.com/GoCodeAlone/workflow.git
cd workflow
# Build and run with an example config
go build -o server ./cmd/server
./server -config example/order-processing-pipeline.yaml
# Or run directly
go run ./cmd/server -config example/order-processing-pipeline.yaml
The server starts on :8080 by default. Override with -addr :9090.
# Chat platform (multi-service with Kafka, Prometheus, Grafana)
cd example/chat-platform
docker compose up
# E-commerce app
cd example/ecommerce-app
docker compose up
CLI Tool (wfctl)
wfctl is a command-line tool for inspecting, validating, and generating workflow configurations.
Install with Homebrew:
brew tap gocodealone/tap
brew install wfctl
Install from source:
go install github.com/GoCodeAlone/workflow/cmd/wfctl@latest
Direct binary downloads are published on
GitHub Releases
with checksums.txt for verification. See
wfctl Installation And Plugin Lifecycle for
Homebrew, terminal download, browser download, Windows, update, and plugin
lifecycle instructions.
Commands:
wfctl inspect <config.yaml> — summarize modules, workflows, triggers, and dependencies
wfctl validate <config.yaml> — deep validation against known module/step types
wfctl doctor — diagnose install, project, lockfile, and plugin lifecycle state
Seed data system for users, affiliates, programs, keywords, and surveys
Order Processing Pipeline
A 10+ module workflow demonstrating module composition with HTTP servers, routers, handlers, data transformers, state machines, message brokers, and observability. See example/order-processing-pipeline.yaml.
Example Configurations
The example/ directory contains 27 top-level example/*.yaml configurations covering:
API gateways and reverse proxies
Event-driven and scheduled workflows
State machine lifecycle management
Data transformation and webhook delivery
Multi-workflow composition
Real-time messaging
Dependency injection patterns
Multi-tenant scenarios
Each example includes a companion .md file documenting its architecture and usage.
Architecture
graph TD
A[YAML Config] --> B[StdEngine / BuildFromConfig]
B --> C[Module Factories]
C --> D[Modules]
D --> E[Application\nDependency Injection]
E --> F[Workflow Handlers]
F --> F1[HTTP]
F --> F2[Messaging]
F --> F3[StateMachine]
F --> F4[Scheduler]
F --> F5[Integration]
G[Triggers] --> G1[HTTP Endpoints]
G --> G2[EventBus Subscriptions]
G --> G3[Cron Schedules]
G1 --> H[TriggerWorkflow]
G2 --> H
G3 --> H
H --> I[Handler Dispatch]
I --> J[Pipeline Execution]
J --> J1[Steps: validate, transform,\nconditional, http_call, db_query...]
Core flow:
StdEngine loads YAML config via BuildFromConfig()
Each module definition is matched to a factory (48+ built-in types) and instantiated
Modules register with the modular Application (dependency injection, service registry)
Register it in engine.go's BuildFromConfig switch statement
Add an example YAML config in example/
Adding a New Workflow Handler
Implement the WorkflowHandler interface in handlers/
Register with engine.RegisterWorkflowHandler() in cmd/server/main.go
Testing
# All Go tests
go test ./...
# With race detection
go test -race ./...
# With coverage
go test -cover ./...
# Single test
go test -v -run TestName .
# UI component tests (Vitest)
cd ui && npm test
# UI E2E tests (Playwright)
cd ui && npx playwright test
# Lint
go fmt ./...
golangci-lint run
cd ui && npm run lint
Test coverage targets: root package 80%+, module 80%+, dynamic 80%+, AI packages 85%+.
docs/tutorials/ — Getting started and walkthrough tutorials
Roadmap
See ROADMAP.md for the full development history (Phases 1-6 complete) and planned work including JSON Schema config validation, performance benchmarks, Helm charts, and security hardening.
DefaultHandlerFactory is a function that returns a slice of default
WorkflowHandler instances. It is set by the setup package via an init
function to break the import cycle between the root workflow package
and the handlers package. Import "github.com/GoCodeAlone/workflow/setup"
(typically as a blank import) to register the default factories.
type EngineBuilder struct {
// contains filtered or unexported fields
}
EngineBuilder provides a fluent API for constructing a fully-configured
StdEngine. It encapsulates the boilerplate of registering workflow handlers,
triggers, dynamic components, and plugins so that CLI tools, MCP servers,
and other consumers can initialise an engine in a few lines.
Import the setup package to register default handler and trigger factories:
BuildAndConfigure creates a fully-configured StdEngine and loads
configuration from the file path set via WithConfigPath(). Returns
an error if no config path was set or the file cannot be loaded.
RunUntilSignal is a convenience method for CLI tools that creates the
engine, loads config, starts it, and blocks until a termination signal
(SIGINT/SIGTERM) is received. It handles graceful shutdown automatically.
WithAllDefaults is a convenience method that enables default handlers,
default triggers, and dynamic components.
Requires importing the setup package: import _ "github.com/GoCodeAlone/workflow/setup"
EngineBuilderFunc is called by the manager to create and configure an engine
from a parsed workflow config. The caller is responsible for registering
workflow handlers, dynamic components, and other setup. The function must
call BuildFromConfig on the engine before returning.
GetEngine returns the underlying engine, satisfying the module.triggerableEngine
interface so the CrossWorkflowRouter can trigger workflows via duck-typing.
PipelineAdder is implemented by workflow handlers that can receive named pipelines.
This allows the engine to add pipelines without importing the handlers package.
buildPipelineSteps creates PipelineStep instances from step configurations.
RoutePipelineSetter is implemented by handlers (QueryHandler, CommandHandler) that support per-route pipelines.
BuildFromApplicationConfig loads a multi-workflow application config and builds
the engine from all referenced workflow configs. Each workflow config file is
parsed independently, and their modules are merged into the shared module
registry. Module name conflicts across workflow files produce a clear error.
This is the entry point for the application-level multi-workflow feature:
ExecutePipeline runs a named pipeline synchronously and returns its
structured output. For use by Go callers (gRPC servers, tests) that
don't need HTTP request/response threading.
If the pipeline uses step.pipeline_output, the explicitly marked output
is returned. Otherwise, the pipeline's merged Current state is returned.
This goes through TriggerWorkflow so all trigger paths share the same
execution code, including StepOutputs preservation via PipelineContextHolder.
ExecutePipelineContext runs a named pipeline and returns the full PipelineContext,
including StepOutputs for each step. This is intended for test harnesses that need
per-step output inspection. For production callers, use ExecutePipeline instead.
This goes through TriggerWorkflow so all trigger paths share the same
execution code. The PipelineContextHolder captures the full context.
GetPipeline returns the named pipeline from the engine's pipeline registry.
Returns nil and false if no pipeline with the given name exists.
This is useful for CLI tools (e.g., wfctl pipeline run) that need to
execute a pipeline directly without starting the HTTP server.
LoadPluginWithOverride loads an EnginePlugin into the engine, allowing it to
override existing module, step, trigger, handler, deploy target, and sidecar
provider registrations. When a duplicate type is encountered, the new factory
replaces the previous one and a warning is logged, instead of returning an
error. This is intended for external plugins that intentionally replace
built-in defaults (e.g., replacing a mock step with a production
implementation, or swapping out deploy targets/sidecar providers).
RegisterTriggerType registers a mapping from a trigger config type key
(e.g., "reconciliation") to a trigger Name() (e.g., "trigger.reconciliation").
This is used for triggers registered directly via RegisterTrigger() rather
than through a plugin's TriggerFactories().
NewWorkflowEngineManager creates a new manager for workflow engine instances.
The engineBuilder function is called to create each new engine instance,
allowing the caller to register handlers and configure the dynamic system.
type WorkflowHandler interface {
// CanHandle returns true if this handler can process the given workflow type CanHandle(workflowType string) bool// ConfigureWorkflow sets up the workflow from configuration ConfigureWorkflow(app modular.Application, workflowConfig any) error// ExecuteWorkflow executes a workflow with the given action and input data ExecuteWorkflow(ctx context.Context, workflowType string, action string, data map[string]any) (map[string]any, error)
}
WorkflowHandler interface for handling different workflow types
Package diffcache caches per-resource Diff results so a wfctl invocation that re-runs a Plan against unchanged inputs can skip the (sometimes network-expensive) provider-side Diff call.
Package diffcache caches per-resource Diff results so a wfctl invocation that re-runs a Plan against unchanged inputs can skip the (sometimes network-expensive) provider-side Diff call.
Package providerclient provides a core-importable gRPC-client adapter that wraps the plugin/external/proto IaC gRPC services as interfaces.IaCProvider (+ optional sub-interfaces).
Package providerclient provides a core-importable gRPC-client adapter that wraps the plugin/external/proto IaC gRPC services as interfaces.IaCProvider (+ optional sub-interfaces).
Package refreshoutputs implements read-only state refresh — it reads current Outputs from providers and updates the persisted state when fields differ.
Package refreshoutputs implements read-only state refresh — it reads current Outputs from providers and updates the persisted state when fields differ.
Package sensitive routes ResourceOutput fields flagged as Sensitive through a secrets.Provider, returning sanitized placeholders for state persistence and a hydrated map for in-process consumers.
Package sensitive routes ResourceOutput fields flagged as Sensitive through a secrets.Provider, returning sanitized placeholders for state persistence and a hydrated map for in-process consumers.
Package migration provides a migration management system with schema providers, distributed locking, and a migration runner for applying database schema changes.
Package migration provides a migration management system with schema providers, distributed locking, and a migration runner for applying database schema changes.
Package ai provides a plugin that registers AI pipeline step types (ai_complete, ai_classify, ai_extract), the dynamic.component module type, and the sub_workflow step.
Package ai provides a plugin that registers AI pipeline step types (ai_complete, ai_classify, ai_extract), the dynamic.component module type, and the sub_workflow step.
Package datastores provides an EnginePlugin that registers NoSQL data store module types (nosql.memory, nosql.dynamodb, nosql.mongodb, nosql.redis) and their corresponding pipeline step types (step.nosql_get, step.nosql_put, step.nosql_delete, step.nosql_query).
Package datastores provides an EnginePlugin that registers NoSQL data store module types (nosql.memory, nosql.dynamodb, nosql.mongodb, nosql.redis) and their corresponding pipeline step types (step.nosql_get, step.nosql_put, step.nosql_delete, step.nosql_query).
Package license provides an EnginePlugin that registers the license.validator module type, which validates licenses against a remote server and gates premium plugin access.
Package license provides an EnginePlugin that registers the license.validator module type, which validates licenses against a remote server and gates premium plugin access.
Package observability provides an EnginePlugin that contributes all observability-related module types: metrics collector, health checker, log collector, OpenTelemetry tracing, OpenAPI generator/consumer, and distributed trace propagation.
Package observability provides an EnginePlugin that contributes all observability-related module types: metrics collector, health checker, log collector, OpenTelemetry tracing, OpenAPI generator/consumer, and distributed trace propagation.
Package platform provides an EnginePlugin that registers all platform-related module types, the platform workflow handler, the reconciliation trigger, and the platform template step.
Package platform provides an EnginePlugin that registers all platform-related module types, the platform workflow handler, the reconciliation trigger, and the platform template step.
Package policy provides an EnginePlugin that registers the policy.mock module type and policy pipeline step types (step.policy_evaluate, step.policy_load, step.policy_list, step.policy_test).
Package policy provides an EnginePlugin that registers the policy.mock module type and policy pipeline step types (step.policy_evaluate, step.policy_load, step.policy_list, step.policy_test).
Package validation provides shared pipeline configuration validation utilities that are used by both the workflow engine (at startup) and the wfctl CLI tool (as static analysis).
Package validation provides shared pipeline configuration validation utilities that are used by both the workflow engine (at startup) and the wfctl CLI tool (as static analysis).
README
Documentation
Source Files
Directories
¶