Skip to main content

5.1 Defining Workflows

CNCF-Compatible Formatting​

Our workflow format follows the CNCF Open Workflow Specification - version 1.0.3 - but uses JSON only.
We keep CNCF-aligned structure and naming, but omit YAML, catalogs, and advanced DSL features.

The Cloud Native Computing Foundation (CNCF) is an open-source foundation that hosts and sustains cloud-native projects — such as Kubernetes — to advance containerization, microservices, and scalable distributed systems.

Triggering Workflows​

Workflows can be triggered the same way as any datapoint: via Bruno, Postman, curl, or any other API client using the Data-Orchestrator.
Each workflow exposes a standard HTTP endpoint, accepts JSON input, and behaves like any other callable datapoint.

Workflow Syntax Overview​

Our DSL is based on the Open Workflow Specification, but simplified and adapted to JSON.

Workflow Structure​

A workflow file has the following top-level structure:

Example

{
"id": "string",
"type": "workflow",
"version": "0.0.1",
"workflow": {
"type": "temporal",
"definition": {
"do": []
}
}
}

A workflow definition has the following fields:

FieldsDescription
id (string)Unique identifier of the workflow.
type (string)Workflow resource type. Always "workflow".
version (string)Version of the workflow definition.
workflow.type (string)Execution engine type. ⚠️ Currently always "temporal".
workflow.definition (object)Root segment for workflow logic.
workflow.definition.do (array)Ordered list of tasks executed sequentially.

A simple hello-world workflow that uses the hello-world-http datapoint created in the OLIVE hello world example would look like this:

Example

{
"id": "helloworld-wf1-http-get",
"name": "helloworld-wf1-http-get",
"type": "workflow",
"version": "0.0.1",
"displayName": "helloworld-wf1-http-get",
"description": "Simple hello-world workflow that demonstrates invoking a single HTTP GET datapoint with a fixed URL parameter. Executes one datapoint call to fetch content from a predefined endpoint, illustrating the basic structure of a Temporal workflow and datapoint invocation in OLIVE.",
"tags": [
"example",
"simple-example",
"scenario-helloworld",
"scenario-example-helloworld"
],
"workflow": {
"type": "temporal",
"definition": {
"do": [
{
"echoCall": {
"call": "olive",
"with": {
"datapoint": "helloworld-dp1-hello-world-http-get",
"parameters": {
"url": "http://info.cern.ch/"
}
}
}
}
]
}
}
}

Tasks (do: [])​

Tasks inside the do array execute sequentially, unless they contain their own control structures such as for or switch.

Example

{
"do": [
{ "taskA": {} },
{ "taskB": {} }
]
}

Each task key (taskA, taskB) is a local name, not a reference to a function.

Call Task​

Executes a connectable element through OLIVE.

Example

{
"taskName": {
"call": "olive",
"with": {
"datapoint": "your-element-id",
"parameters": {}
}
}
}
FieldDescription
call"olive" to invoke a connectable element, or "http" for a direct HTTP request
with.datapointThe id of the element to execute
with.parametersThe parameters passed to it. May contain JSONata expressions
with.datapoint takes any connectable element, not only a datapoint

Despite its name, the field accepts the id of any connectable element — a datapoint, a connector or another workflow. The name is historical.

A nested element runs as its own job, with the calling workflow as its parent: the workflow waits for it and receives its result like any other call. This is what lets a workflow compose connectors and other workflows rather than only individual datapoints, and it is the practical way to keep a large orchestration within the engine's limits — see Workflow Orchestration Limitations.

Result size is limited inside a workflow

Every call runs as a Temporal activity, and Temporal records the activity's input and result in the workflow history. A serialized datapoint result above roughly 2 MB is therefore not carried: the value is stored as null and the following task receives nothing.

This limit comes from the workflow engine, not from the datapoint. The same datapoint called directly, or from a connector, is not subject to it.

Where a step genuinely produces a large payload, write it to the file-storage-service inside the datapoint and let the workflow pass the file key between steps instead of the content. See File Handling in Connectivity and Workflow Orchestration Limitations.


Parameters & Expressions​

Parameter values may contain JSONata expressions, always wrapped in ${ ... }.

Example

"param1": "${input.name}"
"param2": "${input}"
"when": "${input.action = \"ping\"}"
  • We use JSONata (not JQ).
  • No YAML shortcuts, no loose mode.
  • The expression MUST evaluate to valid JSON.

Loops (for)​

The syntax for iterating arrays:

Example

{
"loopName": {
"for": {
"each": "item",
"in": "${input.array}",
"at": "index"
},
"do": [
{ "innerTask": {} }
]
}
}

Switch (Conditional Branching)​

Conditional routing is done via an array of branches:

Example

{
"switchTask": {
"switch": [
{ "case1": { "when": "...", "then": "taskX" } },
{ "case2": { "when": "...", "then": "taskY" } },
{ "default": { "then": "exit" } }
]
}
}
  • Uses JSONata for conditions.
  • Maps to if / else semantics.

Set Task​

The set task creates new variables inside the workflow context.
These values can be simple fields or nested objects.
All assigned values may use JSONata expressions wrapped in ${ ... }.

Example

{
"storeValues": {
"set": {
"simple": "${input.name}",
"complex": {
"nested": {
"id": "${input.userId}"
}
}
}
}
}
  • Defines new workflow-scope variables
  • Makes them available to all following tasks
  • Allows simple and nested assignments
  • The set task does not modify input. It only adds new variables (like simple or complex) that are available to all following tasks.

After this task completes, the following workflow-level variables exist:

  • ${simple} → value of input.name
  • ${complex} → the entire object defined under complex
  • ${complex.nested.id} → nested value inside the structure

These variables may then be referenced in any subsequent task.

Chaining Tasks​

Use the "then": "<taskName>" field to jump forward:

Example

{
"someTask": {
"call": "olive",
"with": {},
"then": "end"
}
}

Supported destinations:

DestinationEffect
another task's nameExecution continues at that task
"end"The workflow completes successfully, returning the current data as its result
"exit"The enclosing block is left: inside a for loop the iteration stops, otherwise the workflow ends

Fork Task​

The fork task executes multiple branches from a single point in the workflow.

Each branch contains its own task definition. Depending on the compete setting, the branches are either executed in parallel or in sequence.

This is useful when the workflow needs to trigger multiple independent actions from the same step, for example notifying several services, calling multiple datapoints, or splitting processing into separate branches.

A fork task definition supports the following fields:

FieldsDescription
fork (object)Indicates that the task is a fork task.
fork.compete (boolean, optional)Controls how the branches are executed. If true, branches are executed in parallel. If false, branches are executed sequentially. If omitted, the default is true.
fork.branches (array)Ordered list of branch task definitions executed by the fork task.

Example

{
"forkTaskName": {
"fork": {
"compete": true,
"branches": [
{
"branchTaskA": {
"call": "olive",
"with": {
"datapoint": "<your-first-datapoint>",
"parameters": {
"message": "First branch"
}
}
}
},
{
"branchTaskB": {
"call": "olive",
"with": {
"datapoint": "<your-second-datapoint>",
"parameters": {
"message": "Second branch"
}
}
}
}
]
}
}
}

With compete: true, all branches are executed in parallel and the workflow waits until all branches have completed.

With compete: false, the branches are executed sequentially in the order in which they are defined.

compete means something different here than in the Open Workflow Specification DSL

Two differences, not one:

  • Meaning. In the Open Workflow Specification, compete: true is a race — the first branch to finish wins and the others are cancelled. In OLIVE it means the branches run in parallel and the workflow waits for all of them. OLIVE has no racing fork.
  • Default. OLIVE defaults compete to true (parallel). The specification defaults it to false.

So compete selects parallel versus sequential execution, and nothing is ever cancelled. Read it as "run the branches concurrently", not as "let them compete".


Wait Task​

The wait task pauses workflow execution for a defined duration before continuing with the next task.

A wait task definition supports the following fields:

FieldsDescription
wait (object | string)Defines the duration to pause workflow execution.
wait.seconds (number)Number of seconds to wait before continuing execution.
wait (string)ISO-8601 duration string (e.g., "PT30S").

The wait task can be declared in two formats.

This format specifies the delay using explicit time fields.

Example

{
"waitTaskName": {
"wait": {
"seconds": 30
}
}
}

Alternatively, the delay can be expressed using an ISO-8601 duration string.

Example

{
"waitTaskName": {
"wait": "PT30S"
}
}

Raise Task​

The raise task immediately stops workflow execution and raises an error.

When a raise task executes, workflow processing terminates and no subsequent tasks are executed.

Errors raised by the workflow follow a structured error definition, which contains metadata describing the error and can optionally include contextual information.

The raise task fields include:

FieldsDescription
raise (object)Indicates that the task raises a workflow error.
raise.error (object | string)Either an inline error definition or a reference to a reusable error defined under use.errors.

And the error definition supports the following fields:

FieldsDescription
type (string)A stable identifier for the error type (often a URI)
status (number)HTTP-like status code (e.g., 400, 404, 500)
title (string, optional)Human-readable summary of the error
detail (string, optional)Human-readable explanation. Supports placeholders (for example, injecting workflow parameters)
instance (string, optional)Optional instance identifier for the specific occurrence
  • Placeholders in the error definition are resolved using workflow parameters.

The raise task can reference an error in two ways:

  • Inline error definition (error object defined directly in the task)
  • Reusable error reference (error defined once and reused via use.errors)

Inline Error Definition

When using an inline error definition, the error object is provided directly inside the raise task. This approach is useful when the error is only used by a single task.

Example

{
"raiseTaskName": {
"raise": {
"error": {
"type": "<error-type>",
"status": "<status-code>",
"title": "<error-title>",
"detail": "<error-description>",
"instance": "<optional-instance-id>"
}
}
}
}

Reusable Error Reference

Errors can also be defined once in the workflow and referenced by name from multiple raise tasks.

Example

{
"use": {
"errors": {
"errorName": {
"type": "<error-type>",
"status": "<status-code>",
"title": "<error-title>",
"detail": "<error-description>",
"instance": "<optional-instance-id>"
}
}
},
"do": [
{
"raiseTaskName": {
"raise": {
"error": "errorName"
}
}
}
]
}
  • Reusable errors are defined under the workflow's use.errors section.
  • The error is then referenced by name from the raise task.

Try / Catch​

The try / catch task allows workflows to handle errors that occur during execution.
Tasks inside the try block are executed normally. If any task throws an error, the workflow engine evaluates the catch configuration and can perform recovery logic.

A try/catch task contains two main parts:

  • try – the list of tasks to execute
  • catch – optional error handling configuration

Example

{
"tryTaskName": {
"try": [
{ "someTask": {} }
],
"catch": {
}
}
}

If no catch handler is defined, the error propagates and the workflow fails.

FieldsDescription
try (array)Array of tasks executed sequentially.
catch (object, optional)Defines how errors thrown in the try block are handled.
catch.do (array, optional)Tasks executed when an error is caught.
catch.as (string, optional)Optional variable name used to store error details in the workflow context.
catch.errors.with (object, optional)Optional filter used to catch only specific errors (e.g., by status or type).
catch.retry (object | string, optional)Optional retry configuration defining how the try tasks should be retried before executing catch.do .

Error Filtering

The catch.errors.with field allows workflows to handle only specific errors.

If the error does not match the filter, the workflow engine rethrows the error, allowing it to propagate.

If no filter is defined it's treated as a "catch all".

Example

{
"errors": {
"with": {
"status": "<status-code>",
"type": "<error-type>"
}
}
}

Accessing Error Details

When an error is caught, the workflow engine extracts structured error information and optionally stores it in a workflow variable defined by catch.as .

This variable can have any name.

Example

"catch": {
"as": "error"
}

This makes the error available to subsequent tasks using:

Example

${error}

The stored error object may include fields such as:

Fields
type (string, optional)
status (number, optional)
instance (string, optional)
title (string, optional)
detail (string, optional)

Retry Policy

The retry policy needs to be a valid configuration, not missing any required fields otherwise it fails.

FieldsDescription
retry (object | string)Defines retry behavior or references a reusable retry policy.
retry.delay (object | string)Delay between retry attempts. Supports duration fields or ISO-8601 duration strings.
retry.backoff (object, optional)Strategy controlling how delays increase between retries.
retry.limit.attempt.count (number, optional)Maximum number of retry attempts before failure. Defaults to 10 attempts if no value is set.

The structure of the retry policy looks as follows:

Example

{
"callWithRetry": {
"call": "olive",
"with": {
"datapoint": "my-flaky-datapoint"
},
"catch": {
"errors": {
"with": {
"status": 503
}
},
"retry": {
"delay": {
"seconds": 2
},
"backoff": {
"exponential": {}
},
"limit": {
"attempt": {
"count": 5
}
}
}
}
}
}

limit.attempt.count is a number of attempts, not a duration. The example above retries a 503 up to five times, waiting two seconds before the first retry and doubling the wait each time.


Timeout​

Timeout is an additional property, which can be defined on task or workflow level, and limits how long a workflow or task may execute before it fails.

Timeouts can be defined inline or declared once under use.timeouts and referenced by name.

A timeout definition supports the following fields:

FieldsDescription
timeout (object | string)Defines the timeout directly or references a reusable timeout definition.
timeout.after (object | string)Duration after which execution times out.
timeout.after.seconds (number)Number of seconds before the timeout is reached.

Timeouts are supported in two scopes:

ScopeDescription
task-level timeoutLimits the execution time of a single task.
workflow-level timeoutLimits the total execution time of the whole workflow.

Reusable timeout definitions are declared under use.timeouts and may be referenced by name from tasks or from the workflow root. The workflow engine resolves the referenced timeout before execution starts.

Reusable Timeout Definition

Timeouts can be defined once and reused across multiple tasks or workflows.

Example

{
"use": {
"timeouts": {
"quickTimeout": {
"after": {
"seconds": 5
}
}
}
}
}

This is useful when multiple tasks should share the same execution limit.

Task-Level Timeout

A task-level timeout limits only the execution of the task on which it is declared.

If the task does not complete before the configured duration elapses, the task fails with a timeout error and the workflow execution stops unless that error is handled elsewhere.

Example

{
"wait30Seconds": {
"wait": {
"seconds": 30
},
"timeout": {
"after": {
"seconds": 5
}
}
}
}

Workflow-Level Timeout

A workflow-level timeout limits the total runtime of the workflow definition.

If the workflow does not finish before the configured duration elapses, the workflow fails with a timeout error.

Example

{
"do": [
{
"firstCall": {
"call": "olive",
"with": {
"datapoint": "datapoint-call",
"parameters": {
"message": "Before waiting"
}
}
}
},
{
"wait30Seconds": {
"wait": {
"seconds": 30
}
}
}
],
"timeout": {
"after": {
"seconds": 15
}
}
}

Why JSON Instead of YAML​

Unlike the Open Workflow Specification DSL, we only support JSON:

  • Predictable structure
  • No YAML ambiguity around types
  • Direct mapping into JavaScript objects
  • Simpler validation
  • Consistent with all examples and tooling in our engine

Differences to Open Workflow Specification DSL 1.0.3​

The following tables summarize how our JSON Workflow DSL differs from the full Open Workflow Specification specification.

Legend

SymbolMeaning
✅Supported
❌Not supported
⚠️Difference in behavior

Supported Task Types​

Feature / taskDSL specOLIVENotes
Call✅ (many subtypes)✅ olive and http onlyolive invokes any connectable element — datapoint, connector or workflow
Do✅✅Sequential execution
For✅✅Standard iteration semantics
Switch✅✅JSONata expressions
Set✅✅—
Wait✅✅—
Raise✅✅—
Try / Catch✅✅—
Fork✅✅Parallel execution
Timeouts✅✅Workflow-level, task-level, and reusable definitions via use.timeouts
Call subtypes​
Call subtypeDSL specOLIVENotes
HTTP✅⚠️output and redirect options not supported
asyncapi✅❌—
gRPC✅❌—
OpenAPI✅❌—
A2A✅❌—
MCP✅❌—
OLIVE❌✅OLIVE-specific

Not Supported Task Types​

Feature / taskDSL specOLIVENotes
Emit / Event✅❌No event publishing
Run✅❌No external containers or scripts
Listen✅❌No event listeners

Expression & Parameter Differences​

AreaDSL specOLIVENotes
Expression engine✅ JQ⚠️ JSONataDifferent expression engine
Loose mode✅❌Must use explicit JSONata (for example ${input.name}); no bare shorthand
Input / output schema validation✅❌No auto-validation
Task context✅ Rich⚠️ LimitedMissing: secrets, authorization, definition, task
Fork default✅ default = sequential⚠️ Default = parallelDifferent default for compete parameter

Format & Scripting Differences​

AreaDSL specOLIVENotes
Use / function catalogs✅❌No catalogs; no auth blocks
Scripting✅❌No embedded scripts