| Taskflow: A General-purpose Task-parallel Programming System

Loading...
Searching...
No Matches
Composable Tasking

Composition is a key to improve the programmability of a complex workflow. This chapter describes how to create a large parallel graph through composition of modular and reusable blocks that are easier to optimize.

# Compose a Taskflow

A powerful feature of Taskflow is its composable interface. You can break down a large parallel workload into smaller pieces each designed to run a specific task dependency graph. This largely facilitates the modularity of writing a parallel task program.

// f1 has three independent tasks
tf::Taskflow f1;
f1.name("F1");
tf::Task f1A = f1.emplace([&](){ std::cout << "F1 TaskA\n"; });
tf::Task f1B = f1.emplace([&](){ std::cout << "F1 TaskB\n"; });
tf::Task f1C = f1.emplace([&](){ std::cout << "F1 TaskC\n"; });
f1A.name("f1A");
f1B.name("f1B");
f1C.name("f1C");
f1A.precede(f1C);
f1B.precede(f1C);
// f2A ---
// |----> f2C ----> f1_module_task ----> f2D
// f2B ---
tf::Taskflow f2;
f2.name("F2");
tf::Task f2A = f2.emplace([&](){ std::cout << " F2 TaskA\n"; });
tf::Task f2B = f2.emplace([&](){ std::cout << " F2 TaskB\n"; });
tf::Task f2C = f2.emplace([&](){ std::cout << " F2 TaskC\n"; });
tf::Task f2D = f2.emplace([&](){ std::cout << " F2 TaskD\n"; });
f2A.name("f2A");
f2B.name("f2B");
f2C.name("f2C");
f2D.name("f2D");
f2A.precede(f2C);
f2B.precede(f2C);
tf::Task f1_module_task = f2.composed_of(f1).name("module");
f2C.precede(f1_module_task);
f1_module_task.precede(f2D);
f2.dump(std::cout);
tf::FlowBuilder::emplace
Task emplace(C &&callable)
creates a static task
Definition flow_builder.hpp:1738
tf::FlowBuilder::composed_of
Task composed_of(T &object)
creates a module task for the target object
Definition flow_builder.hpp:1788
tf::Task
class to create a task handle over a taskflow node
Definition task.hpp:569
tf::Task::name
const std::string & name() const
queries the name of the task
Definition task.hpp:1388
tf::Task::precede
Task & precede(Ts &&... tasks)
adds precedence links from this to other tasks
Definition task.hpp:1258
tf::Taskflow
class to create a taskflow object
Definition taskflow.hpp:64
tf::Taskflow::dump
void dump(std::ostream &ostream) const
dumps the taskflow to a DOT format through a std::ostream target
Definition taskflow.hpp:433
tf::Taskflow::name
void name(const std::string &)
assigns a new name to this taskflow
Definition taskflow.hpp:386

The above example first constructs a taskflow consisting of three tasks, `f1A`, `f1B`, and `f1C`, where `f1A` and `f1B` execute before `f1C`. It then creates a second taskflow with four tasks, `f2A`, `f2B`, `f2C`, and `f2D`. The first taskflow is encapsulated as a module task using Taskflow::composed_of, allowing it to be embedded within the second taskflow. Dependencies are then established so that `f2C` must complete before the module task begins, and the module task must finish before `f2D` executes, thereby integrating the two taskflows into a single execution graph with well-defined ordering constraints.

# Create a Module Task from a Taskflow

The task created from Taskflow::composed_of is a module task that runs on a pre-defined taskflow. A module task does not own the taskflow but maintains a soft mapping to the taskflow. You can create multiple module tasks from the same taskflow but only one module task can run at one time. For example, the following composition is valid. Even though the two module tasks `module1` and `module2` refer to the same taskflow `F1`, the dependency link prevents `F1` from multiple executions at the same time.

However, the following composition is invalid. Both module tasks refer to the same taskflow. They can not run at the same time because they are associated with the same graph.

# Create a Custom Composable Graph

Taskflow allows you to create a custom graph object that can participate in the scheduling using composition. To become a module task, your class `T` must define the method `T::graph()` that returns a reference to the tf::Graph object managed by `T`. The following example defines a custom graph object that can be assembled in a taskflow through composition:

struct CustomGraph {
tf::Graph graph;
CustomGraph() {
tf::FlowBuilder builder(graph); // inherit all task builders in tf::Taskflow
tf::Task task = builder.emplace([](){
std::cout << "a task\n"; // static task
});
}
// returns a reference to the graph for taskflow composition
Graph& graph() { return graph; }
};
CustomGraph obj;
tf::Task comp = taskflow.composed_of(obj);
tf::FlowBuilder
class to build a task dependency graph
Definition flow_builder.hpp:114
tf::Graph
class to create a graph object
Definition graph.hpp:47
tf::Task::composed_of
Task & composed_of(T &object)
creates a module task from a taskflow
Definition task.hpp:1290

The above code defines a custom graph that can participate in taskflow composition. The graph object is represented using tf::Graph, and its constructor builds the internal task graph through tf::FlowBuilder. To support composition, the graph implements the required interface method (`Graph& graph()`) that exposes its internal structure to the Taskflow runtime. Or, you can simply expose the `graph` and pass it to tf::FlowBuilder::composed_of without defining an additional struct.

tf::Graph graph;
tf::FlowBuilder builder(graph); // inherit all task builders in tf::Taskflow
tf::Task task = builder.emplace([](){
std::cout << "a task\n"; // static task
});
tf::Task comp = taskflow.composed_of(obj);

With this interface in place, the custom graph can then be instantiated as a module task within a larger taskflow, enabling it to be seamlessly composed and scheduled alongside other tasks.

Note
  Users are responsible for ensuring the given target remains valid throughout its execution. The executor does not assume ownership of the target object.

# Create an Adopted Module Task

Unlike tf::FlowBuilder::composed_of, which holds a reference to an externally-owned graph and requires the caller to manage its lifetime, tf::FlowBuilder::adopt transfers ownership of a tf::Graph into the task via move semantics. Once adopted, the graph is owned and managed by the executor for the duration of its execution, and the caller must not access the moved-from graph afterward. The following example creates a graph through tf::FlowBuilder and moves it to a taskflow as an adopted module task:

tf::Taskflow taskflow;
// build a graph independently using FlowBuilder
tf::Graph g;
tf::FlowBuilder fb(g);
tf::Task t1 = fb.emplace([]{ std::cout << "task A\n"; }).name("A");
tf::Task t2 = fb.emplace([]{ std::cout << "task B\n"; }).name("B");
t1.precede(t2);
// adopt the graph into the taskflow — ownership transfers here
tf::Task module = taskflow.adopt(std::move(g)).name("adopted module task");
// g is now in a moved-from state and must not be used
tf::Executor executor;
executor.run(taskflow).wait();
tf::Executor
class to create an executor
Definition executor.hpp:62
tf::Executor::run
tf::Future< void > run(Taskflow &taskflow)
runs a taskflow once
tf::FlowBuilder::adopt
Task adopt(Graph &&graph)
creates a module task from a graph by taking over its ownership
Definition flow_builder.hpp:1795

The adopted module task can participate in the taskflow's dependency graph just like any other module task created via tf::FlowBuilder::composed_of. You can chain dependencies before and after it in the usual way:

tf::Task before = taskflow.emplace([]{ std::cout << "before\n"; }).name("before");
tf::Task after = taskflow.emplace([]{ std::cout << "after\n"; }).name("after");
tf::Graph g;
tf::FlowBuilder{g}.emplace([]{ std::cout << "inside adopted graph\n"; });
tf::Task module = taskflow.adopt(std::move(g)).name("module");
before.precede(module);
module.precede(after);
tf::Executor executor;
executor.run(taskflow).wait();

Note
  The key distinction between tf::Taskflow::composed_of and tf::Taskflow::adopt is ownership. Use `composed_of` when the graph is long-lived and shared across multiple taskflows or multiple runs. Use `adopt` when you want to transfer a graph into the taskflow and let the executor manage its lifetime.