Thread/process pool parallel execution utilities — `map` with tqdm, auto CPU count.
Full Documentation · uv pip install scitex-parallel[all]
| # | Problem | Solution |
|---|---|---|
| 1 | concurrent.futures is low-level — users rewrite "map-with-progress-bar" every project |
stx.parallel.run(func, args) — drop-in map with tqdm, auto CPU-count; 1-liner for I/O-bound workloads (HTTP, file reads, API calls) |
| 2 | joblib.Parallel is heavyweight + process-based by default — overkill for threaded I/O |
Thread-based, no dep beyond stdlib + tqdm — right tool for the 80% case |
pip install scitex-parallelfrom scitex_parallel import run
results = run(my_func, items, n_jobs=4)Python API
from scitex_parallel import run
# I/O-bound parallel map with tqdm progress bar.
results = run(fetch_url, urls, n_jobs=8, desc="Fetching")
# Tuple-arg form for multi-arg functions.
results = run(my_func, [(a, b) for a, b in zip(xs, ys)], n_jobs=4)scitex_parallel/
├── __init__.py ← public surface (run, __version__)
└── _run.py ← `run(func, args, n_jobs=…)` thread-pool map
flowchart LR
A[items list] --> B[run func, items, n_jobs=8]
B --> T1[thread 1]
B --> T2[thread 2]
B --> Tn[thread N]
T1 & T2 & Tn --> C[tqdm progress bar]
C --> D[results list]
from scitex_parallel import run
urls = ["https://api.example.com/{}".format(i) for i in range(100)]
results = run(fetch_url, urls, n_jobs=8, desc="Fetching")Fetching: 100%|████████████| 100/100 [00:04<00:00, 22.3 it/s]
scitex-parallel is part of SciTeX. Install via
the umbrella with pip install scitex[parallel] to use as
scitex.parallel (Python) or scitex parallel ... (CLI).
Four Freedoms for Research
- The freedom to run your research anywhere — your machine, your terms.
- The freedom to study how every step works — from raw data to final manuscript.
- The freedom to redistribute your workflows, not just your papers.
- The freedom to modify any module and share improvements with the community.
AGPL-3.0 — because we believe research infrastructure deserves the same freedoms as the software it runs on.
AGPL-3.0
