Senior Python Developer – Data
Science & High-Performance
Computing
About the Role
We are seeking an exceptional Senior Python Developer with a strong mathematical
background and proven expertise in high-performance scientific computing. You will be at the
core of building, optimizing, and scaling data-intensive systems — from low-level numerical
algorithms to distributed multi-server architectures. This is a deeply technical role for engineers
who are equally comfortable deriving a Fourier transform and profiling a memory bottleneck in a
Dask pipeline.
Core Technical Requirements
Python — Senior Level
● 5+ years of professional Python development
● Mastery of Python internals: GIL, memory model, reference counting, object lifecycle
● Advanced use of typing system: generics, protocols, TypeVar, ParamSpec, runtime
type checking
● Deep knowledge of concurrency models: asyncio, threading,
multiprocessing, [Link]
● Proficiency with metaprogramming: decorators, descriptors, metaclasses,
__slots__
● Clean architecture principles: SOLID, dependency injection, design patterns in
Python context
● Expert-level testing: pytest, property-based testing (Hypothesis), mocking,
fixtures, test coverage strategies
Mathematics & Scientific Computing
● Strong foundation in linear algebra: matrix decompositions (SVD, QR, LU, Cholesky),
eigenvalue problems, sparse matrix operations
● Solid understanding of calculus and numerical methods: gradient-based optimization,
numerical integration, differentiation, root finding
● Proficiency in probability & statistics: distributions, hypothesis testing, Bayesian
inference, Monte Carlo methods
● Deep knowledge of signal processing theory: Fourier analysis, convolution,
windowing, spectral estimation, filtering theory (FIR/IIR)
● Familiarity with information theory fundamentals: entropy, mutual information,
dimensionality reduction (PCA, t-SNE, UMAP)
● Knowledge of graph theory and graph algorithms as applied to data science
problems
Fast Fourier Transform & Signal Processing
● Production-grade FFT implementations using [Link], [Link], pyFFTW
● FFT algorithm selection: Cooley-Tukey, Bluestein, Rader — knowing when and why
to use each
● Optimized FFT pipelines: zero-padding, windowing functions (Hann, Hamming,
Blackman), spectral leakage mitigation
● Multi-dimensional FFT (fft2, fftn) for image and volumetric data processing
● Real-time and batch FFT processing at scale with memory-mapped datasets
● Short-Time Fourier Transform (STFT), Wavelet Transforms as alternatives or
complements to FFT
Large-Scale Data Processing
● Expert-level NumPy: broadcasting, advanced indexing, stride tricks, memory layout
(C/F order), [Link]
● Pandas at scale: chunked processing, categorical optimization, efficient groupby,
merge, pivot
● Polars: lazy evaluation API, query optimization, Arrow-backed operations
● Dask: distributed DataFrames, task graph optimization, custom graph construction
● Apache Arrow / PyArrow: zero-copy reads, columnar memory format, IPC and
Parquet I/O
● Data serialization: Parquet, HDF5, Zarr, Feather — format selection based on access
patterns
● Stream processing fundamentals: windowing, micro-batching, backpressure handling
Clustering & Machine Learning at Scale
● Deep knowledge of clustering algorithms: K-Means, DBSCAN, HDBSCAN,
Agglomerative, Spectral Clustering
● Approximate Nearest Neighbor search: FAISS, Annoy, HNSW (hnswlib) for
billion-scale datasets
● Dimensionality reduction as pre-clustering step: PCA, UMAP, t-SNE — performance
vs. accuracy trade-offs
● Distributed ML with Dask-ML, Ray Tune, Spark MLlib
● Model evaluation at scale: cross-validation strategies, silhouette analysis,
Davies-Bouldin index
● Feature engineering pipelines: encoding, normalization, imputation at dataset scale
Performance Profiling & Optimization
● CPU profiling: cProfile, py-spy, Scalene, flame graph interpretation
● Memory profiling: Memray, memory_profiler, tracemalloc, leak detection
strategies
● I/O profiling: identifying bottlenecks in disk, network, and serialization layers
● Low-level optimization: Numba JIT (nopython mode, CUDA targets), Cython, C
extensions via ctypes / cffi
● Vectorization strategies: SIMD awareness, loop unrolling, cache-friendly data layouts
● Algorithmic optimization: complexity analysis, data structure selection, lazy
evaluation patterns
● Benchmarking discipline: pytest-benchmark, timeit, statistically sound
micro-benchmarks
Scalable Distributed Architecture
● Horizontal scaling: stateless service design, shared-nothing architecture, load
balancing strategies
● Distributed computing frameworks: Ray, Dask Distributed, Apache Spark (PySpark),
Celery
● Task queues & message brokers: Kafka, RabbitMQ, Redis Streams — at-least-once
vs. exactly-once delivery
● Cluster orchestration: Kubernetes (multi-node workload scheduling, pod autoscaling,
resource limits)
● Distributed data storage: integration with distributed filesystems (HDFS, S3, GCS),
object store access patterns
● Service mesh concepts: sidecar patterns, inter-service communication, circuit
breakers
● gRPC for high-throughput internal service communication; Protocol Buffers schema
design
● Understanding of CAP theorem, consistency models, and distributed systems failure
modes
API & Web Service Development
● Production-grade FastAPI applications: dependency injection, lifespan events,
background tasks, custom middleware
● RESTful API design: resource modeling, versioning strategies, HATEOAS principles
● Async API development: asyncio-native handlers, connection pooling, async
ORMs (SQLAlchemy 2.x, Tortoise ORM)
● WebSocket and Server-Sent Events for streaming data outputs
● API documentation: OpenAPI/Swagger, automated schema validation
● Rate limiting, request throttling, and API gateway integration (Kong, AWS API
Gateway)
● GraphQL with Strawberry or Ariadne as a plus
Security
● Authentication & authorization: OAuth2, JWT, OpenID Connect — correct
implementation patterns
● Secrets management: HashiCorp Vault, AWS Secrets Manager, environment
isolation
● Input validation and sanitization: preventing injection attacks in data pipelines
● TLS/mTLS configuration for inter-service communication
● Security scanning: bandit, safety, dependency vulnerability auditing
● OWASP Top 10 awareness applied to API and data service development
Quality Assurance & Testing
● Unit testing: pytest, fixtures, parametrize, plugins ecosystem
● Property-based testing: Hypothesis for numerical and algorithmic code
correctness
● Integration testing: containerized test environments with testcontainers
● Load & performance testing: Locust, k6 — baseline regression detection
● Data quality testing: Great Expectations, schema validation, statistical drift
detection
● CI/CD pipeline integration: automated test gates, coverage thresholds, linting (ruff,
mypy, pylint)
● Contract testing for microservices: Pact or similar consumer-driven contract
frameworks
Infrastructure & DevOps Collaboration
● Docker: multi-stage builds, image optimization, non-root containers
● Kubernetes: writing Helm charts, defining HPA/VPA policies, understanding resource
requests/limits
● Infrastructure as Code: Terraform or Pulumi for reproducible environments
● Observability stack: Prometheus + Grafana, distributed tracing with OpenTelemetry,
structured logging (structlog)
● Cloud platforms: AWS (EC2, ECS, Lambda, S3, MSK), GCP, or Azure —
data-workload relevant services
Nice to Have
● GPU computing: CuPy, RAPIDS cuDF/cuML, CUDA kernel fundamentals
● Experience with Rust or C++ for performance-critical Python extensions
● Knowledge of Kafka Streams or Apache Flink for real-time stream processing
● Contributions to open-source scientific Python ecosystem (NumPy, SciPy, Pandas,
etc.)
● Publications or applied research in signal processing, numerical methods, or
distributed systems
The Ideal Candidate Profile
Dimension What We Expect
Mathematical depth Comfortable deriving, not just applying algorithms
Performance mindset Measures first, optimizes second — always with data
Systems thinking Understands failure modes, bottlenecks, and scale limits
Code quality Writes code that is correct, readable, testable, and fast
Ownership Drives features from design to production to monitoring
Communication Articulates complex technical decisions to mixed audiences