Research Engineer
Posted 31 Aug 2026
Advertised as “Research Engineer - Pre-training”
BranchFactor Summary
The original posting with the fluff stripped out
- Management level
- Individual contributor
- Employment
- Remote
- Contract
- Full-time
- Visa sponsorship
- Offered
No salary listed for this role.
Pluralis Research works on Protocol Learning: training and serving large models in a fully decentralized way on small consumer-grade devices connected via the internet. Despite being dismissed as infeasible, we have made significant advances on this problem, most recently Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs spread over the internet, with no single participant ever holding the full weights (tech report). While many of the core research problems have been solved, Protocol Learning unlocks a series of new challenges. For the mission in full, read A Third Path: Protocol Learning.
This setting breaks nearly every assumption of datacenter training: communication-efficient training across different parallelism axes, fault tolerance as nodes join and drop mid-run, heterogeneous compute and networks, and robustness to malicious participants. Our published methods include Subspace Networks, Factored Gossip DiLoCo, AsyncMesh, and Sentinel.
As a Research Engineer you'll build the training system that takes Protocol Learning from the 8B run to frontier scale: large models on heterogeneous hardware, in physically different regions, connected by ordinary internet.
Key Responsibilities
-
Distributed pretraining: Implement and optimize model-parallel training. Data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links.
-
Performance optimization: Implement techniques that reduce communication overhead while maintaining model convergence in challenging network environments.
-
Elasticity and fault tolerance: Make runs survive node churn. Robust checkpointing, state synchronization, and recovery as participants join and leave.
-
Run instrumentation: Build the monitoring that shows throughput, bottlenecks, and model quality across hundreds of devices.
What We're Looking For
-
Hands-on distributed training (required): You've trained models across many devices in PyTorch with FSDP, DeepSpeed, Megatron, or your own implementation. You understand data, tensor, and pipeline parallelism.