Data Engineer
US$100 – US$300 / hr base salary
Posted 20 Oct 2025
Advertised as “Data/Full Stack Engineer, Data Storage & Ingestion Consultant”
BranchFactor Summary
The original posting with the fluff stripped out
- Management level
- Individual contributor
- Employment
- Hybrid
- Contract
- Full-time
| Base salary | US$100 – US$300 / hr |
|---|
About us
At Eon, we are at the forefront of large-scale neuroscientific data collection. Our mission is to enable the safe and scalable development of brain emulation technology to empower humanity over the next decade, beginning with the creation of a fully emulated digital twin of a mouse.
Role
We’re a San Francisco team collecting very large microscopy datasets and we need an expert to design and implement our end-to-end data pipeline, from high-rate ingest to multi-petabyte storage and downstream processing. You’ll own the strategy (on-prem vs. S3 or hybrid), the bill of materials, and the deployment, and you’ll be on the floor wiring, racking, tuning, and validating performance.
Our current instruments generate data at ~1+ GB/s sustained (higher during bursts) and the program will accumulate multiple petabyes total over time. You’ll help us choose and implement the right architecture considering reliability and cost controls.
Outcomes (what success looks like)
-
Within 2 weeks: Implement an immediate data-handling strategy that reliably ingests our initial data streams.
-
Within 2 weeks: Deliver a documented medium-term data architecture covering storage, networking, ingest, and durability.
-
Within 1 month: Operationalize the medium-term pipeline in production (ingest → buffer → long-term store → compute access).
-
Ongoing: Maintain ≥95% uptime for the end-to-end data-handling pipeline after setup.
Responsibilities
-
Architect ingest & storage: Choose and implement an on-prem hardware and data pipeline design or a cloud/S3 alternative with explicit cost and performance tradeoffs at multi-petabyte scale.
-
Set up a sustained-write ingest path ≥1 GB/s with adequate burst headroom (camera/frame-to-disk), including networking considerations, cooling, and throttling safeguards.
-
Optimize footprint & cost: Incorporate on-the-fly compression/downsampling options and quantify CPU budget vs. write-speed tradeoffs; document when/where to compress to control $/PB.
-
Integrate with acquisition workflows ensuring image data and metadata are compatible with downstream stitching/flat-field correction pipelines.
-
Enable downstream compute: Expose the data to segmentation/analysis stacks (local GPU nodes or cloud).