Europe Arbeitnow ·
full-timeSenior Software Engineer - Paris
Hcompany · Hybrid Paris
Hcompany develops computer‑use agents and the underlying models that power them, exposing the functionality through a managed API for enterprise integration. The senior software engineer will own and extend the evaluation framework that runs hundreds of benchmarks across web, desktop and command‑line applications. Day‑to‑day work includes integrating new benchmarks, defining standards and checks, scheduling jobs on Kubernetes clusters, ensuring observability and reproducibility, and optimizing performance and cost. The role requires building and maintaining tooling that researchers and forward‑deployed engineers rely on to assess model quality and guide product releases. Candidates should have five or more years of backend development experience with production‑grade Python, strong familiarity with distributed systems on Kubernetes and public cloud (AWS preferred), and a track record of delivering test, QA or evaluation infrastructure. Experience with APIs (REST/GraphQL), relational and non‑relational databases, message queues, Docker, and monitoring tools such as Grafana or Datadog is essential. Knowledge of LLM evaluation, agents, or automation frameworks like Playwright or Selenium is a plus but not mandatory.
About H: When we released Holo4 on 28 September, we published every trajectory behind its public benchmark scores at trajectories.hcompany.ai . For OSWorld that's 369 desktop tasks, each run 3 times, with the steps, tokens and time of every attempt. This role builds and runs the evaluation framework that produces runs like these. H builds computer-use agents and the models behind them. Developers use them through a managed API , and our forward deployed engineers take them into enterprise workflows. What this team owns The evaluation framework: orchestration, runtimes and observability. Researchers and forward deployed engineers bring the benchmarks, across web apps, desktop applications and the command line. Your job is to make the framework that runs them reliable, fast and cheap, and to make adding a new one quick. Research uses the results to choose checkpoints and decide whether a model ships. Product and the forward deployed engineers use them to measure agents on customer workflows. It carries roughly 50 benchmarks now. That number should be between 100 and 200 soon, and the framework has to keep up. What you'd be doing Integration support for researchers and forward deployed engineers bringing in a benchmark, with a shorter path each time. Setting the standard for how a benchmark enters the framework, and building the checks that enforce it. Scheduling and observability, so cluster capacity isn't left idle while evaluation jobs queue. Reproducible results across trials, so a release decision rests on numbers that hold. Whatever stack a benchmark calls for. One week that's cluster tuning; the next it's a browser extension or desktop environments. Time with customers, from single developers to large companies, to find out what they want measured, then automating it so the results flow back into our harnesses and models. The first few months By 3 months you'll have helped researchers or forward deployed engineers integrate 5 benchmarks, and started fixing what slows the framework down. By 6 months one part of it is yours, for example scaling the runs, observability, or a group of related benchmarks, and a release will have gone out on your numbers. By 12 months you'll know the design and trade-offs of the whole evaluation system, and be the person the rest of H asks about evaluations. Who you'd work with Ceiran Chapman, our VP Engineering, is hiring for this role. You'd join the evaluation team. The people relying on your work day to day are H's researchers and forward deployed engineers. What we think it takes Likely a good fit if you Have spent 5+ years in backend development, with production Python at the core, and use coding agents to go faster without letting quality drop. Have built test, QA or evaluation tooling that other teams depended on, and care whether a number is right. Have operated distributed systems on Kubernetes in a public cloud. AWS experience helps most. Have built and shipped systems end to end, including APIs (REST or GraphQL) and integrations with outside services. Know relational and non-relational databases, and message queues such as SQS, RabbitMQ or Kafka. Instrument what you build, with metrics, tracing and monitoring from the start. Stronger still if you have Measured LLM quality before, or built agents yourself. Packaged and run workloads in Docker and on virtual machines. Used Temporal, Dask, FastAPI, PostgreSQL, Grafana or Datadog. Automated web or desktop software with Playwright, Selenium or a browser extension you wrote. Set standards other engineers follow, through code review, design review or mentoring. You do not need a background in machine learning. We'll work that out with you. If you match most of this but not all of it, apply anyway. How we hire A 30 minute call with our Talent team, a 60 minute technical challenge, a 60 minute system design interview, and a 30 minute final conversation with Ceiran. About 3.5 hours in total. Practicalities Paris posting: Hybrid in Paris. That means 3 office days a week and a London trip about once every 4 to 6 weeks. There is a London posting for the same role. We offer a competitive package. Find Jobs in France on Arbeitnow
| Role | Senior Software Engineer - Paris |
|---|---|
| Company | Hcompany |
| Location | Hybrid Paris |
| Type | full-time |
| Compensation | Not disclosed |
| Deadline | Rolling |
Typical process for this type of role
A general guide — the exact steps for this specific listing may vary; check the original posting for details.
- 1ApplicationSubmit your resume through the apply link.
- 2ScreeningRecruiter reviews your background against the role.
- 3AssessmentA technical test, assignment, or coding round, depending on the role.
- 4Interview(s)One or more rounds with the hiring team.
- 5OfferOffer letter with compensation and start date.
Before you apply
0/4More at Hcompany
Other jobs at Hcompany
- Founder's Associate - CEO Office · Hybrid Paris
- Social Media And Content Strategist - Community · Hybrid Paris
- Digital Manager · Hybrid Paris
- Content Manager · Hybrid Paris
- Business Partner / Executive Assistant To CEO · Hybrid Paris
Explore Related Placements
// similar opportunities
You might also like
Senior Software Engineer
Affirm
At Affirm, we exist for the moments that matter—giving people a clear, predictable way to pay over time, with no hidden fees, no surprises, and no tradeoffs on ...
Senior Software Engineer
Kraken
Help us use technology to make a big green dent in the universe! Kraken powers some of the most innovative global developments in energy. We create the technolo...
Senior Software Engineer
Letly
About Letly Letly is the AI-Native operating company built for the home. We believe the home should be autonomously run and we're building the software that wil...
Senior Software Engineer
Chaosindustries
CHAOS Industries is redefining modern defense with a multi-product portfolio that gives the ultimate advantage—domain dominance. The company's products are powe...
Senior Software Engineer
Deliveroo
Our Global Structure Deliveroo is now part of DoorDash, bringing together teams with even greater reach, scale, and ambition. Depending on your role, you may co...
(senior) Software Engineer
Distran
About the role The software on our Ultrasound acoustic cameras is at the core of Distran technology, with focus on high quality, reliability, and real-time perf...