Hybrid job
HybridSenior AI Engineer (Quality & Knowledge)
PayFit
Job briefing
Read from the posting and checked against our job index
- Office in
- Paris, France
- Work model
- Hybrid, partly on site
- Salary
- Not stated
Good to know
- Listed with us since
- November 20, 2026
- Employer
- 5 more open roles at PayFit in our index.
Generated automatically by nomado24 from the posting and our index. Not an employer statement. The original posting is authoritative.
Job description
Get to know us
Do you want to help us shape what the future of work will look like and how it will best embrace our life's aspirations? If this sounds like a journey you want to embark on, we may have the right role for you! PayFit is an intuitive software as a service payroll and HRIS solution designed specifically for SMBs. Since 2015, we have set ourselves a mission to simplify payroll for SMBs and enable employers and employees to grow together. We are a European company operating from 3 main countries (France, Spain, and the UK) where we are supporting over 20,000 clients.
Creating a fulfilling work environment and culture is also a core mission at PayFit, and our day-to-day work philosophy is reflected in our four values:
Care: We genuinely care for others whoever they are, whatever they think.
Excellence: We aim to improve and achieve better results every day.
Humility: Staying humble and exchanging transparent feedback helps us to grow and improve.
Passion: We are the architects of PayFit's success.
A key part of our culture , and essential for our success, is also improving the diversity of our teams and building an inclusive culture where you can be yourself at work.
This is why our recruitment focuses on the skills you demonstrate, and not only on your academic background or previous professional experiences. At PayFit we understand that you can gain applicable skills through a variety of life experiences and we are interested in knowing them, too.
Location :
Flexible location: work remotely from anywhere in France or Spain
The team
You'll join the team behind Payfit AI , building AI agents that help companies prepare, check, and run payroll, so people can focus on the decisions that need their expertise.
We work closely with Customer Support and payroll experts to understand real customer situations and turn that expertise into useful product experiences. These span a wide range of payroll and HR topics, from everyday questions to complex cases in a highly regulated environment.
You'll contribute to connecting that expertise with the systems we build. Using real cases, you'll help evaluate the whole workflow: did the agent give the right answer, take the right action, or know when to ask for clarification or involve a human? Where a workflow affects payroll, evaluating its quality also means checking the resulting outcome.
This is a hands-on senior engineering role. You'll build on existing models, agents, evaluation tools, and knowledge pipelines. Depending on what you find, you might improve the knowledge base, create a specialized agent, connect a new tool, or evolve the agent harness: how agents receive context, use tools, apply guardrails, and hand off to a person. You'll use evaluations and production feedback to guide these improvements, measure their impact, and catch regressions as models, payroll rules, and workflows evolve.
What you'll do
- Work with Customer Support and payroll experts to understand how they resolve complex cases and turn that expertise into testable AI behavior.
- Build and improve evaluations using representative cases, expert reviews, and production feedback to assess answers, actions, and workflow outcomes.
- Investigate failures across models, context, knowledge, retrieval, and tools, then implement and measure improvements.
- Contribute to specialized agents, orchestration, and harness engineering, including context management, reusable tools, guardrails, and human handoffs.
- Improve knowledge pipelines and retrieval quality, and use regression evaluations and observability to keep workflows reliable as they evolve.
What we're looking for:
- 7+ years of software engineering experience , including designing and operating backend services or pipelines in production.
- Hands-on experience building LLM applications or agents , including tool integration, context management, and handling uncertainty.
- Practical experience evaluating LLM systems using representative datasets, clear criteria, and automated or human review—and using the results to guide improvements.
- Strong backend engineering skills and the ability to contribute to a TypeScript/Node.js codebase.
- Comfort working directly with domain experts, translating ambiguous cases into testable behavior, and collaborating on reliable production solutions.
- Care for security, permissions, observability, and maintainability.
Nice to have
- Experience with Langfuse or similar tools for tracing, prompt management, datasets, and experiments.
- Familiarity with AI SDK or comparable frameworks for building tool-using agents.
- Experience with hybrid search, reranking, OpenSearch, or other search and vector technologies.
- Experience measuring judge agreement, evaluation uncertainty, or agent consistency across repeated runs.
- Experience building annotation tools or review workflows for domain experts.
- Familiarity with AWS, data pipelines, and CI/CD for AI systems.
- …
This role is provided by an external source. Applications are handled on the source website.
