Job Description

Role Overview

Create challenging codebase exploration tasks that help train AI agents to investigate and reason about unfamiliar C software repositories. You will develop high level engineering questions that require evidence from runtime behavior and investigation across multiple files, similar to the work a senior engineer performs when diagnosing a foundational system.

The work covers production C codebases including PostgreSQL, Redis, git, OpenSSH, OpenSSL, curl, libuv, libgit2, libarchive, Open vSwitch, tcpdump, tmux, and zstd.

Key Responsibilities

• Select assignments from a prevalidated set of C repositories, each fixed to a specific git commit and supplied with an engineering question, positive and negative rubrics, foils, and a reference solution.
• Investigate the relevant subsystem and associated pull request or issue history until you understand its behavior.
• Rewrite questions to challenge advanced coding agents while keeping them precise and fairly answerable.
• Refine rubrics and foils to recognize sound reasoning and reject credible but incorrect answers.
• Complete the required validation workflow, Check, Golden, Run, and Analyze, revising the task until each stage passes.
• Document work in the task README, submit it for expert review, and address feedback on returned tasks.

Qualifications

• 3+ years of professional software engineering experience, including substantial production C experience.
• Ability to navigate large, unfamiliar C codebases and explain runtime behavior, not only source code structure.
• Comfort with preprocessor-heavy builds, platform conditionals, manual memory management, and autotools, CMake, or Meson build graphs.
• Experience with databases, cryptography, networking protocols, or systems programming is highly relevant. The task mix emphasizes architecture and system design, 39%, and code onboarding, 34%, with a meaningful security component.
• Ability to use debuggers, sanitizers, tracing tools, and profilers to establish actual program behavior.
• Precise written English, as the questions and rubrics you produce are central deliverables.

Work Terms

• Remote, asynchronous, per-task contract work with expert team leads across US, India, and Nigeria time zones.
• Work is completed in a designated online workspace; time is tracked with Insightful and access is provisioned through Okta.
• Before receiving task access, candidates must complete identity verification, a background check, signed Terms of Work and CIIAA, a tax form, and a short calibration quiz.
• Each completed task is reviewed by a second engineer and a senior reviewer before acceptance.

Compensation

$130 per approved task, paid after the task is accepted through review.

Application Process

Submit a resume or relevant technical background. Qualified applicants may be asked to complete a brief technical assessment or provide additional information.

Job Tags

Similar Jobs