Belcan

HireArt is partnering with an AI innovation initiative seeking DevOps Engineers to support the evaluation and improvement of software engineering-focused AI tasks.

In this role, you’ll use your engineering and systems-level reasoning to assess model competency in real-world technical domains, debugging, architecture decisions, and system behavior analysis.

What You’ll Do

• Create challenging SWE-related prompts based on visuals, datasets, or model outputs.
• Evaluate AI-generated responses using provided rubrics.
• Develop or improve rubrics to ensure they reflect accurate reasoning for complex tasks.
• Review and stress-test model outputs for accuracy and logical consistency within your domain.
• Provide clear, structured feedback based on your subject-matter knowledge.
• 5+ years’ relevant DevOps engineering experience
• Prior exposure to LLMs or AI tools, ideally for use in technical domains
• Strong writing, analysis, and reasoning skills
• Strong English language skills.
• Comfortable working asynchronously with strong self-direction
• Ability to deliver structured, precise, and logically sound evaluations
• Reliable computer and internet connection
• Bachelor’s degree in computer science or related field is a plus, but not required.

About the Work

• This is remote, task-based, independent contractor work.
• Set your own schedule; ideal candidates would be able to work at least 10 hours per week on average.
• Pay is $15 per accepted task, with typical contributors completing 4–6 tasks per hour, depending on task complexity.
• Successful contributors who work at least 10 hours per week are anticipated to yield $45-$75 per hour.

Commitment: This is an independent contract position staffed via HireArt. The role is remote and open to U.S.-based candidates (excluding California and Illinois) who can commit to 10-40 hours per week. Workers should be able to work 5-7 hours per day and may set their own schedules.

HireArt values diversity and is an Equal Opportunity Employer. We are interested in every qualified candidate who is eligible to work in the United States. Unfortunately, we are not able to sponsor visas or employ corp-to-corp.