368,657open jobs
9,442companies
50,883added this week
Browse all
Salary
$120k – $180k per year
Location
Remote (United States, United Arab Emirates, India, Pakistan)
Seniority
Junior · 1+ year exp
Employment
Part-Time
Overview
Company
Impact
Profile match
At Weekday, we help companies hire engineers who are vouched by other software engineers. We are enabling engineers to earn passive income by leveraging & monetizing the unused information in their head about the best people they have worked with.

Job Description:

  • Design realistic, multi-step software engineering challenges based on practical development workflows, including implementation, debugging, system configuration, testing, and documentation.
  • Develop comprehensive reference solutions in Python, ensuring each task is reproducible, verifiable, and supported by appropriate validation checks.
  • Leverage AI coding assistants as part of your development workflow while evaluating their strengths, limitations, and failure modes on complex engineering tasks.
  • Review benchmark tasks created by fellow experts, providing detailed feedback on technical correctness, clarity, completeness, and level of difficulty.
  • Analyze AI-generated solutions to identify implementation errors, debugging challenges, reasoning gaps, and opportunities for improving benchmark quality.
  • Collaborate closely with AI researchers and engineering teams to refine evaluation methodologies and maintain consistently high technical standards.

Requirements:

  • Master's degree, PhD, or equivalent practical experience in Computer Science, Software Engineering, or another STEM discipline involving substantial programming and technical problem solving.
  • Minimum 1 year of professional experience in software engineering, research engineering, applied research, or another technically intensive engineering role.
  • Strong hands-on proficiency in Python, including software design, scripting, debugging, testing, and writing clean, maintainable code.
  • Solid experience using Git, integrated development environments (IDEs), and modern software development workflows.
  • Strong understanding of software architecture, debugging methodologies, and engineering best practices.
  • Experience using AI coding assistants, prompt engineering techniques, or AI agent workflows is preferred.
  • Experience with AI evaluation, benchmark development, AI training, or task authoring is highly desirable.
  • Excellent analytical thinking, creativity, attention to detail, and the ability to solve complex, open-ended technical problems independently.
  • Strong written communication skills for documenting technical solutions and engineering decisions.
  • Ability to commit approximately 35 hours per week on a consistent basis.

Benefits:

    Free account
    Stop reading job ads. Get the ones that fit.
    One free account turns this page into a shortlist built around your stack, your level and your pay.
    Match on every job. Stack, seniority, pay and location, scored against your profile.
    368,657 open roles. Read straight off company career pages, refreshed every day.
    Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
    3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
    Create a free account
    Free forever. No card. Under a minute.

    Your match

    How well do you fit this role?
    Two answers are enough for a real match. No account needed.
    Check my fit
    Answers stay in this browser until you create an account.

    Recommended for you based on this role

    Similar stack
    Same company
    United States
    $53k – $70k per year • Remote/Hybrid • Internship • Bachelor's Degree • Boston
    JavaScript
    Python
    DevOps
    Bitbucket
    Git
    Apply
    $53k – $70k per year • Remote/Hybrid • Internship • Bachelor's Degree • Boston
    Java
    Python
    SQL
    Databases
    Databricks
    AI/ML
    Spark
    DevOps
    AWS
    Azure
    Bitbucket
    GCP
    Git
    Analytics
    ETL/ELT
    Apply
    $79k – $100k per year • Remote/Hybrid • Full-Time • Bachelor's Degree • New York
    Python
    Analytics
    Power BI
    Apply
    $82k – $180k per year (Estimated) • In office • Full-Time • 7+ years exp • Bachelor's Degree • Toronto
    C#
    Java
    Python
    SQL
    TypeScript
    JavaScript
    C#
    .NET
    Java
    Spring Boot
    Frontend
    Angular
    DevOps
    Azure
    CI/CD
    Rest API
    Analytics
    Power BI
    ETL/ELT
    Apply
    $200k – $250k per year • Equity 0.1–0.5% • In office • Full-Time • 3+ years exp • New York
    TypeScript
    Databases
    PostgreSQL
    AI/ML
    AI Agents
    AWS Bedrock
    Claude
    Claude Code
    LLM
    OpenAI
    Function Calling
    Frontend
    React.js
    Vite
    DevOps
    AWS
    Apply
    $32k – $77k per year (Estimated) • Remote • Full-Time
    Python
    AI/ML
    AI Agents
    AutoGen
    CrewAI
    Embeddings
    Fine-tuning
    Function Calling
    Hallucination
    LangGraph
    LangSmith
    Llama
    LLM
    LoRA
    Mistral
    MLFlow
    PEFT
    Prompt Engineering
    RAG
    RLHF
    Weights & Biases
    LangChain
    Transformers
    LLM Guardrails
    LLMOps
    DevOps
    AWS
    Azure
    GCP
    Vector
    Apply
    In office • Full-Time • Bengaluru
    C++
    C
    C++
    CMake
    Qt
    C
    Valgrind
    DevOps
    Git
    Apply
    AI Lead (Technical) 13 days ago
    $32k – $77k per year (Estimated) • In office • Full-Time • Mumbai
    Java
    Python
    Scala
    SQL
    Databases
    Databricks
    AI/ML
    DVC
    Hadoop
    Kubeflow
    MLFlow
    Prompt Engineering
    PyTorch
    RAG
    Scikit-learn
    Spark
    TensorFlow
    Vertex AI
    Weights & Biases
    XGBoost
    Amazon SageMaker
    LLMOps
    DevOps
    AWS
    Azure
    CI/CD
    Datadog
    Docker
    GCP
    Git
    Grafana
    Kubernetes
    Platform Engineering
    Prometheus
    Analytics
    ETL/ELT
    Apply
    $70k – $126k per year • Remote • Full-Time • 3+ years exp • United States
    Python
    DevOps
    Azure
    Azure DevOps
    Bitbucket
    CI/CD
    Docker
    GitHub
    GitLab
    GitLab CI
    Jenkins
    Kubernetes
    Cybersecurity
    SonarQube
    QA
    Postman
    Robot Framework
    Selenium
    Apply
    $58k – $105k per year • Remote • Full-Time • United States
    JavaScript
    Python
    DevOps
    Bitbucket
    CI/CD
    GitHub
    GitLab
    GitLab CI
    Jenkins
    Cybersecurity
    SonarQube
    QA
    Robot Framework
    Apply
    $131k – $237k per year • Remote/Hybrid • Full-Time • 12+ years exp • Bachelor's Degree • United States
    DevOps
    AWS
    Azure
    GCP
    Apply
    $108k – $195k per year • In office • Full-Time • 8+ years exp • Bachelor's Degree • United States
    Python
    AI/ML
    AI Agents
    Amazon SageMaker
    AWS Bedrock
    AWS Bedrock AgentCore
    LLM
    DevOps
    AWS
    CI/CD
    CloudFormation
    Docker
    IAM
    Kubernetes
    Platform Engineering
    Terraform
    Cybersecurity
    FedRAMP
    Apply
    $131k – $237k per year • Remote/Hybrid • Full-Time • 10+ years exp • Bachelor's Degree • United States
    PowerShell
    Python
    DevOps
    Ansible
    CI/CD
    Configuration Management
    GitOps
    Terraform
    VMWare
    Management
    Jira
    Apply
    See all jobs
    This is one of many
    368,657 more open roles from verified company boards, updated every day.