πŸ”” Get instant job alerts delivered to your inbox! Set up your first alert β†’
πŸ“ Local Job Near You

Senior AI Quality Engineer (LLM Evaluation & Automation) 1754

🏒
SOFTGIC
πŸ“ Medellin, Colombia
πŸ“
Location Medellin
πŸ“…
Posted June 26, 2026
πŸš—
Commute Local Area
🎯
Local Opportunity Near You!

This job is in your area. Enjoy a short commute and work close to home.

πŸ“‹
Job Description

Este es un puesto de trabajo remoto. Owns the eval harness and quality gate from the beginning. This role replaces the old late-stage Evals Specialist model with a standing owner for measurable agent quality. Key Responsibilities Build and maintain the MVP eval harness: golden tasks, exception tasks, scorecard metrics, and regression packs. Wire evals into CI so quality regressions fail builds and releases. Define and maintain release-gate thresholds with Product and the Tech Lead. Lay the path for later adversarial and drift-testing expansion without overbuilding MVP scope. Requisitos Must-Have Qualifications Experience evaluating ML, LLM, or non-deterministic systems. Strong test and benchmark design capability. Comfort working with noisy metrics, thresholds, and probabilistic behavior. Good scripting and automation skills. AI-First Expectations Uses AI to generate candidate eval cases and failure hypotheses, but never confuses generated tests with validated quality. Approa...

Apply for This Job

Submit Application

Quick and secure application process

πŸ“ Location Details

πŸŒ†
City
Medellin
πŸ—ΊοΈ
Country
Colombia
πŸš—
Commute
Local Area

πŸ” More Jobs Nearby

Explore other opportunities in Medellin

View Local Jobs