Key Responsibilities



Specification-Driven Extraction Engineering:

1. Design and maintain declarative extraction specifications—using Pydantic models, JSON schemas, or domain-specific languages—that describe exactly which fields to capture, their types, and validation rules.

2. Implement pipelines that translate these specifications into executable extraction plans, leveraging both classical (Scrapy, Playwright) and AI-augmented (LLM-based semantic parsing) backends.

3. Build reusable specification libraries for recurring data types (product prices, tariff codes, regulatory texts) to accelerate onboarding of new sources.

4. Design and implement autonomous data extraction agents that can make decisions about source selection, retry logic, and parsing strategies



Autonomous & Self-Healing Systems:

1. Deploy self-healing spiders that automatically detect website layout changes and repair themselves using Model Context Protocol...