Help shape the backbone of an intelligent research platform by designing and managing robust data systems that power next-generation decision tools. In this role, you'll develop and maintain ETL workflows that process vast volumes of research content, ensuring timely and accurate delivery. You'll implement embedding pipelines that enable semantic search, allowing users to uncover insights through natural language queries.
Key Responsibilities
- Design and automate data pipelines to ingest, transform, and serve research content at scale
- Build and refine document processing systems for customer-submitted materials, including text extraction, segmentation, and citation tracking
- Optimize search performance through intelligent indexing, ranking models, and retrieval accuracy
- Develop infrastructure that supports AI-driven queries with structured, trustworthy responses
- Monitor system health and data integrity, proactively resolving issues to maintain platform reliability
What You Bring
- Minimum of three years in data or search engineering, with hands-on experience in ETL and data warehousing
- Strong proficiency in Python and SQL
- Proven experience with semantic search, embedding models, and retrieval systems
- Background in document processing, content transformation, or text extraction
- A meticulous approach to data quality—committed to accuracy, consistency, and completeness
Preferred Expertise
- Experience with Snowflake or Snowflake Cortex
- Working knowledge of vector databases
- Familiarity with information retrieval or natural language processing techniques
- Experience enforcing data access controls
- Exposure to pipeline observability and data quality monitoring tools
Work Environment
This position follows a hybrid model, requiring approximately 1–2 days per month in-office. You’ll work within a globally recognized research and advisory environment that values intellectual rigor, collaboration, and long-term career growth. The culture encourages curiosity, shared success, and engagement with meaningful, challenging work.
IDC is an equal opportunity employer committed to diversity and inclusion. Employment verification is required. The company participates in E-Verify.
