I KNOW HOW S.A is hiring a Data Engineer

Responsibilities

• Design, construct, and sustain scalable batch and real-time data pipelines using Azure Data Factory, Azure Synapse, and Databricks, sourcing data from policy systems, claims databases, CRM platforms, and third-party providers.
• Build and manage structured data models following a medallion architecture (bronze, silver, gold) with Delta Lake to support data integrity, traceability, and reuse across analytical and machine learning applications.
• Create and refine SQL and PySpark logic to process large-scale datasets efficiently, prioritizing performance, cost-effectiveness, and system reliability.
• Automate pipeline operations using Azure DevOps or GitHub Actions, integrating continuous integration, testing, deployment, and monitoring into the development lifecycle.
• Enforce data quality standards through validation rules, monitoring systems, and alerting mechanisms for key data assets, collaborating on metadata management and lineage tracking via tools like Microsoft Purview or Unity Catalog.
• Partner with data architects to ensure pipeline designs comply with enterprise data models and governance policies, including handling of personally identifiable information, data retention rules, and access permissions in line with insurance regulations.
• Collaborate with analytics, actuarial, and data science teams to convert business needs into reliable, reusable data products and self-serve datasets.
• Engage in code reviews, architectural discussions, and Agile rituals to support engineering excellence and ongoing enhancement of the data infrastructure.

Team

• Team size: 300+ professionals
• Structure: Group structured into specialized subsidiaries focused on distinct technology domains and market verticals.