Responsibilities
- Lead the design and governance of enterprise-wide data ingestion and integration architecture.
- Develop and refine reusable, scalable data interfaces across diverse source systems and business areas.
- Serve as technical authority for data platform components, ensuring high standards in reliability, scalability, and code quality.
- Define and enforce engineering practices including CI/CD, testing, observability, and architectural patterns.
- Act as the go-to expert for resolving complex technical issues in data integration and platform operations.
- Mentor and guide software and data engineers building production-ready data services and platform modules.
- Partner with architecture, security, infrastructure, and business IT teams to ensure compliant and secure designs.
- Promote consistency in engineering methods, frameworks, and architectural patterns across the data platform.
- Design and enhance ingestion frameworks supporting batch, streaming, API-based, and event-driven data flows.
- Set standards for handling structured and unstructured data formats, including schema evolution and compatibility.
- Architect and evaluate ingestion patterns with emphasis on fault tolerance, monitoring, and performance.
- Enforce API design principles, reliability mechanisms, and contract management practices.
- Lead technical reviews for complex data pipelines, focusing on scalability, security, and long-term maintainability.
- Assist teams in implementing error recovery, retries, workflow orchestration, and system observability.
- Work with system owners to define integration specifications and data exchange strategies.
- Ensure ingestion solutions align with overall platform architecture, governance policies, and business needs.
- Enable data pipelines that support advanced analytics, machine learning, and AI/LLM use cases.
- Build and maintain reusable libraries, templates, and framework components to speed up development.
- Champion code quality through reviews, automated testing, and continuous integration practices.
- Document and maintain architectural blueprints, design principles, and engineering standards.
- Create standardized patterns for lakehouse-based data platforms to ensure consistency and efficiency.
- Advise on strategies for scaling systems, optimizing resource usage, and improving performance.
- Support cost visibility and efficient consumption of platform resources through thoughtful design and monitoring.
