Responsibilities
- Design and build scalable data pipelines from ingestion to consumption using Databricks
- Deploy and maintain a Lakehouse environment leveraging Delta Lake table formats
- Create efficient data transformations using PySpark and SQL for large datasets
- Maintain high standards of data accuracy, consistency, and system performance
- Lead efforts in diagnosing system issues, identifying root causes, and improving efficiency
- Work closely with solution architects, business analysts, and data consumers
- Support continuous integration and deployment workflows for Databricks assets
- Guide less experienced team members and conduct code reviews for quality assurance
- Enforce compliance with data security, governance, and regulatory requirements
Work Arrangement
Hybrid
