About the Role
The Senior Spark Data Engineer will develop and maintain scalable data processing systems using Apache Spark, working closely with analytics and engineering teams to ensure high-quality, reliable data pipelines in a cloud-based environment.
Responsibilities
- Design and implement robust data pipelines using Apache Spark
- Optimize large-scale data processing workflows for performance and reliability
- Collaborate with data scientists and analysts to understand requirements
- Develop and maintain ETL processes across diverse data sources
- Ensure data quality, integrity, and consistency across systems
- Troubleshoot and resolve issues in production data environments
- Support integration of data platforms with cloud infrastructure
- Participate in code reviews and contribute to engineering best practices
- Monitor data pipeline performance and implement improvements
- Document technical designs and system architecture decisions
- Work with streaming data using real-time processing frameworks
- Apply software engineering principles to data infrastructure
- Contribute to the design of data models and storage strategies
- Maintain security and compliance standards in data handling
- Assist in capacity planning for data systems
- Evaluate and adopt new tools and technologies for data processing
- Support deployment automation and CI/CD pipelines
- Collaborate on data governance initiatives
- Provide technical guidance to junior team members
- Ensure systems are scalable, fault-tolerant, and maintainable
- Work with structured and unstructured data formats
- Integrate data from multiple sources into unified platforms
- Participate in architectural discussions and planning
- Contribute to reliability and monitoring of data services
- Support disaster recovery and backup strategies for data systems
Nice to Have
- Master's degree in a technical or quantitative field
- Experience with machine learning pipelines
- Familiarity with data governance frameworks
- Knowledge of streaming platforms like Kafka or Flink
- Experience with infrastructure as code tools
- Background in financial or enterprise data environments
- Certifications in cloud or data engineering platforms
- Contributions to open-source data projects
- Publications or presentations in data engineering topics
- Leadership experience in technical projects
Compensation
Competitive salary based on experience and qualifications
Work Arrangement
Hybrid work model with flexibility for remote and office-based work
Team
Part of a cross-functional engineering team focused on data solutions and platform development
About the Role
This position focuses on building and maintaining high-performance data infrastructure using modern distributed computing tools. The engineer will play a key role in shaping how data is processed, stored, and made available across the organization.
Technology Environment
The team works with Apache Spark, cloud data platforms, containerized services, and automated deployment pipelines. Technologies may include Databricks, Kafka, Kubernetes, and various cloud-native data services.
Professional Growth
Engineers are encouraged to pursue certifications, attend industry events, and contribute to internal knowledge sharing. Career development paths include technical specialization and leadership opportunities.
Work Culture
The environment values collaboration, innovation, and continuous improvement. Engineers work in agile teams with regular feedback cycles and transparent communication.
Project Impact
Projects span multiple business domains, enabling data-driven decision-making through scalable and reliable data solutions.
Available for qualified candidates requiring sponsorship
