Responsibilities
- Design and implement cloud infrastructure using IaC – Terraform
- Automate provisioning and management of Dataproc clusters, Dataflow jobs, and other GCP resources
- Integrate tools like GitLab CI/CD, or Cloud Build for automated testing and deployment
- Deploy and manage real-time and batch data pipelines using Dataflow or DataStream
- Ensure seamless integration of data pipelines with other GCP services like Big Query, Cloud Storage, and Kafka or Pub/Sub
- Monitor performance, reliability, and cost of Dataproc clusters, Dataflow jobs, and streaming applications
- Optimize cloud infrastructure and data pipelines for performance, scalability, and cost-efficiency
- Implement security best practices for GCP resources, including IAM policies, encryption, and network security
- Ensure Observability is an integral part of the infrastructure platforms and provides adequate visibility about their health, utilization, and cost
- Collaborate extensively with cross functional teams to understand their requirements; educate them through documentation/trainings and improve the adoption of the platforms/tools
Additional Information
- This is a remote based position in US
- The base pay range for this position varies based on the geographic location
- Individual pay is determined based on location of residence and multiple factors, including job-related knowledge, skills and experience
- San Francisco Bay Area: 156,400 - 265,700 USD Annual
- All Other US Locations: 136,000 - 231,000 USD Annual
- This role may be eligible for a bonus
- For information on our benefits click here
