Job Description
Develop and maintain scalable data pipelines using Databricks, PySpark, Python, and SQL.
Build and manage ETL/ELT processes for data ingestion and transformation.
Work with Apache Spark and Delta Lake.
Develop and manage Databricks workflows and jobs.
Perform data validation, quality checks, and pipeline optimisation.
Work with cloud platforms such as AWS/Azure.
Collaborate with data engineers, architects, and business teams.
Troubleshoot and resolve data pipeline issues.
Required Skills
Databricks, PySpark, Apache Spark, Python, SQL, Delta Lake, ETL/ELT, Cloud (AWS/Azure).