Data engineer (Freelance)

You have experience as data engineer working with Databricks (preferrably hosted in AWS)

Your responsibilities will be to :

• Design, develop and maintain scalable data pipelines on the Databricks platform
• Manage data ingestion processes from various sources into the Databricks environment (xml ingestion, Kakfa, etc.).
• Implement and optimize ETL (Extract, Transform, Load) processes to ensure efficient data transformation and integration.
• Ensure data quality, integrity, and security across all data pipelines.
• Monitor and troubleshoot data pipeline performance, implementing necessary maintenance and updates.
• Document data pipeline processes, technical specifications and best practices.
• Realize knowledge transfer towards the Data engineer team which will take over the responsibility to run the platform in the future

Being fluent in either English / Dutch or English / French is a MUST