Location: Fully Remote, working on an EST schedule
Company: Apex Systems
Job Summary: We are seeking a highly skilled Data Engineer with a focus on Azure and Databricks to join our team. The ideal candidate will have over 5 years of experience in data engineering, with a strong background in the commercial airline industry, flight data, or general aerospace. Experience in manufacturing environments is preferred. The candidate will be responsible for designing, implementing, and maintaining data solutions to support our business objectives.
Key Responsibilities:
* Design and implement data pipelines using Azure Data Factory and Databricks to process and analyze flight data.
* Develop and maintain data lakes and data warehouses on Azure, ensuring data integrity and accessibility.
* Collaborate with cross-functional teams to integrate flight data into business intelligence tools, enhancing decision-making processes.
* Utilize Python and SQL to automate data processing tasks, reducing manual effort and increasing efficiency.
* Conduct performance tuning and optimization of Spark jobs to improve data processing speed.
* Provide technical support and troubleshooting for data-related issues, ensuring minimal downtime.
Required Qualifications:
* Bachelor’s degree in Computer Science, Information Technology, or a related field.
* Over 5 years of experience in data engineering.
* Strong experience with Azure Data Factory, Azure Data Lake, and Azure Synapse Analytics (formerly Azure SQL Data Warehouse).
* Proficiency in Python and SQL.
* Experience with Databricks and Apache Spark.
* Industry experience in the commercial airline sector, with a focus on flight data or general aerospace.
* Manufacturing experience is preferred.
Preferred Qualifications:
* Experience with Git, Docker, and Kubernetes.
#J-18808-Ljbffr