Junior Data Engineer mit DevOps Anteil (Apache Nifi, Pulsar, AirFlow, Grafana, JupyterLab, Python, Linux, SQL, Azure, AWS) (mwd) RemoteFrankfurt

Posted

9/26/2024

Role

Data Engineer

Location

Germany

Remote

Discuss with client

Project Description

Our project IT_1697 is in need of a specialist to assist with the development and testing of an Automatic Damage Detection on Freight Cars project in Frankfurt. The project aims to explore the application of Artificial Intelligence (AI) in analyzing image data of freight cars to detect damages and assess its operational feasibility. Daily, 1400 wagons are captured by each camera bridge, producing around 40 images per wagon. Key tasks include migrating MLOps pipelines to Azure/BuildIT platforms, designing and implementing ETL processes in a Big Data environment, and overseeing troubleshooting and operational testing of AI use cases. Proficiency in Big Data tools such as Apache Nifi, Python, Linux, SQL databases, and basic knowledge of Azure are essential requirements, while familiarity with Apache Hadoop/HDFS, AWS Cloud, agile project environments, and a willingness to learn about rail freight transportation are desirable skills. The successful candidate must demonstrate a track record of relevant experience and expertise to effectively contribute to this innovative project.

Skills

Apache NifiApache PulsarApache AirFlowGrafanaJupyterLabPythonLinuxSQLAzureApache Hadoop/HDFSAWS Cloud
Go to Project Page