Data Engineer / Python Developer

Для маркетплейса премиальной одежды с центральным офисом в UAE требуется Data Engineer.

Резюме опубликовано не HR агенством, а техническим представителем компании в РФ, поэтому собеседование будет проходить исключительно с техническими специалистами напрямую.

Для работы необходимо уметь говорить на английском на среднем уровне, Intermediate и выше.

Skills:
- AWS: S3 Data Lake, Redshift, Glue, Lambdas, EC2’s
- Proficiency with Python in a data engineering context
- High proficiency with SQL in a number of dialects (PostgreSQL, BigQuery, Redshift, MSSQL)
- Proficiency with workflow orchestration concepts (DAGs) and tools (Airflow, make, etc)
- Proficiency with data modeling tools (dbt, Dataform)
- Workflow performance, scaling, and optimization. Big Data solutions in some cases.
- Adaptable to Windows, Linux, and container-based deployment environments
- Source control using Git

Stack:
- AWS: S3 Data Lake, Redshift, RDMS (PostgreSQL), Glue, Lambdas, EC2
- Workflow orchestration is in internal application (makefile based) >>> moving to Airflow
- complex logic is in Python
- data modeling is in internal framework >>> moving to dbt / Dataform
- reporting engine is self-hosted Apache Superset
- version control is in Bitbucket

Responsibilities:
- build and maintain data pipeline integrations with 3rd parties via REST API’s / FTP servers / Google Sheets / direct database connections (MySQL, PostgreSQL, AWS Redshift, etc.) / or simple file downloads
- develop and maintain internal ELT processes + an entire ELT pipeline for event tracking data from scratch
- build data modelling on product catalogue data / financial data
- build a visualisation platform for data sharing (migrate from Superset)
- do your own devops.

Трудоустройство в аккредитованной компании в РФ.