Senior Data Engineer

Ciklum

Remote, Poland

About the role:

As a Senior Data Engineer, become a part of a cross-functional development team engineering experiences of tomorrow. Together, we will work on the project building and scaling a digital platform used globally for learning, communication, and school management. This platform supports over 90,000 students across 24 countries and serves teachers, parents, and students with mission-critical functionality.

Responsibilities:

Responsible for the building, deployment, and maintenance of mission-critical analytics solutions that process terabytes of data quickly at big-data scales;
Contribute design, code, configurations, and manage data ingestion, real-time streaming, batch processing, and ETL across multiple data storages;
Perform development, QA, and DevOps roles as needed to ensure total end-to-end responsibility of solutions;
Responsible for performance tuning of complicated SQL queries and data flows;
Identify gaps and improve the platform’s quality, robustness, maintainability, and speed;
Cross-train other team members on technologies being developed, while also continuously learning new technologies from other team members;
Contribute to the unit’s activities and community building, participate in conferences, and provide excellence in exercises and best practices.

Requirements:

3+ years of experience with SQL, C#, Python and solid CS fundamentals (data structures, algorithms);
2+ years contributing to production deployments of large-scale backend data processing systems;
1+ years of hands-on implementation experience working with a combination of the following technologies: Hadoop, MapReduce, Pig, Hive, Impala, Spark, Kafka, Storm, SQL and NoSQL data warehouses such as HBase and Cassandra;
2+ years of experience with cloud data platforms (AWS/Azure / GCP);
Experience with Databricks and Snowflake;
Strong understanding of professional software engineering practices and full SDLC;
Knowledge of data warehousing, optimization, data quality, automation, and validation;
Experience building BI reports and dashboards (Power BI, Tableau);
Experience working in Agile teams (SCRUM);
Experience designing key components in large distributed systems;
Proven track record of delivering high-quality solutions on complex cross-functional projects;
Experience supporting data scientists and advanced statistical use cases (nice to have).