Data Platform Engineering Lead

TEKEVER

Porto, Portugal

Mission:

The Data Platform Engineering Lead will be responsible for overseeing the design, implementation and maintenance of our Data & Analytics platform, pipelines and infrastructure. This role requires a combination of strong technical expertise, leadership abilities and a strategic mindset. The ideal candidate will have a proven track record in data architecture, data engineering, cloud and on-premise technologies, with experience in managing a deeply technical team and driving complex projects to completion. This role involves collaborating with cross-functional teams to ensure our data platform is robust, scalable, secure and aligned with the company's strategic objectives.

What will be your responsibilities:

Leadership: Lead and mentor a team of data architects and data engineers, fostering a collaborative and high-performance culture;
Platform Development: Oversee the design, development and maintenance of scalable and robust data architectures and pipelines, ensuring efficient data processing, storage and retrieval, including data ingestion, storage, processing, management and analytics components;
Architecture Design: Develop and maintain the architecture of the data platform, ensuring it meets the needs of various business units and supports future growth;
Technology Strategy: Define the technology stack and architecture standards for the data platform, ensuring alignment with industry best practices and emerging trends;
Data Integration: Develop strategies for integrating diverse data sources, ensuring seamless data flow and high data quality;
Scalability and Performance: Design solutions that can scale with growing data volumes and ensure optimal performance across the data platform;
Performance Optimization: Continuously monitor and optimize the performance of the data platform to handle large-scale data efficiently;
Security and Compliance: Implement robust data security measures and ensure compliance with relevant regulations and industry standards;
Collaboration: Work closely with data scientists, analysts, product managers and other stakeholders to understand data requirements and deliver effective solutions;
Innovation: Stay up-to-date with the latest advancements in data technologies and drive continuous innovation within the data platform;
Documentation: Maintain comprehensive documentation of data pipelines, ETL processes and architectural decisions.

Profile and requirements:

Education: Bachelors or Masters degree in Computer Science, Engineering, Information Systems, or a related field;
Experience: 7+ years of experience in data engineering or a similar role, with at least 3 years in a leadership position;
Technical Expertise:
- Strong experience with Data Platform reference architectures (e.g. Lambda architecture, Data Mesh);
- Deep knowledge of big data technologies (e.g., Hadoop, Spark, Kafka) and data warehousing solutions (e.g., Redshift, Snowflake);
- Extensive experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services, with a focus on Google Cloud. Google Cloud certification is preferred;
- Experience with migration from on-premise to cloud and vice versa;
- Good knowledge of relevant security frameworks & standards;
- Proficiency in programming languages such as Python, Java, or Scala;
- Strong understanding of database management systems (e.g., SQL, NoSQL, NewSQL). Experience with SQL and database management systems (e.g., MySQL, PostgreSQL, SQL Server);
- Knowledge of data integration tools and frameworks (e.g., Apache Nifi, Talend, Informatica);
- Familiarity with data modeling, data warehousing and data governance practices;
- Experience with Iaac (e.g. Ansible, Terraform), data pipeline orchestration (e.g. Airflow), log exploration tools (e.g. Streamlit, Dash), data extraction (e.g. PostGIS, Kafka, Airflow, FastAPI), pandas, scikit-learn, Docker;
- Solid knowledge of DevOps best practices and tools: GIT, CI/CD, telemetry and monitoring, etc.
Analytical Skills: Strong analytical and problem-solving skills with a focus on delivering scalable and efficient data solutions;
Leadership: Proven leadership skills with experience in building and leading high-performing teams;
Communication: Excellent verbal and written communication skills, with the ability to effectively collaborate with technical and non-technical stakeholders;
Project Management: Strong project management skills with the ability to manage multiple projects and priorities simultaneously;
Attention to Detail: High attention to detail and a commitment to ensuring data quality and accuracy;
Adaptability: Ability to work in a fast-paced, dynamic environment and manage multiple priorities simultaneously.