Senior Data Engineering Developer
As part of the Data Engineering team you will develop and support implementation of data pipeline and data aggregation processes. We are looking for an experienced Senior Data Engineering Developer who is enthusiastic about developing code and providing data manipulation solutions by using Java and Python languages as well as Spark framework.
Design, develop, implement and monitor a highly scalable system to ingest and process terabytes of data per day
Work with other developers, engineers, data scientists and business stakeholders to continuously explore new capabilities and technologies to drive innovation
Work with different teams to translate business and analytics requirements into a data strategy for the engagement including ETL, data models, aggregations and reports for analysis
Develop standards for data processes and automate routine tasks
Support data transformation testing and production implementation as required
You’re great at…
Developing, monitoring, maintaining, and supporting data analytics, data management and ETL solutions including availability, scalability, performance, data quality and data aggregation
Implementing new components using various emerging Big Data technologies
Executing ingestion of new data sources into the analytics platform including but not limited to ETL code development, data persistence design and processing logic
Assistance with design, implementation and POC of any components related to the Analytical Environment
Building large-scale real-time and batch data processing and analytics pipelines using the latest technologies
Implementing conceptual, logical and physical data models for the data analytics platform
Suggesting scheduling/timing of automatic jobs to avoid conflict with application processes
Translating business requirements into functional and technical specifications
Developing tools and utilities as needed
Supporting the data operations group for problem analysis and resolution.
Participating in rotating on-call support.
Who are you?
You have a University Degree in Computer Science Engineering or an equivalent combination of education and experience
You have 3-5 years’ experience in a similar role and have hands on development experience with Hadoop environment, Spark framework and Java and Python languages
You have strong knowledge of RDBMS and NoSQL databases
You have solid experience with ETL processes and data integration patterns
You have an experience with CI/CD practices within big data development
You like to develop in one or few of the languages like JAVA, Python and PL/SQL
You have exposure to Big Data technologies including the development, performance tuning, and operational activities involved
You have experience with data warehousing, operational data stores and large-scale implementations as well as a lot of experience with ETL processing
Your knowledge related to DevOps, Cloud engineering, AWS experience, Oracle, Cloudera and Talend is an asset
You know that security is one of the most important and critical parts in any data related projects.
You have the ability to apply technology when solving business problems and are able to communicate with both technical and non-technical audiences
You’re great at coaching other members of the Data Engineering team
You’re a strategic thinker who understands business requirements and ensures that the overall application architecture and solutions meet our business requirements with specific attention to future demands and functional growth