Data Engineer – SC Cleared – 6 Month Contract
As a Data Engineer, you will design, develop, deploy, and maintain data architecture which employs various methods to transform raw data into processed data. You will own the data operations infrastructure, manage and optimise performance, reliability, and scalability of the system to meeting growing demands on ingestion and processing pipelines.
To succeed in this data engineering position, you should have strong problem-solving skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages.
Your Skills and Knowledge:
• Technical expertise in designing, building, and maintaining data pipelines, data warehouses, and leveraging data services.
• Proficient in DataOps methodologies and tools, including experience with CI/CD pipelines, containerisation, and workflow orchestration.
• Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.)
• Knowledge of programming languages (e.g. Java, Python, SQL)
• Hands-on experience with SQL/NoSQL database design
• Degree in STEM, or similar field; a Master’s is a plus
• Data engineering certification (e.g IBM Certified Data Engineer) is a plus
Your Role:
• Orchestration ingestion and storage of raw data into structured or unstructured solutions.
• Design, Develop, Deploy and Support data infrastructure, pipelines and architecture.
• Implement reliable, scalable, and tested solutions to automate data ingestion.
• Development of systems to manage batch processing and real-time streaming of data.
• Evaluate business needs and objectives.
• Support implementation of data governance requirements.
• Facilitate pipelines, which prepare data for prescriptive and predictive modelling.
• Working with domain teams to scale the processing of data.
• Identify opportunities for data acquisition
• Combine raw information from different sources.
• Manage and maintain automated tools for data quality and reliability.
• Explore ways to enhance data quality and reliability.
• Collaborate with data scientists, IT and architects on several projects