Your role is to design, build, test and maintain data on the enterprise data platform, making sure available data meet business requirements and user needs.
This role is responsible for developing and helping maintain robust, performant data integration and transformation routines and services to ensure availability of data to support insight needs.
You will be responsible for maximising the automations, scalability, reliability and security of data services, focussing on opportunities for re-use, adaptation and efficient engineering.
Accountabilities:
Design, build and test data pipelines and services, based on feeds from multiple systems using a range of different storage technologies and/or access methods provided by the Enterprise Data Platform, with a focus on creating repeatable and reusable components and products.
Design, write and iterate code from prototype to production ready. Understand security, accessibility and version control. Use a range of coding tools and languages as required.
Work closely with colleagues across the Data & Insight Unit to effectively translate requirements into solutions, and accurately communicate across technical and non-technical stakeholders as well as facilitating discussions within a multidisciplinary team.
Deliver robust, supportable and sustainable data solutions in accordance with agreed organisational standards that ensure services are resilient, scalable and future proof.
Understand the concepts and principles of data modelling and able to produce, maintain and update relevant physical data models for specific business needs, aligning to the enterprise data architecture standards.
Design and implement data solutions for the ingest, storage and use of sensitive data within the organisation, including designing and implementing row and field-level controls as needed to appropriately control, protect and audit such data.
Clearly, accurately and informatively document and annotate code, routines and other components to enable support, maintenance and future development.
Work with QA Engineers to execute testing where required, automating processes as much as possible.
Learn from what has worked as well as what has not, being open to change and improvement and working in ‘smarter’, more effective ways.
Ensure that problems are fixed according to SLA’s or in a timely manner providing proactive communication.
Ensure the use and protection of government and public information is handled securely and with care, following data management principles and policies.
Design and undertake appropriate quality control and assurance for delivery of output.
Promote a strong data culture in line with the organisational data strategy.
Skills and Experience:
Educated to degree level or have equivalent professional experience.
Experience translating business requirements into solution design and implementation.
Extensive experience of data engineering and the development of data ingest and transformation routines and services using modern, cloud-based approaches and technologies.
Ability to apply relevant techniques and approaches to validate, optimise and monitor data transformation to improve the efficiency and robustness of solutions.
Understanding of the principles of data modelling and data flows with ability to apply this to design of data solutions.
Understanding of the fundamentals of solution and data architecture.
Experience of supporting and enabling AI technologies.
Experience following product/solution development lifecycles using frameworks/methodologies such as Agile, SAFe, DevOps and use of associated tooling (e.g. version control, task tracking).
Experience working with Database technologies such as SQL Server, Oracle and Data Warehouse Architecture with knowledge of big data, data lakes and NoSQL.
Experience implementing data flows to connect operational systems, data for analytics and BI systems.
Experience documenting source-to-target mappings.
Demonstrable experience writing ETL scripts and code to make sure the ETL processes perform optimally.
Experience in other programming languages for data manipulation (e.g. Python, Scala).
Experience in assessing and analysing technical issues or problems in order to identify and implement the appropriate solution.
Knowledge and experience of data security and data protection (e.g. GDPR) practices and application of these through technology.
Experience in reconciling conflicting views and articulating coherent rationales for action.
Ability to identify problems and lead the delivery of solutions and preventative measures, escalating where appropriate.