Principal Data Engineer
This job does not exist anymore.
Try running a new search or browse our vacancies.
Or fill in the form below to receive job alerts.
| Job Type | Permanent |
| Area | Homebased, United Kingdom |
| Sector | IT |
| Start Date | |
| Job Ref | PH10529 |
- Description
Salary: Negotiable – 15% bonus (depending on company performance), 28 days holiday + bank holidays, pension, Medicare, car insurance deals
Our client is a financial service provider (insurance) and they are seeking a Principal Data Engineer to join a team of 8-10 people reporting to the Head of Data.
The client operates in an Azure cloud environment so experience in this is essential as is the ability to provide support to junior members of the team.
Applicants must be eligible to work in the UK – Our client is unable to provide sponsorship neither now, nor or in the future.
Must Haves
· Requires extensive experience of designing and building end to end data solutions (10 years +)
· Exceptional at building strong / effective relationships with people from different disciplines
· Experience of carrying out data engineering design and build activities using agile working practices (such as Scrum or Kanban)
· Data Factory / Synapse Workspace – for building data pipelines or synapse analytics pipelines
· Data Lake – Delta Lake design pattern implementation experience in Azure Data Lake Gen2 with file hierarchy namespace and low-level permissions management
· Synapse Warehouse / Analytics – Experience in Synapse data mappings / external tables / schema creation from SSMS / knowledge on how Synapse pool works behind the scenes
· Azure Active Directory – for Managed identities creation and usage or for generating service principles for authentication and authorization
· Version Control – Experience in building Data Ops i.e. CICD pipelines in Azure DevOps with managed identity
· Unit Testing – Experience in writing unit tests for data pipelines
· Data Architecture – Knowledge or experience in implementing/ Kimball style Data Warehouse. Experience in building Metadata with Azure Purview or Data Lake Gen2
· Data Quality – Experience in applying Data Quality rules within Azure Data Flow Activities
· Data Transformation – Extensive hands on with Azure Data Flow Activities for Cleansing / transforming / validation and quality checks
· Azure Cloud – Knowledge and confidence in effective communication on Azure Cloud Subscriptions / Resource Groups / Subnet / VNet / Private Endpoints testing; Firewall rues management on Azure data platform components
· Keen and active interest in the use of data in the wider industry; with practical knowledge and networks
Description
The Principal Data Engineer role is a hands-on technical role where you’ll be providing technical data solution leadership on projects and for persistent data products.
You will give technical direction to engineers who will build solutions against the agreed designs. Whilst not having any direct people management responsibilities, the role holder
will need to provide support, coaching and mentoring to the data engineers, ensuring they are able to deliver to the required quality and integrity.
To be successful in this role, you’ll need to:
· Be an expert in Azure data engineering including experience of Delta Lakehouse architecture
· Possess experience in data modelling concepts
Job responsibilities:
· Responsible for creating or guiding the low-level design of data solutions, taking high level solution architecture artefacts and translating them into workable designs and work packages
· Responsible for the quality of the overall data platform(s), ensuring that data pipelines and database solutions are implemented effectively from a re-use and performance optimisation perspective
· Responsible for coding standards, low level design and ingestion patterns for the data platform(s), that all users, including data engineers follow
· Develop high complexity, secure, governed, high quality, efficient data pipelines from a variety of on and off premise, internal and external data sources
· Set the standards and ensure that data is cleansed, mapped, transformed and optimised for storage to meet requirements for business and technical use cases
· The design and build of data observability and data quality by design into all Data pipelines, promoting self-testing pipelines that proactively identify processing issues or discrepancies
· Build solutions that pipe transform data into data lake storage areas, physical database models and reporting structures across data lake, data warehouse, business intelligence systems and analytics applications
· Build physical data models that are appropriately designed to meet business needs and optimise storage requirements, ensuring maximum re-use
· Carry out unit testing of own code, peer testing of others code to ensure appropriate quality, and be responsible for completeness and integrity of solutions delivered on the data platform(s)
· Ensure that effective, and appropriate documentation that brings transparency and understandability are in place for all content on the data platform(s)
· Coach and mentor Senior Data Engineers, Data engineers & Associate Data Engineers
· Create high complexity BI solutions including data mart, semantic layer, and reporting & visualisation solutions in recognised BI tools such as PowerBI
· Any other reasonable duties as required
Job Title: Principal Data Engineer
Location: Remote – Occasional trips to Manchester / South East offices (expenses paid)
Salary: Negotiable – 15% bonus (depending on company performance), 28 days holiday + bank holidays, pension, Medicare, car insurance deals
Hours of Work: Monday to Friday – 9.00am to 5.00pm
Job Type: Permanent
Job Reference: PH10529
Wilde Associates is working in the capacity of an Employment Agency for this role
