OFG 1037 – Senior Data QA Engineer

2

About us

Ofgem is Great Britain’s gas and electricity regulator. Our goal is to protect

energy customers’ interests, now and in the future. We make sure competition encourages great customer service and bears down on companies’ prices and profits. We do this by scrutinising costs and checking that companies stick to the

rules we set. If a company falls short, we act. Since 2010, we’ve imposed over £100 million in fines.

We also control the revenue that network companies earn from running gas and

electricity grids. This way we make sure customers get value for money from a £6 billion a year programme to upgrade Britain’s networks.

And we are planning for the future too, with the rollout of smart metering over the next few years, and looking ahead to make sure there’s enough capacity in

the system to meet demand. About a third of our staff work in Ofgem E-Serve, a division that runs

government environmental and social schemes. These include the Energy Companies Obligation, the Domestic and Non-Domestic Renewable Heat

Incentives, the Renewables Obligation, the Feed-In Tariff scheme, and the Climate Change Levy exemption. The total value of these schemes is approximately £6 billion per year; our costs for administering them are just

0.6% of this.

We are a diverse and vibrant organisation, with the majority of our staff working in our London (Canary Wharf) and Glasgow offices. You can find the Ofgem structure chart here.

We aim to attract, support, retain and motivate our staff by making sure that

they have the skills and knowledge to perform their roles effectively and provide them with development opportunities to build a career.

We offer a variety of development including formal training, on-the-job training and coaching. There is also a comprehensive management development

programme and support for further education and professional qualifications. Ofgem is a great place to work, and we hope to be welcoming you soon.

Dermot Nolan

Chief Executive

3

Ofgem Privacy Notice

Registration on Ofgem Recruitment Portal

When you register on the Ofgem Recruitment Portal your personal data and information about any applications you have made, will be collected and stored on the platform in order for you to access your details and simplify the

recruitment application process.

How Ofgem will handle your data and job applications

As part of our external recruitment process, personal information will be

collected and only used to process the application and monitor recruitment

statistics.

Where we want to disclose information to any other third parties (for example

where we want to take up a reference or obtain a ‘disclosure’ from either CIFAS,

the Disclosure and Barring Service or Disclosure Scotland), we will not do so

without informing you beforehand unless the disclosure is required by law.

Personal information about unsuccessful candidates will be held on file for six

months. After the recruitment exercise has been completed, it will then be

destroyed or deleted. We retain personalised statistical information about

applicants to help inform our recruitment activities, but this is only available to

HR.

Full details of the Ofgem Privacy Policy and how your data is used can be found

at https://www.ofgem.gov.uk/privacy-policy

4

Your data when applying for open vacancies

Each time you submit an application the data that we collect will be stored and

utilised in line with the Ofgem Privacy Policy, as outlined above.

Your rights to have your data removed and/or destroyed

All candidates have the right to have their data deleted at any point and this can

be done by emailing recruitment@ofgem.gov.uk

Your Consent and confirmation

By submitting an application, you are consenting to us using, and storing your

data, as outlined in this Privacy Notice, full details can be found in the Ofgem

Privacy policy as outlined above.

If you have any concerns you can view the Civil Service Privacy Statement by

clicking here.

This privacy policy will be kept under regular review.

5

Role Profile

Role details

Role Title Senior Data Quality Assurance Engineer

Band Band D

Division and Directorate Delivery Directorate, Data Services

Location London/ Glasgow

Number of positions and

contract type

Permanent

Purpose

The Data Services Team

Ofgem has formed a ‘Data Services’ team to transform its approach to working with and creating

value from its data.

Our cross functional team is made up of product leads, software engineers, user researchers, QA

testers, business and data analysts and uses an agile approach to delivery. We are user focussed and

foster a friendly, dynamic, collaborative and supportive environment that encourages personal

development.

The team provides Data as a Service (DaaS) through developing and maintaining an enterprise data

platform for use by the energy industry and the entirety of Ofgem.

The platform:

supports staff and external stakeholders by servicing them with data and providing modern

software tools for exchanging information and gaining insight;

provides secure and reliable set of services and tools to meet its corporate responsibilities;

promotes automation and quality assurance;

encourages transparency, accessibility and coordination.

6

The Role

You will be responsible for the testing and validation of data for the business. You will be expert at

working in cloud environments, such as on Hortonworks and Cloudera based Hadoop clusters. You

will be able to draw on your experiences to engage in discussion with the technical design lead and

other technical staff, recommending options for continuous improvements to testing methods, such

as automation. Assurance of data security and privacy in the platform will be your greatest concern.

You will work as part of a multidisciplinary team operating in an agile approach to delivery. You will

encourage colleagues to hold open and inclusive discussions about how to deliver the business’

technical needs. Your will seek to maximise knowledge retention and transfer, seeking off the shelf

solutions to delivery. You will continually be seeking to innovate better practices to delivery ensuring

that the Data Service can continue to deliver value to energy consumers on a sustainable basis and in

a way that meets security and privacy corporate responsibilities.

You will knowledgeable about governance concerns, such as data security and security; offer

mentoring and advice to junior members of the team and; deliver new functionality to the Data

Services platform. You will showcase the team’s tested and validated software to business owners

and subject matter experts to learn whether it meets their business outcomes.

Key responsibilities

Validate software delivered by Data Engineers against user story requirements

Ensure principles of privacy and security are assessed and designed into your data solutions

Write high quality, generalised test datasets and test methods for verifying and validating new functional and non-functional software, enabling automated testing wherever possible

Generate golden datasets to efficiently stress-test software and assure version control

Report QA outcomes to Data Engineers and Business Owners

Work in an agile way and to do so closely with colleagues

7

Develop new and maintain existing functionality to QA testing for the Data Services platform

Provide documentation and training material to support knowledge retention

Maintain a high quality audit trail of your software testing

Key outputs and deliverable

Software has had an auditable trail of quality assurance testing conducted on it in accordance

with Ofgem data governance expectations

Business solutions comply with Privacy and Security requirements

Knowledge sharing and best practice training are provided to colleagues and the wider base

of stakeholders as required

Business Owners are satisfied that code has been robustly demonstrated to meet their needs

Documentation regarding QA testing is maintained to a high standard

Ensure that governance rules, particularly security standards are always upheld

Key stakeholder relationships

Work with the rest of the team and its key stakeholders, such as Business Analysts, Subject Matter Experts and Data Engineers, to review their work and demonstrate the quality of data handling scripts delivered.

Support the wider user base with technical support and training, as appropriate

The Data privacy and information security teams, and assure requirements are identified and addressed. Creating processes, automated tools and technics to value, protect and assure data services

Specific skills and capabilities

Essential:

Willingness to constructively challenge the work of others, to ensure high standards are achieved

o Experience of carrying out acceptance demos with stakeholders

Understanding of Quality Assurance data practices

o Awareness of OWASP top 10 security considerations and associated tools

o Hands-on experience writing unit tests and mocking the data

o Hands on experience to analyse regression reports, automated builds and KPI's

o Experience of Test Driven Development applying pair programming and refactoring principles

o Strong experience of writing BDD acceptance criteria or scenarios

o Participated in full development data lifecycle

Knowledge of software development experiences

8

o 3+ years experience in either JS/Java/C# programming language and must be comfortable to

develop Strong experience in SQL/HQL for complicated queries

o automated test framework development from scratch using Python

o Experience of using big data technologies like PySpark/Yarn and Hive

Hands on experience with Linux and Microsoft Azure cloud environment or equivalent

o Experience in using considerable data size (TB/PB)

o Exposure to continuous build and deployment process, with hands on experience in using

build systems such as Oozie/Jenkins

o Familiar with defect management systems either using JIRA or similar

The desire to continuously innovate and bring about improvements to best practice, whether applied

directly to technology or to best practice with regulatory data

Desirable:

Experience in working Agile methodologies (Scrum/Kanban)

Experience with performance tuning in Hadoop configuration systems to maximise performance

Knowledge of Business intelligence tools (examples might be QlikSense, Tableau, Domo, etc)

Experience in accessibility, stress, performance or load testing.

Working knowledge of HBase, Solr, Kafka and Flume

Civil Service Competencies

Changing and Improving

Collaborating and Partnering

Delivering Value for Money

Managing a Quality Service

Delivering at Pace

Building Capability for All

9

How to apply

To apply for this role, you are required to submit a CV and Suitability Statement.

Completed applications should be submitted by midnight on the closing date.

Applications received after that time will not normally be considered. Your

application should consist of:

CV (saved as forename. surname CV)

Suitability Statement

As part of your application process, we will check your details against CIFAS

fraud prevention databases.

Should our investigations identify fraud or the commission of any other criminal

offences by you when applying for, or during the course of your employment

with us, we will record the details of this on the relevant fraud prevention

databases. This information may be accessed from the UK and other countries

and used by law enforcement agencies and by us and other organisations to

prevent fraud.

Further details of the CIFAS fraud prevention databases, through which we share

information, can be obtained by contacting recruitment@ofgem.gov.uk

Your details will also be checked against the Civil Service Internal Fraud

Database (IFD) of employees dismissed for fraud or dishonesty offences. Where an applicant to Ofgem is successful in interview, we will check your name and date of birth against the IFD. Any applicant who is included in the IFD will be

refused employment.

This information will be retained by the Cabinet Office on the IFD for a period of

five years from the date of dismissal (or the date employment ended). GRS and

other recruitment teams will share with the Cabinet Office the outcome of the

checks for monitoring and analysis purposes. The Cabinet Office will use this

information for research on the prevention and detection of fraud. This will not

be shared outside Cabinet Office, GRS, and other recruitment teams, except on

an anonymised basis.

10

Overview of process

Your completed application will be acknowledged and you will be notified of the

outcome of the application.

Your invitation to interview correspondence will give details of specific date and

time. We will also notify you of any presentations / tests etc (not all roles require

this). The interview will be a combination of competency questions and role

specific questions. Please see the competency framework below relevant to this

role.

Please note, Ofgem do not cover the cost of travel to and from your interview.

Competencies

You will be assessed against a selection of the Band D Competencies at

interview stage. These are specified in the role profile.

The competencies can be found here. Please ensure you download and keep this

document in preparation for your potential interview.

11

Terms and conditions

Role Title Senior Data QA Engineer

Salary Glasgow: £52,600 - £76,800 or London: £56,500 - £83,640

Location Glasgow or London

Hours of work London: 36 hours per week Glasgow: 37 hours per week

Annual leave entitlement

As a new starter you are entitled to 25 days annual leave which will rise to 30 days after 2

years’ service. If you are transferring from another Government Department, and have 2

years continuous service your annual leave will be 30 days on joining.

You will also receive 10.5 days public, Bank and privilege holiday. Please note the dates

vary in England, Wales and Scotland.

If your appointment is for less than a year then you will receive a pro-rata and

public/privilege allowance.

Pension The civil service offers excellent pension

arrangements. For further information, please visit:

http://www.civilservicepensionscheme.org.uk/

Other benefits Cycle to work

Childcare vouchers Season ticket loan Employee wellbeing

Enhanced maternity /paternity pay Subsidised Gym

12

Our values

Making a positive difference for energy consumers

Our priority is to protect and to make a positive difference for all energy consumers. We work to promote value for money, security of supply and sustainability for present and future generations. We do this through the

supervision and development of markets, regulation and the delivery of government schemes.

We work effectively with, but independently of, government, the energy industry

and other stakeholders. We do so within a legal framework determined by the UK government and the European Union.

Our values are:

Dedication

Committed to making a positive difference for all energy consumers today

and for a sustainable future Making time to hear and understand views and be open to new ideas

Being experts in our field - learning from everything we do – developing our skills

Integrity

Acting with independence

Being transparent and straightforward in our dealings with each other and our stakeholders

Basing our decisions on the best available evidence and impartial

assessment

Determination

Finding a way to make things happen To deliver - focusing on the consumer outcome, working collaboratively

with each other, and in partnership with stakeholders Thinking ahead, and leading thinking

13

Equalities and diversity

As an employer we are committed to equality. We want to be an organisation

that people are proud to be part of. We welcome applications from all people across all sections of our diverse communities.

We aim to ensure equality of opportunity in our employment practice and processes. This is irrespective of disability, gender, age, marital status, sexual

orientation, race, colour, nationality, ethnic or national origins, religion or religious affiliation, working pattern and responsibility for dependents.

We believe that every employee should be treated with openness, honesty and respect. We won’t tolerate unfair treatment or discrimination. We aim to have a

workforce that reflects the diverse community that we serve. We value and make the best use of this diversity of talent, so that our people fulfil their

potential and create the conditions for business success.