data migration plan 2.1 – unofficial · data migration plan 2.1 – unofficial 15.4.2020 finnish...
TRANSCRIPT
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 2 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Table of contents
1 Executive Summary .............................................................................................................................. 5
2 Parties and responsibilities .................................................................................................................. 7
2.1 Market parties’ responsibilities for delivery of source data .............................................................. 7
2.1.1 Master data delivery responsibility and rules concerning data selection .................................... 8
2.1.2 Metering data delivery responsibility and rules concerning data selection ............................... 10
2.2 Responsibilities of Fingrid Datahub Oy ......................................................................................... 12
2.3 Responsibilities of the data migration service provider ................................................................. 13
2.4 Responsibilities of the Datahub system supplier ........................................................................... 13
2.5 RACI matrix ................................................................................................................................... 13
3 Data migration project ........................................................................................................................ 15
3.1 General description and schedule of the data migration project ................................................... 15
3.2 Requirements for source data quality ............................................................................................ 16
3.3 Requirements for the duration of migration file production and delivery ....................................... 19
3.4 Targets of the project stages and acceptance criteria for checkpoints ......................................... 20
3.5 Detailed schedule of Stage 3 ......................................................................................................... 22
3.6 Detailed preliminary timetable for stages 4 and 5 ......................................................................... 28
4 Instructions for data migration work ................................................................................................. 32
4.1 Titta service ................................................................................................................................... 32
4.2 Description of data migration process ........................................................................................... 33
4.3 Market party tasks during the data migration process ................................................................... 34
4.3.1 Production of data migration files .............................................................................................. 35
4.3.2 Import and checking of master data .......................................................................................... 35
4.3.2.1 File check ............................................................................................................................... 36
4.3.2.2 Integrity check ........................................................................................................................ 37
4.3.2.3 Source data consistency check ............................................................................................. 37
4.3.3 Processing of observations produced in the Titta service ......................................................... 37
4.3.4 Import and checking of metering data ....................................................................................... 38
4.3.5 The transfer of data to the Datahub system .............................................................................. 38
4.3.6 Data migration check ................................................................................................................. 38
4.4 Data check against external data sources .................................................................................... 39
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 3 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
4.5 The deployment of new IDs ........................................................................................................... 39
5 Data security for the data migration service .................................................................................... 40
5.1 General information about data security for server rooms ............................................................ 40
5.2 Data ownership .............................................................................................................................. 40
5.3 Maintenance of data security ......................................................................................................... 40
6 Data protection for the data migration service ................................................................................ 41
7 Definitions ............................................................................................................................................ 42
8 Appendices .......................................................................................................................................... 43
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 4 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Change history
Date Version Change
27 September 2018
1.0 First official version
17 December 2018
1.1
Added a link to appendix 3
In table 6, the word “preliminary” has been removed and the time limit for the submission of measurement data has been specified as 120 hours.
22 February 2019
1.2 The rules for selecting customer information data are specified in Table 2.
30 April 2019 1.3 Applied validation checks for data quality in every stage added to Table 5
10 May 2019 1.4
Responsibilities for gird operators have been specified in Table 2 and in section 2.1.2.
Market party definitions have been specified in Table 1
11 December 2019
2.0
Stages 2 and 3 of the data migration in Chapter 1 updated
Figure 2 updated to reflect the new data migration schedule
Updated data migration checkpoints T2–T5 in tables 5, 6, and 7
Added chapter “3.5 Detailed schedule of Stage 3”
Added chapter “3.6 Detailed schedule of Stage 4”
Removed obsolete appendices:
Appendix 2: Preparation by the industry.
Appendix 7: Tasks of distribution system operators in the different stages of the data migration project
Appendix 8: Tasks of the electricity supply company in different stages of the data migration project
Appendix 9: Tasks of the service providers in different stages of the data migration project
Appendix 10: Tasks of the TSO and regional grid operators in the different stages of the data migration project
Updated appendix numbers and links.
15 April 2020 2.1 Chapter 3.5: Data migration Stage 3 schedule updated to match Datahub Table of Milestones version 1.1.
Chapter 3.6: Data migration Stage 4 and 5 preliminary schedules
updated to match Datahub Table of Milestones version 1.1.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 5 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
1 Executive Summary
Datahub data migration means quality assurance and initial download of master data and
metering data of electricity retail market business processes from source systems to the
new Datahub system before its commissioning.
The source systems are primarily the business applications of distribution system
operators and electricity suppliers, in which customer, accounting point, contract and
metering data are kept. In addition, data is also required from the systems of transmission
system and regional grid operators.
The data migration work has two main goals:
The harmonisation of data supplied from source systems
Implementing the initial load for the datahub system.
The datahub system operations require consistency of the data used in the system.
Fingrid Datahub Oy offers electricity retail market parties and service providers a data
migration service (Titta service), which checks data consistency and via which the data
will be transferred to the Datahub system. The market parties can implement the Titta
service by signing a service agreement with Fingrid Datahub Oy.
The Datahub data migration project is divided into five stages, and the following
preliminary schedules and objectives have been specified for the first three stages. The
schedules will be updated if necessary once the new Electricity Market Act takes effect:
Stage 1: 01/2019 - 05/2019
Targets:
The market parties can deliver all up-to-date master data in the format
specified in the migration file instructions.
o All master data has been delivered to Titta in the migration files.
Metering data is not master data.
The quality of master data that is critical to Datahub is at a sufficient level for
data integrity checks and checking the accuracy of customer information and
addresses.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 6 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Stage 2: 6/2019 - 12/2019
Targets:
The quality of data critical to market functionality is at a sufficient level for
consistency checks that cover the entire market.
Mass checks of the accuracy of customer information and addresses can be
performed against external registers.
Stage 3: 01/2020 - 12/2020
Targets:
The use of new GS1 and EIC IDs will begin in the data migration.
o The GLN party IDs and GSRN accounting point IDs being deployed
are compliant with the GS1 system
The quality of data transferred in the data migration is sufficient to allow a
transition to preparations for Datahub commissioning.
The data migration performance meets the requirements for Datahub
commissioning.
After stage 3, the data migration work will progress to support preparations for
commissioning and the actual commissioning of the datahub system, and the quality of
data will continue to be maintained and improved.
The data migration plan helps the electricity retail market parties and their system
suppliers to prepare for commissioning of the datahub system, and supports the parties
during possible system changes and improvement of the quality of data being delivered
from the source systems. The plan has been compiled in cooperation with the Data
migration work group, which consists of representatives from companies in the industry.
The Electricity Market Act that became valid in 2019 will oblige distribution system
operators and electricity suppliers to participate in the Datahub data migration project and
implement the migration of their company’s data to Datahub in accordance with this plan.
In addition, regional grid operators and the transmission system operator are obliged to
transfer data related to exchange point metering to Datahub in accordance with this plan.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 7 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The parties participating in the data migration project and their responsibilities are
described in chapter 2. The stages of the Datahub data migration project and the
acceptance criteria associated with them are described in chapter 3. Chapter 4 includes
instructions for the practical data migration work.
2 Parties and responsibilities
2.1 Market parties’ responsibilities for delivery of source data
In conjunction with the Datahub data migration, the term ‘market party’ refers to all
companies that have singed a data migration service agreement with Fingrid Datahub
Oy, including companies that do not have an electricity retail market ID. Signature of the
agreement is a prerequisite for use of the data migration system. A market party can
operate in one or more of the roles described in Table 1.
TABLE 1 MARKET PARTIES
Party Definition
Distribution system operator Distribution system operators are companies that practise electricity grid operations in a 0.4 - 70 kV distribution grid and have an electricity grid permit granted by the Finnish Energy Authority.
Closed distribution system operator
Closed distribution system operators are companies that practise electricity grid operations in a closed distribution system and have an electricity grid permit granted by the Finnish Energy Authority and whose grid has a nominal voltage of less than 110 kV. A closed distribution system operator is subject to the same rules concerning the responsibility to deliver data as a distribution system operator.
Electricity supplier company Electricity supplier companies are companies registered in Finland as electricity suppliers and which have an agreement with a balance responsible party or serve as a balance responsible party.
Transmission system operator
The transmission system operator with system responsibility (Fingrid Oyj)
Regional grid operator
Regional grid operators, also known as high voltage distribution grid operators, are companies that practise electricity grid operations in high voltage distribution grids and have an electricity grid permit granted by the Finnish Energy Authority.
Industrial grid
The operator of an internal electricity system of a property or a similar group of properties which does not act as a balance responsible party. The grid operator has agreed with the imbalance settlement unit eSett and the DSO to whose network the internal network of a property or a group of properties is connected, that the operator’s grid area is named as a separate Metering Grid Area.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 8 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Production grid Over 1 MVA production plants will be handled as production grids
Service provider
Service providers that need to use the data in Datahub within the scope of authorisations can deliver authorisation information en masse in conjunction with data migration.
A market party can also be a service provider that delivers source data to the Titta service on behalf of another party.
2.1.1 Master data delivery responsibility and rules concerning data selection
The market parties deliver to Datahub the data for which they have maintenance
responsibility. Data for which maintenance responsibility lies with another market party
can also be stored in the parties’ information systems. With regard to accounting point
data, for example, each distribution system operator is responsible for maintaining the
accounting point data in its own region, but accounting point data can also be contained
in the electricity suppliers’ systems. In this case, only the distribution system operator
delivers the data to Datahub during the data migration work. The table below shows
which market party delivers the data, and the period of time covered by historical data
that is delivered in data migration.
TABLE 2 DATA DELIVERY RESPONSIBILITY AND RULES CONCERNING DATA SELECTION
Migration file Data owner (delivers data to Datahub)
Rules Data to be delivered
Customer information
Distribution system operator (grid agreement)
Electricity supplier (sales agreement)
If the information that different market parties have about the same customer differ, only the customer information of the supplier with the most recent (valid latest) contract with the customer will be delivered to the Datahub.
Customers with valid agreements. Customers with agreements that have ended within the past six weeks. Customers with future agreements (See Agreement information)
Accounting point data
Distribution system operator
If a DSO’s system contains accounting points outside its metering grid area, the DSO only supplies the accounting points for its own area. Only main metering points are transferred.
Accounting points in use are imported. (Under construction, connected or disconnected)
Accounting points removed from use are imported if an agreement specified for transfer is associated with them (see Agreement information).
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 9 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Migration file Data owner (delivers data to Datahub)
Rules Data to be delivered
Agreement information.
Distribution system operator (grid agreement)
Electricity supplier (sales agreement)
-
Valid agreements and agreements that have ended within the past six weeks and confirmed agreements that will begin within the next 90 days.
Party information
All
Market parties deliver their own information.
Regional, industrial and production grid operators as well as the transmission system operator deliver their own party information, if the metering grid area has at least one exchange point with a DSO or closed DSO where the party is a metering responsible for this exchange point.
Only valid information.
Product information
Distribution system operator (grid product)
Electricity supplier (sales product)
- Valid products and prices connected to transferable agreements.
Authorisation information
Service provider (third party)
Authorisations are based on third-party rights to process data for a specified customer and accounting point.
The service provider specifies the valid authorisation information needed for the migration file.
Metering grid area data
All grid operators
The DSO and the operator of a closed distribution grid deliver the MGA of their respective network
The operator of a regional grid, production grid, industrial grid and transmission system deliver the MGA data of their respective network, if the conditions for delivering the data are met on the basis of exchange points.
Only valid data
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 10 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Migration file Data owner (delivers data to Datahub)
Rules Data to be delivered
Exchange point data
All grid operators
Only the metering responsible party delivers data for an exchange point between two distribution grids.
The operators of a regional grid, industrial grid, production grid and main grid system deliver the data of exchange points that fall under the scope of Datahub and are associated with distribution grids or closed distribution grids, and the grid operator is responsible for metering in such an exchange point.
Only valid data
Production unit data
Distribution system operator
- Only valid data
Additional accounting point addresses
Distribution system operator
Additional addresses can be reported if necessary for those accounting points that are imported in the Accounting point data.
See Accounting point data
Accounting point supplier data
Distribution system operator
The valid supplier data is reported for accounting points that are in Accounting point data
Valid agreements and agreements that have ended in the past six weeks. Confirmed agreements that will begin within the next 90 days.
2.1.2 Metering data delivery responsibility and rules concerning data selection
Delivery responsibility for metering data:
The DSO and the operator of a closed distribution network delivers metering data
for accounting points, production units and exchange points in its own metering
grid areas.
Only the metering responsible party delivers metering data for an exchange point
between two distribution grids.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 11 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
A regional grid operator delivers the metering data of an exchange point that lies
between a regional grid and a distribution network, when the operator is the party
responsible for metering the exchange point.
An industrial grid operator delivers the metering data of an exchange point that
lies between an industrial grid and a distribution network, when the operator is the
party responsible for metering the exchange point.
A production grid operator delivers the metering data of an exchange point that
lies between a production grid and a distribution network, when the operator is the
party responsible for metering the exchange point.
The transmission system operator delivers the metering data of an exchange point
that lies between the transmission grid and a distribution grid, when the operator
is the party responsible for metering the exchange point.
The figure 1 below presents the rules concerning the transfer of metering data history.
The red vertical line describes the time prior to which data is transferred to Datahub.
During the data migration project, data is uploaded several times, so a red vertical line
may also depict the moment when data is extracted from the system for data checking.
FIGURE 1. RULES RELATED TO THE TRANSFER OF METERING DATA HISTORY DATA
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 12 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The items presented in figure 1 refer to the following cases:
1. The accounting point grid agreement has been valid for a shorter period than six
years, so metering data is imported from the whole agreement period.
2. The accounting point grid agreement has been valid throughout the six-year period.
Metering data is transferred for the entire six-year period. Metering data for a longer
period is not imported to Datahub.
3. If an accounting point grid agreement has expired no more than six weeks before the
data was extracted from the source system, the metering data from these accounting
points is also taken. Metering data is collected for the validity period of the agreement
after it ends, but only for a maximum of six years from the time of extraction.
4. Only metering data within the scope of hourly time series metering is taken into
consideration in metering data. If the metering method has changed during the
agreement validity period, only the hourly time series metered data is included in the
metering data.
5. If the consumption of the accounting point is measured by some other means than an
hourly time series, the use of these accounting points must be reconciled by means of
an equalising calculation by Datahub Go-Live or sooner, and the practice of using an
hourly time series or hourly time series modelling must be adopted for these
accounting points.
Metering data for production units and exchange points is imported for a six-year period
or for the operating period of the metering point if the metering point has been in use for
less than six years.
The data migration process for metering data will be clarified as planning of
commissioning progresses.
2.2 Responsibilities of Fingrid Datahub Oy
Fingrid Datahub Oy’s responsibilities and obligations for the entire Datahub project and
system are specified in the Electricity Market Act and decrees.
Fingrid Datahub Oy is responsible for the progress and schedule of the data migration
project. Monitoring of progress will be carried out by monitoring the amounts of data
delivered to the Titta system by market parties and the quality of source material and
comparing these observations to the data migration stages presented in chapter 3. As
part of monitoring, surveys can be sent to the market parties, which can be used to
assess the market party situation in more detail.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 13 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Fingrid Datahub Oy is responsible for coordinating, scheduling and communicating about
the simultaneous deliveries of source data for the entire industry.
Fingrid Datahub Oy is responsible for the quality, development and maintenance of the
Titta service.
Fingrid Datahub Oy is responsible for communication and instructions related to the data
migration project. The instructions are centralised in the Titta service and the Ediel.fi
portal. Communication will be carried out as needed using different channels.
Fingrid Datahub Oy is actively monitoring the industry’s preparations for changes brought
by Datahub as part of the data migration project. Monitoring is carried out using a range
of surveys targeting the industry. Monitoring the preparation of the industry is vitally
important in order to carry out the planning and commissioning of the whole Datahub
project in an efficient way.
2.3 Responsibilities of the data migration service provider
The data migration service provider (Solteq Oyj) is responsible for the technical
development and maintenance of the Titta service. The service provider is also
responsible for technical user support provided to the market parties.
2.4 Responsibilities of the Datahub system supplier
The Datahub system supplier (CGI) is responsible for compliance of the Datahub system
with the Titta service and for uploading data to the Datahub system and producing the
data migration reports.
2.5 RACI matrix
The tasks and responsibilities of data migration work are described in the accompanying
RACI matrix (Responsibility assignment matrix). The roles are divided as follows in the
matrix:
R = responsible
A = accountable
C = consulted
I = informed
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 14 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
TABLE 3 DATA MIGRATION TASKS AND RESPONSIBILITIES
Task Market party
Data migration service supplier
(Solteq Oyj)
The Datahub system supplier
(CGI)
Fingrid
Datahub Oy
Communications
Communications and contact directed at data migration project market parties
C/I C C/I A/R
Communications directed at data migration project system suppliers
C/I C A/R
Communications and reporting directed at the data migration project pilot group
C/I R C/I A/R
Communications for other external stakeholders in the data migration project
C C A/R
Training and technical support
Instructions for using the Titta service
C/I R A
Training for market parties and system suppliers
C/I C/I A/R
Data migration project technical support
I R A
Titta service
The commissioning of source data checking rules based on the data standard
C/I R
A
Maintenance and management of the Titta service
R A
Establishment of master data in the Titta service
R A
Maintenance of user information in the Titta service
A/R C I
Source data checking and correction
Compiling migration file examples C I A/R
Extraction tools for production of migration files
A/R C
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 15 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Task Market party
Data migration service supplier
(Solteq Oyj)
The Datahub system supplier
(CGI)
Fingrid
Datahub Oy
The production of migration files based on specifications
A/R
Uploading of migration files to the Titta service
A/R
The checking of uploaded data R A
Production of a company-specific check and correction report
I R A
Overall data migration project reporting
R I A
Data correction and enrichment in the market parties’ own systems
A/R C C
Necessary changes to market parties’ own data systems
A/R
Datahub system upload tool A/R
Data migration from the Titta system to Datahub
R A/R
3 Data migration project
The Datahub data migration will be carried out as a sub-project of the Datahub system
project. The aim of the data migration project is to harmonise the data supplied from
source systems, ensure data quality, and initial load of the Datahub system. The Datahub
data standard, in other words the specified data model, and external data sources will be
utilised when harmonising data and ensuring quality.
The functionality of the data migration process and performance of the Titta service have
been verified in a separate data migration pilot project. The data migration processes
were tested in the pilot project using proper source data from pilot companies and the
instructions were improved on the basis of the experiences gained. The pilot project was
completed in March 2018.
3.1 General description and schedule of the data migration project
In the data migration project, the market parties extract the data to be transferred to
Datahub from their own systems and deliver it to Titta in a format that complies with the
migration file specification. The data quality must meet the quality objectives specified in
the check point for each data migration stage. The criteria for data quality and delivery
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 16 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
are tightened from one stage to another, in order to reach the final target. The quality
requirements are described in chapter 3.2 and the delivery requirements in 3.3.
In order to facilitate achievement of the targets in each stage, Fingrid Datahub Oy has
compiled role-specific task lists that are presented in appendices 7–10. Actions
concerning preparation and actual commissioning are specified in the commissioning
plan.
The Datahub data migration plan is divided into five stages, and the preliminary
schedules and targets specified for the first three stages are presented in figure 2. If
desired, a market party can also progress faster than the schedule presented in figure 2,
taking into account the fact that certain actions that apply to the entire industry must be
performed simultaneously in a centralised manner throughout the industry. These actions
include consistency checks, mass data checks against external registers, and
deployment of GS1 IDs. The schedules can be updated if necessary when the law takes
effect.
FIGURE 2 DATA MIGRATION PROJECT PHASING AND SCHEDULE
3.2 Requirements for source data quality
The data supplied by different parties is checked in the Titta service against rules
specified in advance. Quality targets for the source data have been separately set for
each project checkpoint.
In the data migration project, the parties must ensure that data is corrected in the source
system and that it acceptably passes the check.
The data in the source data is divided into four quality requirement levels according to
table 4.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 17 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
TABLE 4 SOURCE DATA QUALITY REQUIREMENT LEVELS
Level Description Explanation
1 Essential for Datahub
Data necessary in terms of the data standard’s data model is given in a format that complies with the data standard.
Incomplete data causes rejection of the entire record.
2 Essential for electricity market operations
Data necessary in terms of the Datahub business rules, such as conditionally mandatory data that is not mandatory from the data model perspective.
Data in the wrong format is rejected.
3 Useful data, but not essential for market and Datahub operations
Useful data from the perspective of electricity market functionality.
Data in the wrong format is rejected.
4 Possible error Data deviates from the information in the external data source but may not necessarily be wrong.
The quality criteria for the checkpoint in each data migration stage are presented in table
5. The checkpoints are consistent with the data migration stages, and progress to the
next stage only takes place when all market parties associated with Datahub have
achieved the quality criteria.
The quality criteria presented in table 5 apply to large changing materials, which are:
Customer information
Agreement information.
Product information
Accounting point data
Accounting point supplier information
Metering data (starting from checkpoint 2)
Additional accounting point addresses
Authorisation information
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 18 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The quality of the following migration files must be 100% at all levels starting from
checkpoint T1:
Party information
Metering grid area data
Exchange point data
Production units, depending on the number of units
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 19 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
TABLE 5 PARTY-SPECIFIC QUALITY CRITERIA FOR SOURCE DATA BY CHECKPOINT
Level Description T1
5/2019 T2
12/2019 T3
12/2020 T4
5/2021 T5
2/2022
Party, metering grid area and exchange point as well as production units, depending on the number of units
100% 100% 100% 100% 100%
1 Essential for Datahub
95.00% 98.00% 99.50% 99.90% 99.98%
2 Essential for electricity market operations
80.00% 95.00% 98.00% 99.00% 99.50%
3
Useful data, but not essential for market and Datahub operations
No requirement
No requirement
No requirement
No requirement
No requirement
4 Possible error No requirement
No requirement
No requirement
No requirement
No requirement
Applied validation checks for data quality
File Check Integrity Check
Consistency Check
Consistency Check
Consistency Check
3.3 Requirements for the duration of migration file production and delivery
Migration file production means all the necessary actions related to extracting the source
data from the source system and writing in a format that complies with the migration file
instructions.
In terms of master data, migration file delivery means that all the migration files have
been transferred to Titta and the user has pressed the “Lataa kaikki” button. In terms of
metering data, migration file delivery means that all the migration files have been
transferred to Titta’s SFTP server.
The duration of source data production is measured from the beginning, but the related
requirements are only added to the acceptance criteria as the project progresses. The
requirements are market party specific.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 20 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
TABLE 6 REQUIREMENTS FOR PRODUCTION OF SOURCE DATA AND DURATION OF
DELIVERY
Material T1
5/2019
T2
12/2019
T3
12/2020
T4
5/2021
T5
2/2022
Master data migration file production and delivery to Titta.
Measured 4 hrs 4 hrs 4 hrs 4 hrs
Metering data migration file production and delivery to Titta.
No requirement
Measured 120 hrs 120 hrs
120 hrs
3.4 Targets of the project stages and acceptance criteria for checkpoints
Table 7 lists the key targets for each data migration stage and the acceptance criteria for
the related checkpoints.
TABLE 7 TARGETS AND ACCEPTANCE CRITERIA FOR STAGES 1–3 OF THE DATA
MIGRATION PROJECT
Targets Acceptance criteria for checkpoints
Checkpoint T0
The Titta service is ready to start the data migration project
Instructions concerning data migration are sufficient and up-to-date
Data security and data protection for the Titta service has been verified
The legislation concerning Datahub has taken effect
Stage 1, 01/2019 - 05/2019
All market parties have signed the data migration service agreement.
The market parties have implemented the master data extraction tools as a system function.
The market parties deliver up-to-date master data in its entirety from their source systems.
The market parties have routines for maintaining data quality.
The market parties have a plan for collecting missing personal identification numbers.
Checkpoint T1
All the fields of the master data files uploaded by the market parties to the Titta service are in a format that complies with the data standard and the migration file specification.
The scope and quality of the source data meets the criteria set for the stage in chapters 3.2 and 3.3.
Each market party has measured the duration of all master data extraction.
All distribution system operators have provided Fingrid with a complete list of exchange points and adjacent grids.
Stage 2 6/2019 - 12/2019 Checkpoint T2
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 21 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Targets Acceptance criteria for checkpoints
All distribution system operators have implemented the metering data extraction tools.
All market parties have delivered data for the coordinated mass check of customer information and addresses.
The mass check of customer information and addresses has been performed and the continuous checking service is in use.
The market parties have performed the party-specific integrity checks on all the source data.
o The parties perform a sufficient number of integrity checks until the data quality is at an acceptable level in accordance with the description in chapter 3.2.
The market parties have ordered new party IDs (GLN), metering grid area IDs (EIC) and metering point IDs (GSRN).
o Only distribution system operators order EIC and GSRN IDs
The data has been checked against external data sources maintained by third parties.
All the parties have performed integrity checks.
The scope and quality of the source data meets the criteria set for the stage in chapters 3.2 and 3.3.
Master data extraction performance meets the criteria set for the stage.
Stage 3 01/2020 - 12/2020
The market parties have deployed the new IDs in migration files.
The GSRN ID service has been taken into use. The service maintains the GSRN IDs until commissioning of Datahub.
A consistency check covering the entire industry has been performed.
o A sufficient number of consistency checks are performed until data quality and performance are at an acceptable level in accordance with the description in chapters 3.2 and 3.3.
Migration source data that covers the entire industry has been uploaded to the Datahub system and the data migration process has been verified based on reports produced by Datahub.
The data migration process is ready to move to the Datahub commissioning preparation stage.
Checkpoint T3
Metering point IDs have been changed to GSRN IDs.
Metering grid area IDs have been changed to EIC IDs.
The parties have begun using GLN party IDs.
Consistency checks covering the entire industry have been performed.
The quality of the source data meets the criteria set for the stage.
The master data and metering data extraction performance meets the criteria set for the stage in chapters 3.2 and 3.3.
Performance of Datahub uploading and data migration verification meets the requirements for Datahub commissioning.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 22 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The preliminary targets and acceptance criteria for Datahub commissioning preparations
and the actual commissioning are described in table 8. The targets and acceptance
criteria will be clarified as the commissioning plan progresses.
TABLE 8 TARGETS AND ACCEPTANCE CRITERIA FOR COMMISSIONING PREPARATIONS AND
FOR COMMISSIONING WITH REGARD TO DATA MIGRATION
Targets Acceptance criteria for checkpoints
Stage 4 01/2021 - 05/2021
Preparations for commissioning
The quality of data and data migration process performance is at a sufficient level for commissioning of Datahub in accordance with the description in chapters 3.2 and 3.3.
The commissioning exercises have been completed, and possible observations made during the exercises have been corrected and the corrections verified.
The data migration process is ready to move to the Datahub commissioning stage.
Checkpoint T4
The source data imported via data migration has been verified in the Datahub business processes.
The source data meets the criteria set for the stage in chapters 3.2 and 3.3.
Other criteria are specified in a separate commissioning plan.
Stage 5 06/2021 – 02/2022
Commissioning
The quality of data is at a sufficient level for commissioning of Datahub in accordance with the description in chapters 3.2 and 3.3.
Checkpoint T5
The source data meets the criteria set for the stage in chapters 3.2 and 3.3.
Other criteria are specified in a separate commissioning plan.
3.5 Detailed schedule of Stage 3
In Stage 3 of data migration, the quality check of the source data will be based on
consistency checks. The consistency checks compare the data submitted by the parties
to Titta. Due to the nature of the consistency check, low quality in the data submitted by
one party produces errors for other market parties. Therefore, each market party must
submit their entire source data (basic and metering data) for consistency checks in a
comprehensive and timely manner. In addition, the market parties will be required to meet
specific quality requirements in an integrity check performed for each party before the
consistency check. The requirements and milestones for the quality of the integrity check
are set out in Table 9. Based on the results of integrity checks, it is easier to demonstrate
likely sources of error. The market parties shall ensure in advance that the quality of their
integrity check that is a part of the consistency check is sufficient. This shall take place on
the market parties’ own schedule by running integrity checks to the source data and
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 23 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
amending the data as appropriate between the consistency checks. Figure 3 presents a
timeline of the milestones of stage 3 of data migration.
FIGURE 3. SCHEDULE OF STAGE 3 OF DATA MIGRATION
Table 9 describes the schedule of data migration stage 3 in more detail, including
milestones and their requirements, and whether failures to meet the acceptance criteria
are reported to the supervising authority. Stage 3 of data migration will concentrate on the
deployment of new IDs, the quality of data and the simultaneous submission of data
throughout the entire industry. The emphasis is on the new IDs and data quality. The
rationale of the consistency checks to be carried out in stage 3 is that all market
participants extract basic data from the same market situation. The time window for
extracting data is stated separately for each milestone to enable reliable comparison of
data between the parties. The market parties shall agree with each other on procedures
for correcting errors noticed in the consistency check.
The schedule of stage 3 is designed to have a delay of one week between the extraction
window and the consistency checks. The scheduled delay is intended as a buffer against
any challenges the market participants might meet in the supplying of information to Titta.
A primary requirement for consistency checks is that all parties are able to extract the
data from the same market situation and deliver the source data to Titta before the
consistency check begins. The performance requirement (basic data 4 hrs / metering
data 120 hrs) is a secondary quality requirement. However, in the later stages,
shortcomings in performance are also reported to the supervising authority, but any
challenges met in performance do not constitute a reason for not submitting the source
data, as long as the source data can be delivered before the beginning of the consistency
check.
In order to verify the performance of the metering data extraction, the extraction of
metering data shall be started within the same extraction window as for the basic data
(Titta enables the comparison of the closing time of the extraction window to the
submission time of the metering data).
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 24 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
When the integrity check has been approved, a market party shall not deliver any more
data to Titta, nor run a new integrity check before the consistency check. This rule is in
place to ensure that the consistency check is performed on the correct and desired data.
TABLE 9 MILESTONES AND REQUIREMENTS FOR DATA MIGRATION STAGE 3
Check or event Acceptance criterion
1. Consistency check 21 January 2020 at 10:00 am The market parties submit all master data to Titta and will perform an integrity check for the master data by themselves before the consistency check begins. Old IDs shall be used in the checks.
The master data must correspond to the market situation on 14 January 2020 from midnight to noon (Extraction window 12 hrs, extraction time 4 hrs).
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 98.00% Quality Category 2: ≥ 95.00%
If necessary, the requirements of future consistency checks might be specified further or changed based on the results of this consistency check. The results are not reported to the authority
Submission of GLN IDs to Titta 31 January 2020 All market parties must submit their GLN ID to Titta by the end of January 2020. The submission is made manually from the user interface. The required functionality will be implemented in Titta by the end of 2019. Titta will also contain a view for converting new and old party IDs.
The transmission of GLN IDs to Titta will not impose any requirements on the market party’s own systems. It is sufficient that the ID exists and has been delivered to Titta, even if it cannot be stored in the market party’s own systems at the moment.
The result is not reported to the authority
Check against external registers 4 February 2020 at noon The data to be included in the check must be submitted to Titta by noon on 4 February 2020. The
The result is not reported to the authority
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 25 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
check will inspect the master data in Titta that have passed the file check without category 1 errors.
The results of the inspection check shall be reported separately for each party after the check is complete. The completion depends on the controller of the external register. After the check, the basic data are checked in real-time (on-line).
2. Consistency check 17 March 2020 at 10:00 am The market parties submit all master data to Titta and will perform an integrity check for the master data by themselves before the consistency check begins. Old IDs shall be used in the checks.
The master data must correspond to the market situation on 10 March 2020 from midnight to noon (Extraction window 12 hrs, extraction time 4 hrs)
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 98.50% Quality Category 2: ≥ 96.00%
Acceptance criteria for the party-specific consistency check:
Quality Category 1: ≥ 95.00% Quality Category 2: ≥ 90.00%
The results are not reported to the authority
The deployment of new IDs 20 April 2020 – 30 April 2020
Starting from noon on 20 April 2020, Titta will only accept transfer files with new IDs. The new GLN IDs must also be used in transfer file names.
By noon on 23 April 2020, the distribution system operators shall deliver the transfer files of supplier data for market parties, accounting points and exchange points to Titta with the new IDs (own GLN, supplier GLN and GSRN IDs). The accounting point data must indicate both the new and old identifiers. Titta will report to the suppliers the GSRN IDs of the accounting points based on the supplier data of the accounting points reported by the grid. Each party can download their report from Titta in an xlsx format. All parties can download from Titta a conversion table for converting between new metering area IDs (EIC IDs) to the old metering area IDs.
By noon on 30 April 2020, all market parties must submit to Titta all master data with new IDs for a
The results are not reported to the authority
With the deployment of the new IDs, consistency checks are not run because the data collection is not coordinated.
The grids ensure individually that the metering data (SAF files) also contain the GSRN IDs.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 26 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
consistency check and run an integrity check on the master data. The data extraction window is not set at this point. The extraction can be started as soon as the new IDs are available in the transfer files.
On 30 April 2020 at noon, a consistency check shall start to verify that all market parties have deployed the IDs successfully for basic data.
3. Consistency check 29 May 2020 at 10:00 am The market parties submit all master data to Titta and will perform an integrity check for the master data by themselves before the consistency check begins.
The master data must correspond to the market situation on 22 May 2020 from 6:00 am to noon (Extraction window 6 hrs, extraction time 4 hrs)
Preliminary quality requirement: New IDs (GS1, GLN, GSRN and EIC) have been deployed in the data migration and the data submitted to Titta does not contain any old IDs The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.00% Quality Category 2: ≥ 97.00%
Acceptance criteria for the party-specific consistency check:
Quality Category 1: ≥ 98.00% Quality Category 2: ≥ 95.00%
Deviations and deficiencies in pre-qualification requirements or approval criteria shall be reported to the authority separately for each party
Submission and inspection of metering data and master data 11 June 2020 – 16 June 2020
DSOs submit all master data to Titta and perform an integrity check on the data by themselves.
The master data must correspond to the market situation on 11 June 2020 from 6:00 am to noon (Extraction window 6 hrs, extraction time 4 hrs)
The grid operator shall start the extraction of metering data at the same time as the extraction of master data and submit the metering data to Titta within 120 hours. The grid operator performs a check for the metering data in Titta.
The metering data and master data must be submitted to Titta by 16 June 2020.
The extraction and delivery of metering data is subject to a performance requirement of 120 h
Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 98.00%
The grid operator shall correct any errors and deviations in the metering data in relation to
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 27 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
The event does not apply to electricity suppliers
the submitted accounting point data.
The results are not reported to the authority
4. Consistency check 22 September 2020 at 10:00 am The market parties submit all master data to Titta and will perform an integrity check for the master data by themselves before the consistency check begins
The master data must correspond to the market situation on 15 September 2020 from 8:00 am to noon (Extraction window 4 hrs, extraction time 4 hrs)
In addition, the grid operator shall start the extraction of metering data at the same time as the extraction of basic data and submit the metering data to Titta within 120 hours. The grid operator performs a check for the metering data in Titta.
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.50% Quality Category 2: ≥ 98.00%
Acceptance criteria for the party-specific consistency check:
Quality Category 1: ≥ 99.00% Quality Category 2: ≥ 97.00% Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 99.00%
The results are not reported to the authority
The grid operator should also correct any errors found in the check of metering data. These errors are not reported to the authority.
Performance requirements: Extraction of basic data ≤ 4 hrs
Extraction of metering data ≤ 120 hrs Performance requirements will not be reported to the authority at this stage
5. Consistency check 25 November 2020 at 10:00 am Checkpoint T3 The market parties submit all master data to Titta and will perform an integrity check for the master data by themselves before the consistency check begins
The master data must correspond to the market situation on 20 October 2020 from 8:00 am to noon (Extraction window 4 hrs, extraction time 4 hrs)
In addition, the grid operator shall start the extraction of metering data at the same time as the extraction of basic data and submit the metering data to Titta within
Acceptance criteria for the party-specific consistency check: Quality Category 1: ≥ 99.50% Quality Category 2: ≥ 98.00% Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 99.50%
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 28 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
120 hours. The grid operator performs a check for the metering data in Titta.
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.50% Quality Category 2: ≥ 98.00%
Deviations and deficiencies in pre-qualification requirements or approval criteria shall be reported to the authority separately for each party
The grid operator should also correct any errors found in the check of metering data. These errors are not reported to the authority.
Performance requirements: Extraction of basic data ≤ 4 hrs
Extraction of metering data ≤ 120 hrs Performance requirements will not be reported to the authority at this stage
3.6 Detailed preliminary timetable for stages 4 and 5
The schedule for data migration stages 4 and 5 follows the testing and commissioning
schedule of datahub. According to the testing and go-live plans, several datahub
migration verifications and trial production runs of datahub will be performed before
Datahub Go-Live. For these purposes, the industry is required to submit its own source
data to Titta for data migration in order to ensure that datahub is always loaded with new
source data. Table 10 shows a preliminary schedule and requirements for data migration
stages 4 and 5. The market parties must provide all master data to Titta for all checks in a
comprehensive and up-to-date format. In addition, DSOs shall provide complete metering
data for all checks. The requirements concerning the submission of master data and
metering data for data migration stages 4 and 5 also apply to all other grid operators that
have exchange points to distribution networks and are themselves responsible for
metering at these exchange points.
Please note! In stages 4 and 5, it is likely that the metering data will be submitted
phased, with the majority of the metering data being submitted in advance and only a
small part of the metering data on the due date. This approach will be decided by June
2020 at the latest.
The schedule, requirements and objectives for stages 4 and 5 will be specified further in
2020.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 29 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
TABLE 10 MILESTONES AND REQUIREMENTS FOR DATA MIGRATION STAGES 4–5
Check or event Acceptance criterion
Stage 4 1. Verification of the migration Consistency check 26 January 2021 The market parties submit all master data to Titta and will perform an integrity check for the data by themselves before the consistency check begins
The market situation from which the data shall be extracted shall be determined later (Extraction window 4 hrs, extraction time 4 hrs) The grid operator shall schedule the extraction of metering data so that it completes at the same time the extraction of master data completes, or earlier. The grid operator must submit the metering data to Titta within 120 hours from the start of the extraction. The grid operator performs a check for the metering data in Titta.
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.90% Quality Category 2: ≥ 99.00%
Acceptance criteria for the party-specific consistency check: Quality Category 1: ≥ 99.50% Quality Category 2: ≥ 98.50% Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 99.90% Deviations and deficiencies in pre-qualification requirements or approval criteria shall be reported to the authority separately for each party
The grid operator should also correct any errors found in the check of metering data. These errors are not reported to the authority.
Performance requirements: Extraction of basic data ≤ 4 hrs
Extraction of metering data ≤ 120 hrs Performance requirements will not be reported to the authority at this stage
Stage 4 2. Verification of the migration (Checkpoint T4) Consistency check 30 March 2021 The market parties submit all master data to Titta and will perform a data integrity check by themselves before the consistency check begins
Acceptance criteria for the party-specific consistency check: Quality Category 1: ≥ 99.90% Quality Category 2: ≥ 99.00% Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 99.90%
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 30 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The market situation from which the data shall be extracted shall be determined later (Extraction window 4 hrs, extraction time 4 hrs)
The grid operator shall schedule the extraction of metering data so that it completes at the same time the extraction of master data completes, or earlier. The grid operator must submit the metering data to Titta within 120 hours of the start of the extraction. The grid operator performs a check for the metering data in Titta.
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.90% Quality Category 2: ≥ 99.00%
Deviations and deficiencies in pre-qualification requirements or approval criteria shall be reported to the authority separately for each party
The grid operator should also correct any errors found in the check of metering data. These errors are not reported to the authority.
Performance requirements: Extraction of basic data ≤ 4 hrs
Extraction of metering data ≤ 120 hrs Performance requirements will not be reported to the authority at this stage
Stage 5 1. Trial production run Consistency check 7 June 2021 The market parties submit all master data to Titta and will perform an integrity check for the data by themselves before the consistency check begins
The market situation from which the data shall be extracted shall be determined later (Extraction window 4 hrs, extraction time 4 hrs) The grid operator shall schedule the extraction of metering data so that it completes at the same time the extraction of master data completes, or earlier. The grid operator must submit the metering data to Titta within 120 hours of the start of the extraction. The grid operator performs a check for the metering data in Titta. This round does not provide all historical metering data to the datahub unless there is a special need for it. The time interval to be provided will be specified as part of the planning of the production trial run.
Acceptance criteria for the party-specific consistency check: Quality Category 1: ≥ 99.95% Quality Category 2: ≥ 99.25% Extraction of basic data ≤ 4 hrs Extraction of metering data ≤ 120 hrs Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 99.95% Deviations and deficiencies in pre-qualification requirements or approval criteria shall be reported to the authority separately for each party
The grid operator should also correct any errors found in the check of metering data. These
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 31 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.95% Quality Category 2: ≥ 99.50%
errors are not reported to the authority.
Stage 5 2. Trial production run (Inspection point T5) Consistency check 7 September 2021 The market parties submit all master data to Titta and will perform a data integrity check by themselves before the consistency check begins
The market situation from which the data shall be extracted shall be determined later (Extraction window 4 hrs, extraction time 4 hrs)
The grid operator shall schedule the extraction of metering data so that it completes at the same time the extraction of basic data completes, or earlier. The grid operator must submit the metering data to Titta within 120 hours of the start of the extraction. The grid owner performs a check for the metering data in Titta. This round does not provide all historical metering data to the datahub unless there is a special need for it. The time interval to be provided will be specified as part of the planning of the production trial run.
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.98% Quality Category 2: ≥ 99.50%
Acceptance criteria for the party-specific consistency check: Quality Category 1: ≥ 99.98% Quality Category 2: ≥ 99.50% Extraction of basic data ≤ 4 hrs Extraction of metering data ≤ 120 hrs Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 99.98% Deviations and deficiencies in pre-qualification requirements or approval criteria shall be reported to the authority separately for each party
The grid operator should also correct any errors found in the check of metering data. These errors are not reported to the authority.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 32 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Final rehearsal of Datahub Go-Live Consistency check 30 November 2021 The market parties submit all master data to Titta and will perform a data integrity check by themselves before the consistency check begins
The market situation from which the data shall be extracted shall be determined later (Extraction window 4 hrs, extraction time 4 hrs)
The grid operator shall schedule the extraction of metering data so that it completes at the same time the extraction of master data completes, or earlier. The grid operator must submit the metering data to Titta within 120 hours of the start of the extraction. The grid operator performs a check for the metering data in Titta.
The requirements also apply to grid owners that have exchange points to distribution grids and are parties responsible for metering at the exchange points
Preliminary quality requirement: The results of the integrity check must meet the following quality requirements: Quality Category 1: ≥ 99.98% Quality Category 2: ≥ 99.50%
Acceptance criteria for the party-specific consistency check: Quality Category 1: ≥ 99.98% Quality Category 2: ≥ 99.50% Extraction of basic data ≤ 4 hrs Extraction of metering data ≤ 120 hrs Acceptance criteria for the party-specific metering data:
Quality Category 1: ≥ 100.00% Quality Category 2: ≥ 99.98% Deviations and deficiencies in pre-qualification requirements or approval criteria shall be reported to the authority separately for each party
The grid operator should also correct any errors found in the check of metering data. These errors are not reported to the authority.
4 Instructions for data migration work
This chapter presents an upper-level description of the tasks to be performed in data
migration. The chapter presents the principles according to which the market parties
deliver data to the Titta service and describes how the service processes the data. More
detailed descriptions of the issues in the chapter are available in the appendices of this
document.
4.1 Titta service
The data migration service – “Titta” – is the most important tool in data migration work.
The Titta service is separate from Datahub, so that the market parties can independently
check the data that they have transferred to datahub during the data migration project.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 33 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The parties can repeat the upload-check-upload process until the quality of the data is at
the required level.
Among others, Titta offers the following functions for market parties:
Delivery of master data via an interface
Delivery of metering data via an SFTP connection
Checking of master data against the data standard
Integrity check of metering data structural information
Checking of customer information and addresses against external data sources
Reports about the quality of the delivered source data
Observation reports for the checks
Market party users will be provided with user support directly from the Titta service. More
detailed instructions regarding use of the Titta service and the technical description are
available in the User instructions for Titta and the Data migration service description
appendix to the service agreement.
4.2 Description of data migration process
Figure 4 presents a description of the data migration process. Data migration work is by
nature a repetitive process, and the number of repetitions depends on matters such as
source data quality, source system compliance with the data standard, the quality of
source data extraction tools, and the market party’s ability to maintain quality during the
data migration.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 34 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
FIGURE 4 DATA MIGRATION PROCESS
4.3 Market party tasks during the data migration process
In each stage of the data migration project, the market parties repeat the process stages
until the data quality meets the acceptance criteria for the checkpoint.
Production of data migration files Each market party produces the data
specified by the data standard from its own data systems for the migration files.
The structure and data content of the migration files are described in greater detail
in the document, Siirtotiedosto-ohjeistus (Appendix 1).
Importing migration files to Titta. The market parties upload the master data to
the Titta service via an interface. Metering data is transferred to Titta via a
separate SFTP connection.
Data checking. Titta checks that the data delivered by the market parties
complies with the data standard. A detailed description of the checks is presented
in the document, Data checking in Titta (Appendix 3).
Processing of observations. Titta produces party-specific check reports by
source data. The check reports are in xlsx format and they are saved in the file
archive. The parties can retrieve their own reports from the Titta service interface.
Data correction and enrichment. Based on the check reports, the market parties
correct the data, enrich data in the source systems or correct the data extraction
tools.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 35 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
Tasks to be performed by Fingrid Datahub Oy. Fingrid Datahub Oy is responsible for
transferring the data to Datahub and verifying the data migration.
Data transfer to datahub. Data that has passed the check is transferred to the
Titta service publishing area, from where it is available to the Datahub system.
Data uploading to datahub. The Datahub system retrieves data from the Titta
service and uploads it to its own database.
Data migration verification. Verification of data migration is performed on the
basis of the data migration reports produced by Datahub by comparing the reports
to the data transferred to Datahub. The market parties can utilise the Datahub
reports to ensure that the data uploaded to Datahub corresponds to the data in
the source system.
The following chapters describe the work stages on a more detailed level.
4.3.1 Production of data migration files
In order to ensure reliability and performance, the production of migration files must
primarily be implemented as a semi-automated system function and the use of manual
phases is not recommended. The migration files must be produced from up-to-date
material that corresponds to the market situation.
The importance of reliability and performance is emphasised when running consistency
checks, in which case each market party must deliver up-to-date data to Titta in a very
short time so that it is possible to ensure that every party’s data is current. Minimisation of
run time is also of utmost importance in conjunction with Datahub commissioning.
However, migration files that contain only a small amount of data, are nearly unchanged
by nature, and can presumably be made completely error-free with little work can be
created manually. These migration files are:
Party information
Metering grid area data
Exchange point data
Production units, depending on the number of units
4.3.2 Import and checking of master data
The market party delivers the source data as migration files to the Titta service for
checking. The source data should be extracted from the production system or from a
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 36 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
system containing up-to-date data equivalent to production data. In each data upload, the
party delivers a migration file containing all the rows of master data, not only the changed
rows. However, this does not mean that all the migration files have to be delivered in
every upload. For example, the party can deliver only customer information several times
as part of the data quality improvement work.
Delivery of the material takes place as self-service, in which case each market party
handles their own schedules and ensures that all the material is delivered to Titta for
checking.
If necessary, source material is also delivered in a coordinated and simultaneous manner
by the entire industry. The market party’s obligation is to ensure that the source material
is delivered to Titta at the requested time.
The following checks are performed on the source data in Titta:
File check: The format of the migration file delivered by the party complies with
the data standard.
Integrity check: The source data delivered by the party is internally complete
from the perspective of the data standard and Datahub business processes.
Consistency check: The source data delivered by the market party is consistent
with the material supplied by other parties.
The market parties monitor the quality of the uploaded data by means of reports
produced by the Titta service, which can be downloaded from the user interface. The
check report lists all the errors found in the data, including error type and category, data
content of the entire incorrect row, and a possible correction recommendation.
The following sections provide an upper-level description of the data migration checks. A
detailed description is provided in Appendix 3.
4.3.2.1 File check
The file check involves checking an individual migration file to ensure that its format
complies with the specification in Appendix 1. In addition, every data field is checked
against the Datahub data standard.
Quality requirement levels are specified for the material imported in the data migration,
and the manner in which errors in different quality requirement data affect importing of the
data is specified in the Titta service. For example, errors in quality requirement level 1
data cause rejection of the entire row. More detailed specifications related to the check
processes are provided in Appendix 3.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 37 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
4.3.2.2 Integrity check
After a market party has delivered all the mandatory migration files for its role, an integrity
check can be performed on the data. This involves checking the reference integrities
between the migration files. For example, a customer specified in the agreement must be
found with a given ID from customer information imported to Titta by the market party.
Integrity check also takes into account that a value of a field in one migration file may
depend on a value of another field in another migration file. The comparison only targets
data uploaded by the party itself.
The market party launches the integrity check when the migration files containing the
reference integrities have been delivered to the Titta service.
4.3.2.3 Source data consistency check
Fingrid Datahub Oy initiates a Datahub-level consistency check after the parties have run
integrity checks and the quality measured in the file and integrity checks is at a level that
makes it appropriate to run consistency checks.
The following checks are done as part of a consistency check:
Reference integrity checks between parties (for example, the accounting point ID
in an electricity supplier’s agreement must be found in the distribution system
operator’s material).
Unambiguous periods of validity (for example, the same accounting point may not
have agreements simultaneously valid between two different suppliers).
Functional integrity (for example, an accounting point has a sales and a grid
agreement, metering time series are available for the validity of the agreements).
Duplicate checks (for example, the same accounting point may not have several
records).
Only material that complies with the data standard data model is published in Datahub.
Rows in which quality requirement level 1 errors have been observed are not transferred
to Datahub. Final approval of the source data is done in this stage.
A consistency check is not performed on a party-specific basis, so the progress of its
process is not visible in the party’s user interface.
4.3.3 Processing of observations produced in the Titta service
The task of the market parties is to check the error reports produced by the Titta service
and, if necessary, correct the material in their own source system as required. The errors
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 38 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
in the reports may also be due to an error in the extraction tool, in other words, the error
may not necessarily be in the source system data.
Performance of file and integrity checks and correction of the material based on the error
reports produced by these checks takes place as self-service. Each market party ensures
that all of its material meets the quality requirements described in chapter 3.2 in each
stage of data migration.
Each market party is responsible for the correctness of its own material.
4.3.4 Import and checking of metering data
The distribution system operators, transmission system operator and regional grid
operators deliver metering data via a separate SFTP server. The Titta service user
launches the metering data check after the metering data and master data have been
imported to Titta.
Among others, Titta carries out the following checks:
The format of the migration file structural data complies with the specification.
The metering point ID in the metering data can be found in the master data.
The active metering points reported in the master data can be found in the
metering data.
The time series for energy leaving the metering grid area and energy entering the
metering grid area have been imported for the exchange points (from the metering
responsible party’s perspective).
4.3.5 The transfer of data to the Datahub system
Once a market party’s data has passed the consistency check and meets the quality
requirements specified for data acceptance (see chapter 3.3), the data is transferred to a
separate publishing area. The Datahub system uploads the data from the publishing area
to the Datahub system database.
4.3.6 Data migration check
After the datahub data upload, the datahub system creates so-called data migration
reports, which are returned to Titta. The data migration reports contain all master data
uploaded to Datahub on a market party-specific basis.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 39 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
The Titta service publishes the data migration reports for the market parties so that they
can compare the data in the data migration reports with the data in the source systems,
and thus verify that the data in Datahub and the source systems is consistent.
Titta also checks that the data uploaded to Datahub corresponds to the data transferred
to the Datahub system. Based on the check results, the Titta service produces reports
about possible deviations for the administrator users.
With regard to metering data, Datahub delivers the following reports by distribution
system operator:
A list of accounting points, production units and exchange points that are missing
all or some metering data during the period of validity of the grid agreement.
Metering time series that Datahub was not able to link to a master data structure.
Metering grid area-specific loss series.
Supplier-specific total series by metering grid area.
That is, Datahub does not return the uploaded metering data to Titta.
4.4 Data check against external data sources
In the mass data check against an external register carried out in stage 2 and 3 of the
data migration project, each market party must ensure that all basic data have been
delivered to the Titta service prior to the check so that the basic data is included in the
check. After the mass checks, the system will begin using real-time on-line checking in
conjunction with file checks.
The checks performed and the registers utilised in the checks are described in more
detail in the document, Tietojen tarkastus Titassa (Appendix 3).
4.5 The deployment of new IDs
GSRN accounting point IDs and GLN party IDs will be deployed in stage 3 of the data
migration project, and every market party must obtain these IDs itself. The instructions
are available in the EDIEL.fi portal in the document "Use of GS1 IDs in the electricity
retail market" (https://www.ediel.fi/Datahub/business-processes/Datahub-ohjeet).
In addition, several coordinated checks and other actions that apply to all market parties
will be carried out in the data migration project. Participation will be obligatory.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 40 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
5 Data security for the data migration service
5.1 General information about data security for server rooms
Data security controls are observed in the server room service provider’s server rooms.
The server rooms meet the requirements of Vahti classification and all the service
provider’s services are Katakri-audited.
The confidentiality, integrity and availability of data have been taken into account when
implementing data security. The server room service provider has a data security policy
approved by the management and a valid ISO20001 certificate that sets requirements for
data security policy, including a deviation management process and risk management.
The data security practices of the server room service provider are based on the
accompanying certification requirements and on ITIL practices:
KATAKRI II auditing was acceptably completed in 2016.
The ISO20000 IT service management system was certified in 2017.
The ISO27000 information security management system was certified in 2017.
5.2 Data ownership
Ownership of the data delivered to the data migration service by the market parties
remains with the market parties. The obligation to secrecy is specified in the service
agreement between Fingrid Datahub Oy and the market parties and in the agreements
between Fingrid Datahub Oy and the subcontractors.
Data will not be transferred outside the borders of Finland during any stage of Fingrid
Datahub Oy’s processing.
5.3 Maintenance of data security
Datahub’s data migration service applications were data security-audited prior to the start
of the data migration pilot stage and again after the commissioning project.
The following general principles apply to data security auditing:
All medium-serious and serious observations must be corrected in order to allow approval of the service’s data security and so that the service can be taken into production use.
All new external and inter-application interfaces must be data security-audited before production use.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 41 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
If necessary, the service will be audited again if significant changes or a large number of
functional changes are made to the service.
6 Data protection for the data migration service
In terms of data protection, Fingrid Datahub Oy acts as a personal data processor within
the scope of the data migration service. In other words, Fingrid Datahub Oy processes
personal data on behalf of the market parties. When the data is transferred to the
Datahub system, a personal data register is formed from the data, and Fingrid Datahub
Oy is the registrar.
Data protection for the data migration service is described in Appendix 3.
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 42 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
7 Definitions
Terms and abbreviations
Description
Datahub A centralised information exchange solution for the electricity retail market.
Data standard A specification that describes the Datahub business data and dependencies between the data.
Integrity check
A check on the source data of one market party carried out by Titta service. The system ensures that the source data supplied by the market party is complete with regard to the data model in the data standard. The integrity check is a different concept than the consistency check, which examines the consistency of the source data at the market level.
Fingrid Datahub Oy A subsidiary established for the operational activities of Datahub and wholly owned by Fingrid.
Business process A group of tasks linked to each other, which are done in order to achieve a set target, for example, the switching of a customer's supplier.
Source data Data delivered to the Titta service by the market parties.
Master data Master data are all the data delivered to Titta in migration files.
Migration file A file used in data migration that is used to export data from the source system to Datahub in the specified format.
File check Checking performed on an individual file of the data migration service. This includes a syntax check, logical checks of data content and duplicate tests.
Market party
A company that has signed a data migration service agreement with Fingrid Datahub Oy. In conjunction with Datahub data migration, Market party is also used to refer to companies that use the Titta service but do not have a retail electricity market party ID.
Data migration service
A system service in which a check of data coming from market party source systems is carried out, and about which possible deviations are reported back to the market parties. The data migration service gives summary data to Fingrid about the progress of the migration process and what stage each market party is at. The name of the data migration service is “Titta”.
Data migration work group
A group of 11 pilot companies (pilot group) selected by Finnish Energy that, together with Fingrid and the data migration plan partner, participates in specification of the documentation and data migration process associated with data migration.
Data model A data model is an abstract model, which organises data fields and specifies how they are connected to each other.
Consistency check A check performed on the entire source data by the Titta service. The system checks that the source data delivered by different market
Data Migration Plan 2.1 – UNOFFICIAL
15.4.2020
Finnish version updated
27 March 2020 43 (43)
Fingrid Datahub Oy
Street address Postal address Telephone Fax Business ID 2745543-5, VAT
reg.
Läkkisepäntie 21 P.O. Box 530 [email protected]
00620 HELSINKI 00101 HELSINKI +358 (0)30 395 5000 +358 (0)30 395 5196 www.fingrid.fi
parties is unambiguous and non-contradictory. The consistency check is a different concept that the "integrity check", which is performed on source data delivered by a single market party.
8 Appendices
Appendix 1: Migration file instructions.
https://www.ediel.fi/Datahub/tietokonversio/siirtotiedosto-ohjeistus
Appendix 2: Data check in Titta: Source data check in Titta service
https://www.ediel.fi/datahub/tietokonversio/siirtotiedosto-ohjeistus
Appendix 3: Data protection for the data migration service
https://www.ediel.fi/Datahub/tietokonversio/tietokonversiopalvelun-tietosuoja
Appendix 4: User instructions for Titta
https://konversio.Datahub.fi/Content/Tietokonversiopalvelun_kayttoohje.pdf
Appendix 5: Description of the data migration service
https://www.ediel.fi/sites/default/files/Liite%201%20Tietokonversiopalvelun%20kuvaus.pd
f