the declarative approach to database schemas23276.pcdn.co/wp-content/uploads/2018/11/douglas... ·...

61
The Declarative Approach to Database Schema Upgrades in Magento 2 My XML Schema brings all the boys to the yard…

Upload: others

Post on 30-May-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

The Declarative Approach to Database Schema Upgrades in Magento 2

My XML Schema brings all the boys to the yard…

Page 2: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Douglas Radburn Head of Development at Pinpoint Designs

- Working with Magento for 8 years

- Previously a backend developer

- Single Father to 3 under 3.

- Love PWA

Twitter: @douglasradburn

About Me

Page 3: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Some History

Page 4: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Magento 1

Page 5: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Don’t just cover database structure:

- Attributes being added to Products, Categories, Customers etc

- CMS Block content

- Database data

Setup Scripts

Page 6: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

People include files just because they can or they think they should.

Blank Files

Page 7: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

In an evolving site, you’re going to end up with a lot of setup scripts for some modules, and not so many for others.

Keeping track of these can be a royal pain

Lots of Files

Page 8: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Upgrade scripts will have missing numbers.

In theory, nothing happened to structure/data in that version.

Or, the dev didn’t commit something …

Or, two devs committed the same thing

File Sequencing

Page 9: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

If you’re looking at your database and your column type is TEXT.

You open mysql-upgrade-1.0.0-1.0.1.php

It’s defined as a VARCHAR.

You’ve got 20+ files to look through and see how it made it to a TEXT type (and whether it ultimately should be)

Hard to follow

Page 10: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Database Specifics in M1

Page 11: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

RAW SQL

Implementation

Page 12: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

RAW SQL

Implementation

Page 13: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Since version 1.6, Magento (in theory) supports more database backends than MySQL only

DDL (Data Definition Language)

Abstraction from direct SQL

Implementation

Page 14: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Magento keeps track of the “current” version loaded in the core_resource table.

Version number tracked against the version supplied in etc/config.xml for your extension / for the core module.

Keeping Track

Page 15: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Magento 2

Page 16: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

InstallData and InstallSchema - Scripts, that are executed on a clean (empty) database

UpgradeData and UpgradeSchema - Incremental scripts, that update an existing Magento database

Implementation Changes

Page 17: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

The logic moves from Magento reading file names to Magento utilising version lookups within code using version_compare.

DDL still exists

RAW SQL still exists

Problems still exist

Logic

Page 18: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Magento keeps track of the “current” version loaded in the core_resource table.

Version number tracked against the setup_version supplied in etc/module.xml for your extension / for the core module (not composer version)

Keeping Track

Page 19: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

The FUTURE

Page 20: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Structure declarations

Page 21: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Magento has the answer

Page 22: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Everyone knowsthat developers

Fucking Love XML

Page 23: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

But seriously,how can structure declarations help?

Page 24: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Avoids missed or repeated SQL operations

Page 25: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Dry-run mode for SQLstructure and data

Page 26: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Performance improvements

Page 27: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Database data rollback

Page 28: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Ability to revert to a previous version

Page 29: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Database structure validationbefore installation

Page 30: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Ability to modify core database structure

Page 31: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Developers can modifyexisting columns

Page 32: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Show Me

Page 33: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

<Module_Vendor>/<Module_Name>/etc/db_schema.xml

Declares a module’s database structure

Page 34: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

lib/internal/Magento/Framework/Setup/Declaration/Schema/etc/schema.xsd

Reference for what a module’s database structure can be

Page 35: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Some Examples

Page 36: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table
Page 37: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

<table name="directory_country_region_name" resource="default" engine=“innodb” comment="Directory Country Region Name">

</table>

Page 38: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

<table name="directory_country_region_name" resource="default" engine=“innodb” comment="Directory Country Region Name"> <column xsi:type="varchar" name="locale" nullable="false" length="8" comment="Locale"/> <column xsi:type="int" name="region_id" padding="10" unsigned=“true" nullable="false" identity="false" default="0" comment="Region Id"/> <column xsi:type="varchar" name="name" nullable="true" length="255" comment="Region Name"/>

</table>

Page 39: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

<table name="directory_country_region_name" resource="default" engine=“innodb” comment="Directory Country Region Name"> <column xsi:type="varchar" name="locale" nullable="false" length="8" comment="Locale"/> <column xsi:type="int" name="region_id" padding="10" unsigned=“true" nullable="false" identity="false" default="0" comment="Region Id"/> <column xsi:type="varchar" name="name" nullable="true" length="255" comment="Region Name"/> <constraint xsi:type="primary" name="PRIMARY"> <column name="locale"/> <column name="region_id"/> </constraint>

</table>

Page 40: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

<table name="directory_country_region_name" resource="default" engine=“innodb” comment="Directory Country Region Name"> <column xsi:type="varchar" name="locale" nullable="false" length="8" comment="Locale"/> <column xsi:type="int" name="region_id" padding="10" unsigned=“true" nullable="false" identity="false" default="0" comment="Region Id"/> <column xsi:type="varchar" name="name" nullable="true" length="255" comment="Region Name"/> <constraint xsi:type="primary" name="PRIMARY"> <column name="locale"/> <column name="region_id"/> </constraint> <constraint xsi:type="foreign" name=“DIR_COUNTRY_REGION_NAME_REGION_ID_DIR_COUNTRY_REGION_REGION_ID" table="directory_country_region_name" column="region_id" referenceTable=“directory_country_region" referenceColumn="region_id" onDelete=“CASCADE" />

</table>

Page 41: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

<table name="directory_country_region_name" resource="default" engine=“innodb” comment="Directory Country Region Name"> <column xsi:type="varchar" name="locale" nullable="false" length="8" comment="Locale"/> <column xsi:type="int" name="region_id" padding="10" unsigned=“true" nullable="false" identity="false" default="0" comment="Region Id"/> <column xsi:type="varchar" name="name" nullable="true" length="255" comment="Region Name"/> <constraint xsi:type="primary" name="PRIMARY"> <column name="locale"/> <column name="region_id"/> </constraint> <constraint xsi:type="foreign" name=“DIR_COUNTRY_REGION_NAME_REGION_ID_DIR_COUNTRY_REGION_REGION_ID" table="directory_country_region_name" column="region_id" referenceTable=“directory_country_region" referenceColumn="region_id" onDelete=“CASCADE" /> <index name="DIRECTORY_COUNTRY_REGION_NAME_REGION_ID" indexType="btree"> <column name=“region_id"/> </index></table>

Page 42: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Perform common database operations

Page 43: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Create a table

Page 44: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Drop a table

Page 45: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Create a foreign key

Page 46: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Add a column to table

Page 47: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Drop a column from a table

Page 48: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Change the column type

Page 49: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

To rename a column, delete the original columnand create a new one.

UseonCreate=“migrateDataFrom(entity_id)"

onCreate="migrateDataFromAnotherTable(catalog_category_entity,entity_id)"

Rename a column

Page 50: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Add an index

Page 51: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Things you can’t do

Page 52: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Converting to DB Schema

Page 53: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

To prepare a module for declarative schema, you must:

Develop a data patch and/or a schema patch

Configure the declarative schema for your module

Convert upgrade scripts to declarative schema- if your module already has upgrade scripts.

Once a module is converted to the declarative schema approach, it cannot be reverted to upgrade scripts.

Page 54: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Magento provides a Schema Listener Tool to convert pre-Magento 2.3 migration scripts into declarative schema

magento setup:install --convert-old-scripts=1

magento setup:upgrade --convert-old-scripts=1

Page 55: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

The tool supports only DDL operations from \Magento\Framework\DB\Adapter\Pdo\Mysql.

The tool ignores all raw SQL in your InstallSchema or UpgradeSchema scripts.

Any DDL statements in a Recurring file will not be transferred to the new schema, because this file should be designed to run during

each installation or upgrade.

Page 56: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Safe Installations

magento setup:upgrade --safe-modemagento setup:install --safe-mode

Rollback

setup:upgrade --data-restore

Page 57: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Backward Compatibility

Declarative schema does not automatically delete database tables, columns or keys that are not defined in a db_schema.xml

<module_vendor>/<module_name>/etc/db_schema_whitelist.json

Can be generated withmagento setup:db-declaration:generate-whitelist --module-name=MODULENAME

Page 58: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

We essentially list out all of the elements we’ve created via XML schema declarations.

JSON Format

Page 59: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

This is now a lot harder…

Nothing is stored in setup_module anymore as it is state dependant on the db_schema.xml during setup:upgrade.

Keeping Track

Page 60: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Summary

Page 61: The Declarative Approach to Database Schemas23276.pcdn.co/wp-content/uploads/2018/11/Douglas... · Magento keeps track of the “current” version loaded in the core_resource table

Twitter: @douglasradburn

That’s all folks

🌈⭐