I keep hearing from everyone that Drupal to Drupal migration is a slow, costly task. This is probably why the latest statistics as of April 26, 2020, reported 680,725 running Drupal 7 copies, nearly twice as many as the 358,427 instances reported for Drupal 8+.

There are quite a few cases where this finding really stays true; however, I feel the need to share a few comments on the mistakes often made during Drupal 7 → Drupal 8+ migration.

Using fully custom migrations

When the Migrate API was committed to Drupal core, the usual concept was fragmented: we suddenly had two types of migration:

  • The migrations in Drupal core became plugins whose plugin definition (“configuration”) are stored in YAML files in the module’s migrations subfolder.
  • And at the same time, Migrate Plus introduced a configuration entity which acts as any other configuration entities in core: it can be exported / imported; but in the meantime it only acts as an intermediate layer between core’s migration plugin discovery and the instantiated migration plugin instances.

Before you start writing any migrations, or start creating the content types, vocabularies and the fields of the entities on the destination site, please check out what you have in the module’s migrations subfolder! Don’t be surprised: the migration plugins in Drupal core can handle the migration of not only your content but also your configurations (including for instance your content types, field storages, field instances, view mode configurations)!

And my suggestion in regard to Migrate Plus migration configuration entities is: do not use them at all. Their only feature is that if you break their structure, you cannot use your Drupal instance until you find what’s wrong. And you have to re-import them every time you do a small change in them. Create a standalone migration-specific custom module, and implement the missing migrations in the module’s migrations folder.

Because of the above, I strongly suggest not to use Migrate Upgrade at all. If you have to make a small fix to a core or contrib module provided migration, then use a hook_migration_plugins_alter() implementation!

Content structure change is performed with the migration

I often encounter cases where the structure of the content is also changed during the Drupal 7 → Drupal 8+ migration:

  • Content types are merged together.
  • We migrate into a field with a different name than we have on the source.
  • Certain fields aren’t migrated at all.

I acknowledge that these may be needed indeed, but I must note that they should not be implemented during migration:

On one hand, it is much easier to run the migration while retaining the content structure of the Drupal 7 page and to handle the change in the content structure in the next step. On the other hand, porting the Drupal 7 theme will also be very easy, and using the core and contrib migrations, the upgrade can be done much faster.

If you do it this way, the project can start using a modern, longer-supported foundation much earlier; and the business side is, that the customer can also decide whether it is so important for them to change the structure and create a completely new theme to sacrifice resources for it.

What makes a cool URI? A cool URI is one which does not change. What sorts of URI change? URIs don’t change: people change them.

Please don’t let your URIs be changed! Cool URIs don’t change.

Failures of the source are fixed in migration

One of your content is referencing a completely missing entity? There isn’t any file in the file_managed table with the ID you have in a file field? Or the file cannot be found on Drupal 7’s file system? The author of a content or comment was deleted and does not exist anymore?

These are wrong in the source Drupal 7 instance too! You should fix these issues there! Keep in mind that until you arm your new Drupal 8+ instance, your users will use your current Drupal 7 site. And if you fix the problem in the source, then it will be fixed on the new site as well!

We use imperfect tools

Using Drush and Migrate Tools to execute the migrations is very handy for us, developers. I think we prefer following the process through a terminal rather than in a browser window on Drupal’s user interface.

But unfortunately the Drush command we use has some issues you have to be aware of:

We don’t use joinable source and ID map databases

Are you aware that SqlBase-based migrate source plugins are trying to join the ID map tables to their database query before it is executed? I wasn’t. But it is a brilliant feature, and you should use it whenever it is possible.

How does it work?

If a migration is about to be executed, the first thing that happens is that its source plugin performs a database query, and then it acts as an iterator which returns the data we may migrate to the destination. And in regard of the joinability of the source and the ID map (destination) databases, the migration works slightly different:

If they aren’t joinable, then each time SourcePluginBase validates the actual Row, it checks whether the ID map plugin has any data about the row. And it only prepares and returns the actual source record if the ID map plugin does not have any info about the actual source record2, or if the ID map plugin tells that the actual record needs update3. In every other case, it proceeds by checking the next record. This means that if the actual migration has 1000 records, and no extra queries are performed except these above, then executing the migration performs 1001 database queries while it prepares all the source items: There will be one query sent against the source database, and 1000 ID map queries on the destination site.

However, if joining of the source and the ID map queries is possible, SqlBase will do so: it will initialize itself having only those source records which haven’t been fully migrated yet!

So if you want to save some time and IO load, consider making the source database joinable with the destination.

Summary

I definitely think that Drupal 7 → Drupal 8+ migration is a much simpler task than many people think. And if you are aware of the above, I hope that you will also agree.

Footnotes:

  1. Migrate Drupal UI instantiates the migrations by calling MigrationPluginManagerInterface::createInstances(), the migrate import command of Migrate Tools only uses this order if you ask it to execute any migrations. But during development, we usually execute certain migrations – and because of this behavior, we have to take extra care on the order of the IDs we pass to the command as argument; or use --execute-dependencies. ↩

  2. This check is performed based on the identifiers of the source record: Drupal core’s Sql ID map plugin performs a query to its migrate map table, and if there is no result, then it means that:

    1. The actual record wasn’t ever migrated, or it was rolled back before the current operation.
    2. The migration operation ended up in an uncatched PHP exception which completely broke the process, so MigrateExecutable wasn’t able to save any source-destination map data or migration message with the ID map plugin.

    ↩

  3. This means that the actual row’s ID map plugin returns with the value MigrateMapInterface::STATUS_NEEDS_UPDATE. Drupal core’s Sql ID map plugin stores this status in its migrate_map_* table in the source_row_status column. ↩