I’ve blogged a bit about updating the Drupal GovCon site to Drupal 9 in recent weeks. Now that we’re there, the next major revamp I’m focused on (post 2020 conference) is to update our media setup on the site. Why? Well, there’s a few reasons:
We are “using” media (in that it’s configured) but we are hardly using media anywhere on the site. This means that our images and videos are stored in image and video embed fields on nodes (vs. using an entity reference to media images and media videos). In general, I strong recommend using Media for media! So this is an opportunity for me to follow my own advice.
Lightning Media will be deprecating support for the video embed field module and moving to Drupal core’s embed options. While this isn’t a “required” update, I want to try and stay in-line with Lightning’s recommendations.
Similarly, Lightning is removing support for Entity Embed (in favor of Drupal’s Media Library) and Views Infinite Scroll.
Finally… the GovCon site hasn’t been significantly re-architected in 2+ years. As a result of that, there’s a lot of config bloat and duplication. Just look at the list of Media types on the site:
For those of you keeping score at home, that’s 2 audio and 3 video types (that’s a little much).
What i did and what this post covers
OK so here’s the high level list of what I am going to cover in this post:
cleaning up entity config for media to remove duplicate / unused media (yes, this should have been done a while ago and/or the config should never have been merged. no argument there!)
add an entity reference to the session content type to reference remove videos
migrate all existing “embedded” youtube videos out of the video embed field, create remote video media, and reference
remove the old video fields
disable old modules / update config (e.g. replace Entity Embed with Media Library)
Moving Duplicate Media Bundles
Disclaimer: I want to start this section by saying what I did is highly destructive. Remember, configuration directly impacts content. If you change config (e.g. delete an entity) and you have content in that entity bad things will happen. Hopefully Drupal will stop you from doing this… but I make no promises!
So the first thing I did here was carefully review my /admin/content/media page. GovCon currently is minimally using “real” Media so there wasn’t much there. The reason for this review was to see specifically which bundles were in use. As it turned out, we only had images and videos (and nothing else).
The next thing I did (locally) was actually delete all of the media entities out of my local Drupal through the UI. Note that in my case Drupal DID warn me for Images and Videos that there was content and therefore could not delete.
I exported the config (to actually capture the deletion).
Then I did a clean installation of Drupal and Lightning. Why? Lightning will automatically install all of its media bundles (via Lightning Media) with the desired config state. Remember I wasn’t actually using any of the Media bundles other than Images / Videos. So in my case, nuking and rebuilding the config was no big.
On Drupal installed, I exported my config again and carefully diffed all of the media config. It fell into one of three categories:
It was “the same” as it was before with only a UUID change. I discarded these changes and left my original config unaltered (before I started this process). Warning: changing UUIDs on existing bundles will cause Drupal to try and delete / recreate the bundle. This is not good if you have content!
It was no longer needed (meaning that when I deleted it, it stayed deleted).
It was out of date (meaning that the reinstall of Lightning changed it).
Obviously in scenarios 2 and 3 I did config changes. You can see those in this PR.
The end result of this cleanup was a similar (but shorter) list of media:
There were also lots of little improvements to the fields, display modes, etc. due to the more current version of Drupal and Lightning Media. Good changes all around!
Removing Entity Embed
The other big thing that happened during this activity was config changes to swap to Media Library from Entity Embed. The re-install / config dump caught most of these. The only outstanding config that was still needed to change was in my Text Format config (my WYSIWYG was configured to use the Entity Embed vs. Media Library, so I had to make a few additional tweaks, which you can see in this commit).
Dealing with the “Content” Changes
This is one of those scenarios where I really wish Drupal would let you change a field type. But you can’t. So here’s the scenario:
All my videos are in a field that is going away
I need to add a new field (that references media)
I need to create media
I need to reference media
I need to remove old field data
While it may be “possible” to do this with a migration, I just did it with a simple update hook. Warning: I did this across 3 PRs that I fully deployed to production. Here’s why!
My sessions did not have the necessary fields for the media reference. These needed to be added before I could migrate data. So, the first PR added the fields.
I needed code to do the “migration.” Obviously this code wasn’t in prod. So I went ahead and deployed this after the previous PR was deployed. Note: I could have written some code that wasn’t a true “database update” and run it manually. I opted to do a database update that would run automatically as part of my deployment.
I needed to remove fields (again, a config change). Once the data was migrated I did a third deploy to remove the fields.
Remember! Order of operations are critical. The three tasks I listed above fall into 2 categories:
config changes
add new field
remove old field
update hook / database update
migrate the data
A good configuration workflow runs database updates first and then executes config updates. Meaning that the database changes (which require config in this case) would fail. While you can do “config changes” as part of code in the update hook… in this situation I saw no reason to write an infinitely more complex database update when I could just do a few successive deployments. Your situation may not be the same!
Just remember:
you can’t migrate data into fields that haven’t been created yet
you can’t migrate data from fields that have already been deleted
Order of operations is critical!
The code that I ultimate wrote for this update isn’t super fancy. Here it is in the PR.
first update hook:
Creates new taxonomy terms for Video Type (new Taxonomy created as part of this process)
second update hook:
Finds the taxonomy term id for one of the new video type terms
Looks for all existing sessions with an embedded video
Looks for Media that has the YouTube URL found on the video
Creates new Media if non exist
Reference Media (either existing or New) in the Session
Removes the old Video data
Updates the Session
In our case, we had roughly 80 sessions with videos (out of a few hundred sessions that have been created). The script isn’t super fast, but it runs in under a minute (and it only has to run once). So I didn’t focus too much on performance optimization.
The end result? I have a bunch of new media on my production site and all my sessions are properly referencing the media. I have a final PR that will go and remove the old video field from sessions to avoid future confusion.
In Conclusion
“Controlled burning, also known as prescribed burning, involves setting planned fires to maintain the health of a forest. These burns are scheduled for a time when the fire will not pose a threat to the public or to fire managers.”
This is a pretty disruptive change to a site, and it’s something that should be carefully tested and done at a time that if something does go wrong, you have the opportunity to roll back. Having said that, doing “spring cleaning” with your architecture from time to time can be a really healthy solution. Much like with a controlled burn in an overgrown forest, the end result can be a much healthier ecosystem. But, also like a controlled burn, the operative word is “controlled.” You don’t want to do this sort of update and just “hope it works out.”
Make a plan. Test your plan. And implement your plan in a way that isn’t going to light the rest of your content on fire.
By the way, a quick shoutout to the Lightning team and all the hard work they are doing to prepare for Lightning Media 5 (particularly Adam Hoenich, Adam Balsam, and Katherine Druckman). There continue to be many many improvements to Drupal core’s Media features (so the team has their work cut out for them in keeping up with these improvements).
I also want to acknowledge that most of the notes in this post are things I personally opted to do and are not strictly speaking required to upgrade to Lightning 5! But please carefully note the deprecated modules and changes coming, as you may need to at least add modules (like Entity Embed, Views Infinite Scroll, etc.) to your composer.json file to keep using them after the update. As always, Lightning’s release notes will tell you what you need to do to upgrade.
Photo by Breno Machado from StockSnap
Related Content
An overview of all the things I’ve been trying / testing to get ready for PHP 8.1.