Working in an open source community is mostly a fantastic experience. Folks from all over the world from all walks of life come together to build something better for that community. Ever since the Drupal community adopted Symfony (and Composer) we’ve been exposed to an even broader community of PHP developers and plugins / libraries that we consume as part of our sites and platforms.
This has also introduced a whole new dependency chain for getting things done. I’d go so far as to identify this as a new(ish) challenge for the Drupal world; we certainly never ran into these problems back in ye olden days of Drupal 7 (and before) where dependency management was something those “other projects did.” That challenge? How do you make meaningful contribution to projects you don’t own/control but you do in fact depend on.
XKCD perfectly captured this scenario some years ago.
Necessary Changes
Often when we start talking about upstream dependency changes, there is some event that starts us down a path of updating. In the Drupal world that is PHP 8.1. I’ve been working off and on during 2022 on Acquia’s Build and Launch Tools (BLT) to get ready for the Drupal 10 release (coming December 2022). Ironically, this update to PHP 8.1 has been much harder than previous updates in the PHP world (even a major version update from PHP 7.4 to 8.0). Why? It comes down to Symfony versions. Symfony 3/4/5 all work with PHP 7.4 and 8.0. But, Drupal 10 requires Symfony 6 which requires PHP 8.1. So, any projects (like BLT and many of its upstream dependencies) that directly require Symfony components require quite a bit of refactoring.
Let’s follow this dependency chain (and explore why it’s so complex). BLT uses grasmash/yaml-expander. This package only has 2 dependencies (see its composer.json): one is symfony/yaml and the other is grasmash/expander. grasmash/expander requires several things (see its composer.json), but the big one is dflydev/dflydev-dot-access-data. So, to get BLT up to date we had to get yaml-expander, expander, and dflydev updated. None of which the BLT team actually own.
The next challenge is that due to the way composer dependencies are calculated, you can’t easily patch upstream dependencies’ composer files. Meaning, that if I want to make BLT compatible with PHP 8.1, I can’t just locally patch the composer.json file for one of these packages. Someone actually has to make the change (could be me) in the upstream repo and merge it (definitely wouldn’t be me) so that BLT (or BLT’s dependencies) can consume an updated version of that plugin. On the BLT front this resulted in numerous tickets to update the dependencies in their own repositories so that we could actually consume these changes (see https://github.com/acquia/blt/issues/4504 for some links out to other repos and their changes).
Thankfully, the maintainers of these other projects had either already made changes themselves or were very receptive to our suggested changes and we got the changes and releases done. In some cases I did reach out to maintainers on Slack / Github to ask for help getting things merged (which helped greatly).
Impact
The biggest downside to having upstream blockers is that you can’t update your own package until those blockers are out of the way. So, in this context, BLT was actually VERY incompatible with PHP 8.1 / Symfony 6. Symfony had refactored all of its command structure in the background but we didn’t realize this. Once we managed to get all of the dependencies updated and do the PHP 8.1 and Symony 6 update, then we also had to go back and refactor nearly ever command structure in BLT to be compatible.
Today we are in fantastic shape to support Drupal 10 well ahead of the release, but we started this work early in 2022 to make sure we’d be ready on time. I’d also point out that BLT is a bit of a different experience than many Drupal community module maintainers might experience. Since most Drupal modules don’t have to pull in Symfony directly, they haven’t had to deal with some of the same level of refactoring and changing that other PHP plugins that do require Symfony (and other PHP libraries) directly.
At the end of the day, this whole PHP 8.1 update has really illustrated the need to plan ahead and take into consideration what’s going on with your ecosystem. Testing is critical. Working and collaborating with folks who you may never have interacted with before is critical. And it again restores my faith in the community to see so many folks coming together to contribute and work on this update. This is a fantastic opportunity for you to get involved… updating dependencies isn’t a super difficult process, it just has to be done in many different places. I promise you, the maintainers will be thankful for your help.
How to Update an upstream dependency
Let’s look at my PR that I opened for https://github.com/grasmash/yaml-expander/pull/26.
In this case, I:
forked the yaml-expander repository and pulled the code locally
opened an issue (see https://github.com/grasmash/yaml-expander/issues/25)
updated my local machine to use PHP 8.1
ran a composer install (which of course, failed)
updated the yaml-expander composer.json to pull in appropriate PHP 8.1 compatible dependencies
"grasmash/expander": "^1.0.0 || ^2",opened pull request
collaborated with Matthew Grasmick to get the PR merged
In this case, I also worked with him on the expander plugin as well.
This whole process took me about 5 minutes to do. It was only a handful of commands…
fork (in Github UI)
git clone (both my fork and upstream)
composer install
brew uninstall php@8.0
brew install php@8.1
modified composer.json
composer install (again)
git commit / git push
opened PR
In Conclusion
It is absolutely time to start thinking about your Drupal 10 and PHP 8.1 update. Obviously you have plenty of time to do it, but if you have an established project it’s not too early to start testing these updates.
Related Content
Dependency trees can be massive, and with the upcoming Drupal 10 release PHP 8.1 could have a significant impact on your project.