This is Part 3 in a series of posts on Drupal 9 (and Drupal 8) configuration management. If you haven’t yet read parts 1 (Introduction to Drupal 9 Configuration Management) and 2 (Drupal 9 Configuration Management Strategies) I would recommend that you check these out before diving into this one!
As an architect I care a lot about development workflow. When I’m leading a team of any size, but particularly a large team, the collaboration between the developers on the team is a critical part of the success of our team and project.
What is a Development Workflow?
Development workflow is the series of steps each developer takes to get their work done. Typically this includes steps such as picking up a story from Jira (or another ticket manager), creating a new Git branch, doing development, writing tests, opening a pull request, integrating the work, deploying, etc.
While the specifics of the workflow may change a bit depending on the project, the specific technologies that are required to accomplish our goals, etc. the basics are typically the same. More, we work hard to ensure that each developer on the team is working more or less in the same way, with the same tools and processes to ensure that we’re all swimming in the same direction.
Development workflow is a large topic and can encompass many smaller tasks and workflows. I recently wrote about critical components of a Devops workflow for Drupal 9. Today we’re going to dig into another component of the development workflow: the configuration management workflow.
What is a Configuration Management Workflow?
Drupal 9 (and Drupal 8) is missing some critical components for managing configuration. Oh sure, it gives you the ability to export and import your configuration files. But there’s a lot more to a development workflow than just throwing your work someplace. You should be testing your work. You should be integrating your work. You should be doing this work in a development environment and then deploying it. Fundamentally, all of these processes require scripting / task runners to automate. Unfortunately, Drupal just doesn’t provide the necessary tools to do any of this automation. The out of the box tools are UI-based and require direct human interaction. Not ideal!
Thankfully, as is frequently the case, the community has provided. More on this later in the post.
As I discussed in my Configuration Management Strategy post, there are a number of ways of managing configuration on a Drupal 9 projects. Your configuration management workflow is, in essence, the application and automation of that strategy.
Example Workflow: Config Split
In this example, we’ll be using Config Split to build out subtle configuration differences between our environments on a single Drupal 9 site. The feature we want to build in this example doesn’t require any splits however. We’re going to create a new view for a content type.
Create a new branch in the code off of your integration branch
Perform a clean installation of Drupal (including configuration import) to insure you have the most up to date code + config
Build your view
Export the configuration for your view (most likely, a single YML file)
Commit your code change
Write some tests
Open a Pull Request
Continuous Integration runs
Performs a clean installation
Imports configuration (including your new view)
Tests execute (including your new tests)
And so on.
This workflow will not work with “just” Drupal. In this case, we need Drush for the importing of config. While technically you can export without Drush, it’s much easier to do so with because you can do it programatically AND you can automate / script it. Testing also requires other tools. Drupal core has dependencies on both PHPUnit and Nightwatch, either of which could be used to test your view. You could also use Behat (or any number of other test frameworks). We won’t dig too deep into the testing workflow on this post.
The point is, this workflow allows developers to build new features that are “primarily” site building, but still apply the same rigor to “just a site building task” as one might to a more strenuous development task! I mean, sure, Site Building mostly just clicking around in the Drupal UI. But that doesn’t mean site building is a second class citizen. Site building, if done improperly, can be just as destructive to a site as any PHP. Configuration Management Workflow is the answer to this challenge.
How it all fits together
First, your team should establish its configuration management strategy. This means that you will figure out if you’re doing a complete config sync or a partial. You’ll figure out which modules you’ll be using (config split, config ignore, etc.). If you’re using config split, you’ll figure out what types of splits you’re using, etc. Once you know all this, the config workflow helps you implement the strategy.
At Acquia, we’ve built the Build and Launch Tools (BLT) to help standardize a configuration management workflow. Blt provides everything necessary out of the box to automate the workflow for common tasks like:
installing a clean copy of Drupal (useful for local development and continuous integration)
applying updates to an existing copy of Drupal (useful for local development and deployments).
You don’t have to use BLT (and it’s usefulness is drastically reduced if you’re hosting somewhere other than Acquia) but I definitely recommend looking at it as a model for how you could establish your own tool to help automate your configuration management workflow.
Why Automation is Important
I trust developers. I trust myself. At the end of the day though, the more you can automate a workflow the less likely someone is to forget something. One of the most frequent examples I give is this:
Out of the box, BLT executes both the database update and configuration important commands for you. It always runs those commands in this order:
drush updbdrush cimdrush cim(It does a redundant config import to ensure that all the necessary config split config is properly imported).
Now, if I’m doing code review locally on work that you did and for whatever reason, I do a drush cim first, then a drush updb, it’s possible I will get a different result than you, the continuous integration, and the deployment that all run it in the prescribed order above. By automating, I don’t have to worry about anyone doing it any different. For the BLT users out there, there’s even a command you can run:
blt drupal:updateInvesting in your workflow
Part of the reason BLT exists today is that the Acquia Professional Services team wanted to create a tool that we collectively could use on all our builds. There are a number of supported configuration workflows built into BLT (config split, core only, and features—albeit we don’t use the features option much these days). You could do something similar for your team. There are only so many combinations of available modules and technologies to support the various configuration management strategies. Ideally, your team / organization would come up with your preferred way(s) of handling configuration, and then invest in a suite of tools to facilitate the automation of this workflow / these workflows.
BLT is built on Robo, which is a PHP task runner. BLT does a LOT and you can totally use composer to wire up some basic commands that pull shell scripts together and do things in a certain way. At the end of the day, whether you do something simple or less simple, I hope that you do something. Start to change your documentation and the culture of your team to rely on these tools. We do this with configuration and database updates. We do it with code validation and automated testing. My goal? Lower the bar for using these tools the right way every time. There’s one command. If you can run blt validate BLT will handle everything else for checking code quality. If you can remember blt drupal:update you don’t have to remember the rest of the specifics.
What we’ve seen with BLT is a decrease in startup costs on projects for our teams. All of our architects and developers are trained with it. We use it on (basically) all of our projects. If I swap onto a project I don’t have to learn a bunch about the tooling, because we always use BLT. So, the up front cost / effort / commitment to developing a configuration workflow is non-trivial, but it will continue to pay dividends for years to come as you continue to work with it and maintain it. The more you can standardize around a common tool to manage your configuration workflow, the better.
In Conclusion
At the end of the day, what’s important is that you have an automated workflow for your configuration. Look at BLT! Liberally borrow / steal what we did there. It’s all open source, it’s built on open source technologies, and it heavily relies on things like Drush and Composer that you are hopefully already using every day anyways.
Whatever you do:
make sure that the workflow you piece together supports your configuration management strategy
invest a little bit more effort into building something that is reusable (so it’s not just a one off)
evangelize what you’re doing with the rest of your team and make sure they are on board
use the same tools EVERYWHERE to ensure that local development, continuous integration, and deployments all “work the same way.”
Good luck!
Related Articles
I hit a couple of issues updating to Drupal 9.3, so I wanted to share my experience. Config changes incoming!