I still remember the frustration I felt five years ago while I was experimenting with Drupal 8 for the first time. I had built dozens of Drupal 7 sites by that time, I had attended a couple of DrupalCons where Drupal 8 was the principal topic of many sessions. I was really anxious to start experimenting and dive in.
To do this, I was focused on recreating the combination of technologies on sites that we had built (for instance, https://tethys.pnnl.gov. A site that my team had built in MediaWiki and migrated to Drupal 7 (and the team has since moved to Drupal 8). Tethys is a heavily data-centric site and there are thousands of articles that require search and facets. So, as I was starting to experiment with Drupal 8 one of the very first things I began experimenting with was Search API and Solr. I got as far as enabling Search API and Search API Solr and my whole test site white screened, completely dead in the water.
Managing Dependencies In the Past
In ye olden days (circa 2015) I was still “managing my dependencies” by downloading archived versions of everything and committing the results into version control. The results of this were tens of thousands of files in each git repository. Anytime I wanted to update something, I downloaded it and updated the whole thing in the repository. “Most” of the time if a module needed something to work, it was bundled up in the archive with that module. If it wasn’t, hopefully it was documented in the ReadMe (assuming there was one) and I downloaded another archive with the dependency. It was a familiar, if tedious way of handling things.
It’s also worth mentioning that during this timeframe, if we decided we needed some new PHP library or other addition that was more “server” focused, I would have to chase down someone with administrative access to all of our webservers and have the library added to each. This was a costly and slow process and was not always guaranteed to get done by the time I needed it by.
Jump to that time five years ago when I was dead in the water thanks to enabling Search API Solr. The thing I didn’t realize at the time (and spent quite a bit of time debugging and digging into) was that in order for the Search API Solr module to work, I needed the Solarium library (Solarium is a PHP Solr client library that accurately model Solr concepts.) In my experience with Wordpress and Drupal 6 and 7, if I needed something like Solarium it should have been in the archive with the module. But it wasn’t! So when I finally started seeing the errors and reading into the issue, I was incredibly confused.
How did I get Solarium?
How did I install it on my webserver?
Why wasn’t it included in the module?
Why didn’t I know about it? [NOTE: I didn’t know about it because I didn’t read the README file which says “install this library”]
What Does a Dependency Manager Do?
Basically, all of my questions above have a single answer: Composer (or, insert your favorite dependency manager here. They all do it!).
Dependency managers handle the installation of a dependency, allows for the execution of that dependency (so that you don’t have to do some sort of package installation on your webserver), and it makes it so you don’t “really” have to know about all the underlying requirements of what you’re trying to use.
Take for example the Search API Solr module’s requirements:
"require": {
"php": "^7.2",
"ext-dom": "*",
"ext-json": "*",
"ext-simplexml": "*",
"drupal/search_api": "~1.17",
"consolidation/annotated-command": "^2.12|^4.1",
"maennchen/zipstream-php": "^1.2|^2.0",
"solarium/solarium": "^6.0",
"laminas/laminas-stdlib": "^3.2"
},
If I use composer to add Search API Solr to my project, composer will automatically ensure that each of these requirements is also added to my project. Going a step deeper, Solarium also has a requirements list that must be met to function. Those look like:
"require": {
"php": "^7.2",
"ext-json": "*",
"psr/event-dispatcher": "^1.0",
"psr/http-client": "^1.0",
"psr/http-factory": "^1.0",
"symfony/contracts": "^1.0 || ^2.0"
},There are a couple of duplicates there, but overall, Solarium has some additional dependencies that have to be added to the project in order to function.
As the person “using” composer in this case, all I have to know is how to add the Search API Solr module. Composer handles drawing all of the other dependencies down the dependency tree for me. As long as there are no conflicts, everything will work.
How Does Composer Work?
Composer largely relies on three things to function:
composer.json file (defines all the things)
composer.lock file (records hyper-specific decisions)
Packagist (aggregator for dependencies)
When you make a request with composer (e.g. composer require drupal/search_api_solr) composer reaches out to Packagist (or in this example, the Drupal-specific Packagist to attempt to locate my requested module. If found, the composer.lock file is updated with an entry that references a commit hash.
Once shared amongst a project team, the composer.lock file becomes the tool that allows everyone to get the exact same version of a module.
Let’s look at an example:
This project requires Acquia’s BLT in its composer.json file:
Composer asked Packagist what it knows about BLT. You can see it for yourself, if you’re interested.
The result (as of the time of this writing) was to come back with BLT version 11.4.2. The lock file looks something like this:
{
"name": "acquia/blt",
"version": "11.4.2",
"source": {
"type": "git",
"url": "https://github.com/acquia/blt.git",
"reference": "63fbbddf5ac24b045e997f3835f2ad74a12c2a2f"
},
I’ll save you having to read a ton of json. You get the idea!
Why version 11.4.2
A quick note about composer notation:
When requiring a package like acquia/blt, you essentially have a handful of options:
ask for a very specific version: acquia/blt: 11.4.2
In general, this is bad practice (because it means composer will never update blt beyond 11.4.2). You typically want composer to be able to update your dependencies for you! So, avoid this one.ask for a very broad version: acquia/blt: ^11
In this case, composer will allow you to update to any 11.x release of BLT when you run a composer update command. This is my recommended approach. Note that this will not automatically update you to version 12, but it would update to version 11.5.ask for a focused version: acquia/blt: ~11.4
In this case, composer would update you any new releases in the 11.4.x version, but would not increment to 11.5.
As I mentioned above, I typically recommend the ^ syntax here unless there are very specific reasons that you might want a specific release.
Note that in the final two cases of this example, composer will only update your dependencies if you run the composer update command!
Getting Started
I would recommend before you get started with Composer that you make sure your local machine has been setup for local development. From there, I would suggest setting up a new composer project using Acquia Lightning:
$ composer create-project acquia/lightning-project MY_PROJECTThis will automatically setup a project for you for local Drupal development (note, it does not include a VM). Once this initial composer script has completed, cd into the new project directory (whatever you named MY_PROJECT) and run:
$ composer installThis will install all of the dependencies stored in the composer.lock file to ensure that your local is entirely up to date and ready to work.
Once this has completed, start playing. Once I had the basics of composer down, I spent a lot of time adding and removing dependencies, patching them, and trying to make sure that I understood how I got into trouble (when I did) and how to get back out of it again (when I did).
Some recommended exercises:
add Drupal modules to your composer project
add other packages to your composer project
remove packages from your composer project
try to add something goofy (e.g. drupal/core: ^7) and see what happens
For extra credit:
try to patch a package using composer
I highly recommend initializing a git repository and committing some files when you perform these activities. Watch what happens in your project as you make changes and run commands!
Project Makeup
Now obviously composer does a LOT for you in terms of setting up and scaffolding a project. You’ll (hopefully) notice once you’ve installed Lightning Project that most of the composer dependencies live in a vendor folder that is gitignored. All the Drupal stuff though, should be automatically in the docroot folder (much of which is also gitignored). This isn’t magic, it’s part of what Composer does for you out of the box! I’m not going to dig super deep into this today. But I did want to call out why many of these folders are gitignored.
The whole point of using a dependency manager is that you don’t actually “need” anything that composer manages to live in your repository. Now, obviously, you will need all of it to host / serve your website. But for development purposes you can literally cut tends of thousands of files from your git repo just by adding composer into your workflow!
This does assume that each of your developers and your continuous integration / deployment processes are enabled to use composer. Obviously if you yank out a bunch of module and Drupal without a way of installing them during deployments, that’s going to be an issue.
If you check out a composer-managed project (like Drupal GovCon) you’ll notice that none of the composer dependencies are committed anywhere in the repo. Just the composer.json and composer.lock files. One of the very first things our continuous integration script does is run composer install.
Command Checklist
composer require
this command adds a new dependency to the project. this command results in both a composer.json and composer.lock change.composer install
this command will install all dependencies. should be run anytime code updates are made / pulled into local environmentcomposer update / composer update --with--all--dependencies
updates all / specific packages. should be run rarely / intentionally. this command results in a composer.lock change.
In terms of development workflow, the composer require command will only be run when new packages are added. Composer install will be run constantly. Composer update should only be run when an update needs to be performed (and I highly recommend only running the composer update --with--all--dependencies version of the command to ensure more specific control over what is updated).
For more reading on the topic, I recently blogged about the 7 composer hacks and commands you should be including in your 2020 development workflow.
In Conclusion
Everything I talked about in this article broadly applies to other dependency managers (e.g. NPM).
Dependency management can be a maddening process to learn, but once you wrap your head around it and integrate it into your workflow it makes your life so much easier. This is something I use daily and have used daily for years. I highly recommend investing the energy into learning it and incorporating it into your development workflow if you haven’t already!
Image by Free-Photos from Pixabay
Related Articles
Dependency trees can be massive, and with the upcoming Drupal 10 release PHP 8.1 could have a significant impact on your project.