Drupal module selection in the enterprise: lists and processes

We are driving ourselves crazy, folks. Choosing modules is really hard. And it's only getting harder on enterprise Drupal sites (and enterprise just means big teams and with big sites with big requirements).

A recent conversation on twitter started by Drupal rock star Katherine Bailey shows how module selection on a big site can drive you crazy:

So, today I'm going to lay out some ideas I've found for reducing the madness: choosing good modules both as an individual and as a member of an enterprise Drupal site. Of course the enterprise practices build on the set of guidelines for an individual site builder. I'd love to get feedback on other techniques people have used for module selection in big team, big site, enterprise environments.

Module selection best practices

1. First, identify your need and try to be as generic as possible. If you say "Building an event system" you may skew your results toward the Event module when what you really need is some combination of Date, Calendar, Views, and Signup.
2. Next, search around for modules that match what you need and look for recipes and tips from blogs or groups.drupal.org. This is often where you will find great advice on which is the best module to choose. Try the similar module review group which is an amazing resource.
3. Test out the modules. Build a test site that's a copy of your live site, read the README.txt and any documentation you can find, and then install the modules. Once you finish exploring one module, rebuild that test site and try again with any other potential modules.
4. Look for statistics about the health of the module. Things like the popularity of the module, the number of maintainers and the recency that those people have committed code. Look for issues in the issue queue that have been fixed recently. More issues isn't necessarily bad.
5. Check to see if the module matches Drupal best practices. Is it forward-looking and based on cck? Or does it rely on its own data storage. Does it pass tests from the Coder module?

If you want to see more discussion on this, particularly as it relates to the redesign, check out Project metrics for drupal.org redesign.

Enterprise module selection best practices

You've done all that, right. Now, what if you work on a major site or a monster huge site. Or what if you work in an organization that has hundreds of Drupal sites. In these cases you need to go a few steps further than those listed above to ensure the sanity of your team and company.

1. Create a Drupal working group with a mailing list and some wikis. You probably already have this, but it's important that you've got an organization-wide forum to share this kind of information.
2. You need secure modules with good performance. Not all modules are written with security and performance in mind. So, now that you've got a wiki, one great step is to have your development team and security department create a list of modules approved by the organization for stability/security. The University of Pennsylvania maintains one such list based on research their team (led by Justin Klein Keane has done over the last few years.
3. Create a process for adding new modules onto that page. It should look more or less like the steps I mention in the first part of this article and end with "send an e-mail proposing the right module to the developer mailing list for discussion with the team." Ideally you have a dedicated security resource in-house who can be called on to help review the module and perhaps a senior developer or sysadmin who can look for performance problems.
4. Some additional measurement criteria that are more important in large organizations are whether the module has Simpletests, whether it is known to cause problems with Pressflow, and whether it is flexible in ways that are likely to be useful for your organization. If you are building one site then this isn't as important as it is when the module has to satisfy a team of 30 site builders.

This bloated process is likely to upset some of the site builders in your organization. If they are used to just picking their own modules and installing them then this just slows them down.

Hopefully they'll see the benefits of this process:

• Increased ability to move site builders between projects. Now that people are mostly using the same modules it's easier for them to get up to speed on a new project.
• Your sites are more likely to be high-performance, scalable, and secure.
• You reduce the work for external teams (like the cyber security department, or sysadmin teams) because you aren't asking them to review new modules all the time for security/performance issues.
• You can share tips and tricks with other site-builders on your internal mailing list reducing the time it takes to build new sites.

Fighting/Embracing the Not Invented Here Syndrome

One major problem that "enterprise" organizations can get into is the "not invented here" syndrome known as NIH. Often people will refuse to use community modules because they couldn't possibly be good enough to use on our awesome site. Sadly, that is often the case. But I think a more accurate question than "to use, or not to use" a module is "to use, or to enhance until it's ready to use." It's easy to discard modules because you don't have to answer to another maintainer. On the other hand, working with the community process can be much more efficient in the long run since you get maintenance and new features for free. One great example of an enterprise working with the community is this patch for Voting API indexes from David Strauss and Moshe Weitzman as they worked on the Economist.com project. That same module can now be used on many other large-scale sites.

This article is based on best practices I've seen in practice at Economist.com, from the UPENN page, and some government clients of ours that are rolling out dozens of sites. That said, I'm sure there are other ideas people have to make the process even better and I hope we can share them.