Menu

An update on projects created for Drupal

October 7th, 2017

About six months ago we made a significant change to the way that modules, themes, and distributions are created on Drupal.org.

In the past, contributors had to first create a sandbox project, and then request manual review of their project in the Project Applications issue queue. The benefit of this community-driven moderation process was that modules were vetted for code quality and security issues by a group of volunteers. Project maintainers who completed this process also received the benefit of security advisory coverage from the Security Team for stable releases of their projects.

Unfortunately, the rate of project applications outpaced what volunteers could keep up with, and many worthy projects were never promoted to full project status, or moved off of Drupal.org to be hosted elsewhere.

To ameliorate this issue, we changed the process so that any confirmed user on Drupal.org may now make full projects.

To mitigate the risks of low code quality or security vulnerabilities we added new signals to project pages: including highlighting which release is recommended by the maintainer, displaying recent test results, and indicating whether the project receives security coverage both on the project page and in the composer 'extra' attribute. We're continuing to work on identifying additional signals of project quality that we can include, as well as surfacing some of this information in Drupal core. We also converted the project applications issue queue into a 'request security advisory coverage' issue queue.

What we hoped to see

We knew this would be a significant change for the project and the community. While many community members were excited to see the gates to contribution opened, others were concerned about security issues and Drupal's reputation for code quality.

Our prediction was that the lower barrier to contribution would result in an increase in full projects created on Drupal.org. This would indicate that new contributors or third party technology providers were finding it easier to integrate with Drupal and contribute those integrations back for use by others.

At the same time, we also expected to see an increase in the number of full projects that do not receive coverage from the security team. The question was whether this increase would be within an acceptable range, or represent a flood of low quality or insecure modules.

The results

The table below provides statistics about the full projects created on Drupal.org in the 5 months before March 17th, 2017 - when we opened the creation of full projects to all confirmed users.

Full projects created from 2016-10-16 to 2017-03-17…

#

% of projects created in this period

… without stable release

431

55.76%

… with stable releases

342

44.24%

… with usage >= 50 sites

237

30.66%

… with usage >= 50 sites and without stable release

68

8.80%

… with usage >= 50 sites and with stable release

169

21.86%

… with an open security coverage application*

18

2.33%

Sub-total with security coverage

342

44.24%

Sub-total without security coverage

431

55.76%

Sub-total with security coverage and >=50 usage

169

21.86%

Sub-total without security coverage and >= 50 usage

68

8.80%

Total

773

* note: full projects that did not have stable releases were not automatically opted in to security coverage when we opened the full project creation gates.

… and this table provides statistics about the projects created in the 5 months after we opened the creation of full projects to all confirmed users:

Full projects created from 2017-03-17 to 2017-08-16…

#

Diff

% of projects created

Diff %

… without stable release

851

+420

69.53%

+97%

… with stable releases

373

+31

30.47%

+9%

… with usage >= 50 sites

156

-81

12.75%

-34%

… with usage >= 50 sites and without stable release

64

-4

5.23%

-6%

… with usage >= 50 sites and with stable release

92

-77

7.52%

+46%

… with an open security coverage application

62

+44

5.07%

+344%

Sub-total with security coverage

182

-160

14.87%

-53%

Sub-total without security coverage

1,042

+611

85.13%

+242%

Sub-total with security coverage and >=50 usage

54

-115

4.41%

-32%

Sub-total without security coverage and >= 50 usage

102

+34

8.33%

+150%

Total

1,224

+451

+58%

As you can see, we have an almost 58% increase in the rate of full projects created on Drupal.org. We can also see a significant proportional increase in two key areas: projects with greater than 50 site usage and no security coverage(up 150% compared to the previous period), and projects that have applied for security coverage(up 344% compared to the previous period). Note: this increase in applications is for projects *created in these date ranges* not necessarily applications created overall.

This tells us that reducing friction in applying for security coverage, and encouraging project maintainers to do so should be a top priority.

Finally, this last table gives statistics about all of the projects currently on Drupal.org, regardless of creation date:

Full projects (7.x and 8.x)

#

% of Total

Rate of change after 2017-03-17

… with the ability to opt into security coverage

8,718

36.15%

-1.33%

… with security coverage and stable releases

8,377

34.74%

-1.49%

… without security coverage

15,396

63.85%

+1.33%

… without security coverage and with stable releases

464

1.92%

+1.04%

… with security coverage and >=50 usage
 

6,475

66.91 / 26.85%

-0.54%

… with security coverage and stable releases and >=50 usage

6,308

65.19 /26.16%

-0.65%

… without security coverage and >=50 usage

3,202

33.09 /13.28%

+0.54%

… without security coverage and with stable releases and >=50 usage

130

1.34 /0.54%

+0.51%

Sub-total with >=50 usage

9,677

40.13%

-1.72%

Total

24,114

From the overall data we see approximately what we might expect. The increase in growth of full projects on Drupal.org has lead to a modest increase in projects without security coverage.

Before the project application change, all full projects with stable releases received security advisory coverage. After this change, only those projects that apply for the ability to opt in(and then do so) receive coverage.

What has this meant for security coverage of projects hosted on Drupal.org?

1.92% of all full 7.x and 8.x projects have stable releases, but do not receive security advisory coverage. It is likely no accident that this translates into 464 projects, which is nearly equivalent to the number of projects additional projects added compared to our old growth rate.

Of those only 130 of those projects report more than 50 sites usage(or .54% of all 7.x and 8x full projects).

Next steps

From this analysis we can conclude the following:

  1. The opening of the project application gates has dramatically increased the number of projects contributed to Drupal.org.

  2. It has also increased the number of projects without security coverage, and the number of applications for the ability to opt in to coverage among new projects.

In consultation with the Security Working Group, we recommend the following:

  • For now, leave the project creation projects as it stands today - open to contribution from any confirmed user on Drupal.org.

    • Less than 2% of all Drupal projects with stable releases currently lack security coverage. The rate at which this is increasing is significant (and in the wrong direction) but not rapid enough to merit changing the project application policy immediately.

  • Solve the problem of too many security advisory coverage applications. The security advisory application queue has the same problem that the old project applications queue had - not enough volunteers to manually vet all of the applications - and therefore a significant backlog of project maintainers waiting on the ability to opt into coverage.

    • Recommendation: Implement an automated best practices quiz that maintainers can take in order to be granted the ability to opt into security advisory coverage. If this process is as successful as we hope, we may want to consider making this a gate on stable releases for full projects as well.

We look forward to working with the Security Working Group to implement this recommendation and continue to improve the contribution experience on Drupal.org, while preserving code quality and security.