June 20th, 2017
Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.
After returning from DrupalCon Baltimore at the end of April, we spent May regrouping and focusing on spring cleaning tasks. It’s important for any technical team to spend time on stability and maintenance, and we used May to find improvements in these areas and look for some other efficiencies.
🎉 UTF8MB4 Support
Support for the UTF8MB4 character set has been a long outstanding issue for Drupal.org and the sub-sites. This expanded character set supports supplementary characters outside of the basic unicode multilingual character plane, including symbols and emoji.
Previously the use of any of these characters on Drupal.org would result in an error. This extended support has been rolled out to Drupal.org and all of the sub-sites except Groups, our legacy Drupal 6 site on LTS.
Protecting Localize.Drupal.org from Spam
After a spike in spam form submissions was reported (thanks Gábor!) we enabled form protection on Localize.drupal.org. Hopefully this will keep our many translation volunteers focused on the hard work of localizing Drupal, instead of on spam fighting. The techniques that spammers use to bypass protections continue to escalate, so we’ll be continuing to evaluate new ways to fight spam as time goes on.
Stability and Maintenance
We spent a portion of our time in May focused on some basic infrastructure issues. One of the Drupal.org production webnodes experienced a filesystem and networking issue and had to be removed from the rotation. We performed some forensics to identify the cause of the issue, and then rebuilt the virtual machine and put it back into rotation.
We also spent some time updating the remote access configuration with our data center, to make remote troubleshooting easier and more efficient for our internal team.
Finally, we performed an audit and inventory of our owned hardware. This helped us to identify underutilized resources that we could re-purpose, and will help us more quickly on-board our new managed infrastructure services partner at the conclusion of our RFP process.
The deadline for responses to our Managed Infrastructure Services RFP was Monday May 8th. Once we’d received proposals from all participating vendors, we began our process to review those proposals internally and schedule interviews with the vendors. As we move into June this RFP process is wrapping up, and we will be announcing the results of the RFP soon.
One of the primary features of DrupalCI is that it allows developers to test against a variety of environments. To make sure that we’re more easily able to keep up with the latest PHP patch releases (e.g: 7.0.x/7.1.x/5.6.x), the PHP environment containers are now rebuilt nightly.
Coding standards test results were added in April, and to make it easier for developers to see where the code standards issues appear within the code base, we’re now linking the standards results to CGIT.
More efficient test result saving
Since we began parsing DrupalCI test results onto Drupal.org we pretty rapidly reached more than 100,000,000 database rows of test results, taking up more than 100G of database space. To make offering this service more sustainable, we’ve implemented changes to how we store test result data. Instead of storing complete results for each test, we now only store the diff between the current test and the last test. This has resulted in a dramatic reduction in the amount of space consumed.
Re-purposing owned hardware for bots
DrupalCI is also the most expensive single service that the Drupal Association provides to the community. In addition to the labor costs involved in building and maintaining the system, the amazon spot instance costs average between $2000-$4000 each month. After spending some time doing an owned hardware inventory audit, we’ve realized that we can repurpose some of our existing hardware as VM hosts for additional testbots. These testbots will not be as fast as the AWS instances, so we’ll be reserving them for use with the nightly test builds, however we hope that even this change will represent a significant savings. Work on this continues into June.
As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:
- Amazee Labs – *NEW* Signature Supporting Partner
- Aten Design Group – Renewing Premium Supporting Partner
- Phase2 – Renewing Premium Supporting Partner
- Lullabot – Renewing Premium Supporting Partner
- Studio Present – *NEW* Classic Supporting Partner
- Chapter Three – Renewing Classic Supporting Partner
- Access – Renewing Classic Supporting Partner
- Softescu SRL – Renewing Classic Supporting Partner
- Linalis – Renewing Classic Supporting Partner
- Druid – Renewing Classic Supporting Partner
- Promet Source – Renewing Classic Supporting Partner
- Adapt A/S – Renewing Classic Supporting Partner
- Booz Allen Hamilton – *NEW* Classic Supporting Partner
If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.