#developer workflow

AI coding tools are entering a second phase. The first phase was about whether one model could help one developer. The next phase is about orchestration: how many subtasks can run at once, how clearly those subtasks are defined, and whether the workflow feels like working with a capable teammate instead of a very fast autocomplete engine. GitHub’s new /fleet announcement is a clean example of…

Maybe you have this problem too

You manage or are part of a team that is responsible for a certain functional area of code. Everyone on the team is at different points in there career. Some people have only been there a few years, or maybe even only a few months, but they're hungry and eager to learn. Other team members have been around forever, and due to that longevity, they are go-to resources for the rest of your organization when someone needs help in that functional area. More-senior people get buried under a mountain of review requests, while those less-senior engineers who are eager to help and grow their reputation get table scraps.

This is the situation I walked into with the Developer Workflow team.

This was the first time that Mozilla had organized a majority (4) of build module peers in one group. There are still isolated build peers in other groups still, but we'll get to that in a bit.

With apologies to Ted, he's the elder statesman of the group, having once been the build module owner himself before handing that responsiblity off to Greg (gps), the current module owner. Ted has been around Mozilla for so long that he is a go-to resource for not only build system work but many other projects, e.g. crash analysis, he's been involved with. In his position as module owner, Greg bears the brunt of the current review workload for the build system. He needs to weigh-in on architectural decisions, but also receives a substantial number of drive-by requests simply because he is the module owner.

Chris Manchester and Mike Shal by contrast are relatively new build peers and would frequently end up reviewing patches for each other, but not a lot else. How could we more equitably share the review load between the team without creating more work for those engineers who were already oversubscribed?

Enter the shared bug queue

When I first came up with this idea, I thought that certainly this must have been tried at some point in the history of Mozilla. I was hoping to plug into an existing model in bugzilla, but alas, such a thing did not already exist. It took a few months of back-and-forth with our reisdent Bugmaster at Mozilla, Emma, to get something setup, but by early October, we had a shared queue in place.

How does it work?

We created a fictitious meta-user, [email protected]. Now whenever someone submits a patch to the Core::Build Config module in bugzilla, the suggested reviewer always defaults to that shared user. Everyone on the teams watches that user and pulls reviews from "their" queue.

That's it. No, really.

Well, okay, there's a little bit more process around it than that. One of the dangers of a shared queue is that since no specific person is being nagged for pending reviews, the queue could become a place where patches go to die. As with any defect tracking system, regular triage is critically important.

Is it working?

In short: yes, very much so.

Subjectively, it feels great. We've solved some tricky people problems with a pretty straightforward technical/process solution and that's amazing. From talking to all the build peers, they feel a new collective sense of ownership of the build module and the code passing through it. The more-senior people feel they have more time to concentrate on higher level issues or deeper reviews. The less-senior people are building their reputations, both among the build peers and outside the group to review requesters.

Numerically speaking, the absolute number of review requests for the Core::Build Config module is consistent since the adoption of the shared queue. The distribution of actual reviewers has changed a lot though. Greg and Ted still end up reviewing their share of escalated requests — it's still possible to assign reviews to specific people in this system — but Mike Shal and Chris have increased their review volume substantially. What's even more awesome is that the build peers who are *NOT* in the Developer Workflow team are also fully onboard, regularly pulling reviews off the shared queue. Kudos to Nick Alexander, Nathan Froyd, Ralph Giles, and Mike Hommey for also embracing this new system wholeheartedly.

The need for regular triage has also provided another area of growth for the less-senior build peers. Mike Shal and Chris Manchester have done a great job of keeping that queue empty and forcing the team to triage any backlog each week in our team meeting.

Teh Future

When we were about to set this up in October, I almost pulled the plug.

Over the next six months, Mozilla is planning to switch code review tools from mozreview/splinter to phabricator. Phabricator has more modern built-in tools like Herald that would have made setting up this shared queue a little easier, and that's why I paused...briefly

LOL. A new classic!

Source Safe

You can only ever have one pair of shoes. After a while, your girlfriend moves in and being forced to share a single pair of shoes becomes really annoying.

Subversion

You can have as many pairs of shoes as you want, but switching between them is really painful. You choose a pair each and stick to it until it is worn completely out and a replacement is inevidable.

CM Synergy

You have hundreds of individual shoes in a huge pile. Every morning you spend half an hour trying to find two shoes that match… Or at least go well together.

Team Foundation Server

You have several pairs of shoes in a closed box. Each morning you put you reach in and most of the time you get the right pair. Sometimes you get two that don’t go together, so you try again. After a few attempts you get a pair and you don’t bother to try and find out what went wrong.

Accurev

They look like shoes, but don’t quite feel right. You curl up your toes and adapt a new style of walking. As long as you walk straight ahead on level surfaces everything is fine. After a while you convince yourself that your feet would probably hurt no matter what shoes you wore.

Git

They don’t look or feel like any other pair of shoes you’ve tried before. You put them on and they hurt like hell at first. After a while they grow on you like fungus and soon they become the last pair of shoes you ever want to wear. You recommend them to everyone you meet and become arrogant and condescending towards those who don’t share your enthusiasm.

Mercurial