Code for Canada: Toronto Squad 2018

The MOVE Project launched back in October 2018.  Since then, this Tumblr blog has been our window to the world.  You’ve followed us as we work with transportation engineers at the City to understand the complex, ever-changing field of road safety.  You’ve seen how we gather feedback from users to shape that understanding into a great product.  You’ve seen our releases, our triumphs, our setbacks, our challenges.

That doesn’t change with this move!  Starting today, all new updates will be posted to Medium, and can be viewed at:

https://medium.com/move-project

By moving MOVE to Medium, we join Code for Canada and many other civic tech organizations on a platform designed for thoughtful discussion and open dialogue.

🌹 Rose (what are we celebrating?):

Thanks to excellent efforts from Ruth and Shine, we have a backlog stuffed with enough near-future clarity to get us most of the way to replacing FLOW and CRASH.

This is combined with a renewed product story.  If Kanban / backlog tasks are the “what?” and design / dev planning is the “how?”, the product story is the “why?”  A good product story helps key stakeholders understand, accept, plan for, and champion your vision.

Finally, some great news on the technical front: we’ve been approved to access City sign-in systems from the cloud!  This will help us offer City staff a familiar log in experience while keeping their credentials secure.

🌱 Bud (what are we anticipating?):

With a new product story in hand, and with the 18-month contract extension approved, we’ll resume our regular Project Panel meetings.  These create space for stakeholders to provide feedback and ideas, and it allows us to update them on our progress and plans.

On the design / dev front, we’re looking at Vue Material as a promising UI component library to build out higher-fidelity versions of MOVE.  By integrating a well-tested, widely-used UI framework, we can spend more time and effort on the important problems unique to our project.

📌 Thorn (what are we fixing?):

We’ve rapidly moving towards a stable design / dev process, one that reliably takes us through ideation, prototyping, development, testing, and refining each 2-week sprint.

To get there, though, we still need one key technical piece: a staging environment.  Right now, all releases - even unstable ones - are deployed to a single cloud environment.  When we conduct usability tests, this often means holding off on new releases.  This in turn makes longitudinal studies, which measure behaviour and usage over longer periods of time, all but impossible.  As we get closer to replacing FLOW and CRASH with MOVE, these longitudinal studies become even more important.

With a staging environment, we’ll be able to keep a stable version in front of users while still maintaining a quick release cycle.

Thoughts/Reflections:

🌹 Rose (what are we celebrating?):

We gained a lot of validation for this vision. Big Data Innovation Team lead Aakash Harpalani’s face lit up when he first saw our newest version - that’s the reaction product teams live for!

Shine led multiple feedback sessions around our whiteboard mocks, with plenty of insights, questions, and excited “so when can I use this?” inquiries. Ruth met with stakeholders on the business side, gathering their crucial input to help ground this vision. Evan quickly prototyped the heatmap you see above, so that we could put something tangible and usable in front of users to gather even more feedback.

🌱 Bud (what are we anticipating?):

All this validation and excitement means we have cause to celebrate what we’ve got, but even more cause to anticipate what’s coming. To get there, we’re starting to move towards a more stable sprint cadence: one that takes us from design to prototype to test to release, every two weeks.  Our collaboration around getting early validation of this vision is a prime example of this cadence in action, and there will be more examples to come.

📌 Thorn (what are we fixing?):

Of course, this close to end-of-year it’s sometimes hard to book usability tests and feedback sessions with our users; some of that work will have to continue into next sprint.

In addition, we have ongoing discussions with IT around how this project will eventually be sustained. That’s a very good thing, but in the short term it meant reworking our sprint schedule a bit to accommodate these discussions.

Lead times on design engagement can be longer in big organizations, which makes it especially important to think not just about next sprint but a few sprints into the future when planning work.  That means having a slightly longer backlog - something that we’ve worked on over the last couple of sprints, and that we’ll continue to emphasize going forward.  

Thoughts/Reflections:

We’ll leave off with a quote from one of our users when they saw the whiteboard mocks:

🌹 Rose (what are we celebrating?): In the last sprint, we created our first PDF report using the new MOVE Reporter service.  Since then, we’ve successfully deployed that service, and are now using it to serve up four report types.  More importantly, these reports are backed by a series of automated tests:

fuzz tests to ensure that reports work properly on a variety of input datasets;

edge case tests that assess whether report logic is robust to empty, missing, or otherwise invalid values;

regression tests that match the output of MOVE Reporter against reports produced by legacy applications on the same data.

At this point, we’re nearly ready for our usability test on Oct. 8 (next Tuesday) - we just need to deploy the system one more time with the latest changes and tests.  Since we deployed an initial version early in the sprint cycle, we have confidence that this will be a relatively painless process, rather than a last-minute scramble.

🌱 Bud (what are we anticipating?): Since last sprint, we’ve made some strides in Airflow adoption!  Raphael Dumas led a Lightning Talk series to highlight what team members had learned as they build pipelines in Airflow.

Some examples: Cathy Nangini showed us how to use Airflow’s connection management to securely manage credentials; Natalie Chan demonstrated how to connect Airflow to Slack to quickly notify team members of task failures.

Between these presentations and documentation on our team wiki, this adoption process has been a great example of team-wide peer learning and knowledge transfer.

📌 Thorn (what are we fixing?): this sprint brought a number of unexpected demands on time.

Developer Evan Savage helped Code for Canada with technical interviews for their upcoming cohort.  He’d estimated 5 hours of time for this, but it ended up being closer to 10 hours.

There was also an unexpected bug in the processes that move data from Toronto Police Services to Transportation Services, resulting in missing collision location data.  By bringing together Big Data, IT, and the MOVE team, we found a way to restore this location data from MOVE’s mirrored collision dataset.  This took another 3 hours - ultimately a success from an internal collaboration standpoint, but impossible to account for during our sprint planning!

Finally, one of our report types took 6-8 hours more than budgeted, due to unexpected complexity in the report logic.  This report requires that we identify major vs. minor roads next to an intersection; we hadn’t yet built this feature into the system, and we underestimated how hard it would be to build.

Overall, that’s 15 extra hours, or roughly 20-25% of a two-week sprint!  This highlights the importance of building spare time into schedules during sprint planning.

What makes a tool right?  It’s a great question, though one that’s often underspecified: until you know what the right job is, how can you possibly select the right tool?

The job is often harder to define than you think:

“Make more useful reports”: which reports?  useful how, and for whom?  in which formats?

“Manage our source code”: which code?  from whose perspective - engineers, reviewers, or managers?  are there other non-code assets, models, etc. that need to be stored?

“Move data from one place to another”: which data?  where from, and where to?  once or repeatedly?

In this sprint update, our Rose, Bud, and Thorn all revolve around jobs, tools to do them, and ways to use those tools.

🌹 Rose (what are we celebrating?): Above, a screenshot of the first PDF report generated by MOVE Reporter, a backend service that centralizes and standardizes the generation of data-driven reports in MOVE.  MOVE Reporter’s modular design will make it possible to quickly add support for new report types and download formats.

By getting this first PDF report together, we’ve taken a big step towards our upcoming Oct. 4 release of new reporting features in MOVE.  This puts us in an excellent position to complete those features over the coming sprint!

🌱 Bud (what are we anticipating?): Charles Zhu facilitated a Big Data Innovation Team discussion on GitHub practices around source code organization, code review, and coding standards.  From that, we have a clearer sense of where we can improve:

Code review system + practices

create GitHub pull request templates

create a code style guide, and integrate tools for style checking

write a code review guide

assign owners / reviewers for different repositories

Repository reorganization

decide how to clearly separate “data pipelines” from self-contained “app / tool / dashboard” projects

clarify role for our “data sources” repository, and improve documentation there

define process around exploratory / “sandbox” code: where does this go, and when / how does it get “promoted” to more production-worthy modules?

📌 Thorn (what are we fixing?): Airflow is a tool for managing and automating data pipelines.  We’ve been experimenting with it in the hopes of reducing project overhead around data cleaning, normalization, and migration.

Right now, we’re at a crossroads: do we ramp up Airflow adoption, or do we step back and assess other tools again?  Unlike other teams, such as the City of Boston’s Citywide Analytics Team, we don’t have a lot of in-house data engineering expertise to draw on here.  We’ve had some initial successes with Airflow, but also some issues with its default configuration.  Adoption has been largely ad-hoc, with different team members trying out Airflow on their individual workstations for different tasks.

Last post we had a lot of preamble, so this time we’re leading off with our Rose, Bud, Thorn: 

🌹 Rose (what are we celebrating?): Jim Smith, one of the supervisors here at Transportation Services, is retiring after 35 years with the City!  We met with him - and many others - to understand more about day-to-day tasks, database formats, desired reports, and other considerations around collision data.

Above, the output of a workshop with Big Data (purple) and Vision Zero (yellow).  We asked them which tasks are most frequent, which tasks are hardest to complete, and which tasks they’d like to be able to do.

As a result of all these conversations, we’ve gained a wealth of detail into the user-facing side of collision data usage.  We’re also getting valuable technical context, which will help us make sense of the existing collision data intake process.  Ultimately, this will make planning the work to improve this process easier.

🌱 Bud (what are we anticipating?): Progress on the usability testing front!  With help from a couple of our users in Traffic Ops, we ran through a practice remote usability testing session.  The idea was to make sure that everyone could connect and hear each other, that they could load MOVE without assistance, and that screen sharing was possible.  The practice was a resounding success, and we’re excited to incorporate remote testing into our process moving forward.

Separately, we also created a process for supervisors and managers in Traffic Ops to schedule regular usability tests with their staff.  We now have two tests in two different districts scheduled through this process!  We’re still working on true Citywide adoption, but this process will help us continuously validate MOVE with our users as we build out new features.

📌 Thorn (what are we fixing?): Right now, MOVE is a one-person team!  This means time on communications and process is time not spent on development.  While we believe we made the right call to prioritize communications and process this sprint, longer-term we need to allocate more time to development.

To put that in context: last sprint had 9 working days; 3 were spent on development, or 33%.  For next sprint, the target is 50-60%, reflecting the higher priority of feature work leading into upcoming usability tests.

Welcome back to Toronto Squad!  In our most recent post, we unveiled the method behind the madness: a hypothesis dashboard that feeds into our sprint board to help us focus design and development.

A lot has happened since then: we launched our first alpha, got some great user feedback, presented our fellowship work in Ottawa and Edmonton, and renamed Flashcrow to MOVE.

Apologies to our readers!  We were aiming for updates a bit more frequent than quarterly.

Flashcrow is what’s known as a project codename - in this case, it’s a tongue-in-cheek mix of CRASH and FLOW, the two systems our project aims to improve upon.  To prepare for launch, product manager Andrew Konoff gave it the launch name MOVE, and designer Pallavi Thampi whipped up a fantastic logo.

MOVE echoes the short, single syllable names of CRASH and FLOW while reflecting our goal to meet the needs of active, multi-modal transportation.  No matter how people navigate Toronto, our data should capture their experiences.

Meanwhile, developer Evan Savage is continuing work on MOVE through end of 2019, with further extensions into 2020 in the works.  (If you’re a PM or UX / UI designer excited about civic tech, let us know!  We’ll have more details on these positions shortly.)

Since the current 5-month extension is being referred to as the bridge contract, we called this 1-week mini-sprint the Bridge Kickoff Week.

OK!  Enough preamble: it’s time for our first Rose, Bud, Thorn - a sort of open retrospective.

🌹 Rose (what are we celebrating?): we finally have a working Continuous Delivery process, complete with notifications!  This means we can quickly and reliably get new features, bugfixes, or other improvements in front of our users.

🌱 Bud (what are we anticipating?): speaking of key stakeholder conversations, we’ve started talking with stakeholders both internal (e.g. Vision Zero, IT) and external (e.g. Toronto Police) about our process for collision data intake.  By co-designing with the people who rely on that collision data, we can make this process more useful as we stabilize it.

When a City of Toronto resident requests a new traffic signal, City engineers use collision and traffic data to assess the benefits, drawbacks, costs, and logistics of installing that signal.  This process is called a signal warrant.  (There are also all-way stop sign warrants, pedestrian signal warrants, etc.)

Sometimes, the traffic data they need hasn’t been gathered yet, and the engineers need to request it!  In our early user tests, our interface framed this data request as part of the warrant process: “Request Data for a Warrant”.

This framing made sense to us, but it confused our users: they understood the warrant not as a process, but as a report created once you have all the data.  They also sometimes view and request data outside the warrant process, and couldn’t see how this interface would accommodate that.

Some key takeaways here:

test early, test often: by putting lo-fi prototypes in front of users quickly, we gained an important product insight in weeks instead of months.

(Thanks, xkcd!)

First off: that kind of coding does, in fact, exist.  It's often called Cowboy Coding, and it primarily sees real-world use in three scenarios:

junior developers not used to working on production-quality software in teams;

organizations that develop software incidentally, rather than as a core product (e.g. research labs, business Excel spaghetti, etc.)

super-pre-alpha experimentation / prototyping.

Junior developers are ideally mentored out of scenario #1, at which point they can sense when #3 is applicable (and when it’s not).  People in scenario #2 often write code as though they’re in scenario #3, which leads to pain when that code outlives its originally intended use case.

So, here’s the dirty secret: #3 is a tiny slice of the project lifecycle.  It’s almost never applicable, especially in a cross-functional agile team.

In agile, your first product code often comes after a lot of user research, design, and low-fidelity prototyping.  As an agile developer, part of your job is to put the right processes and tools in place before that point.  Otherwise, you’ll struggle to communicate progress to teammates, onboard new developers, maintain feature velocity, and collect useful operational / business metrics.

Without process and tooling, your real product is team burnout, angry users, and maintenance nightmares.