Miguel Dias

Outcome Mapping Case Study

Reflections on a workshop I facilitated, for other workshop facilitators and flow enthusiasts.

Table of Contents

Key Learnings

Introduction

This case study is about an Outcome Mapping workshop I facilitated, inspired by the book Flow Engineering. It includes comments generously provided by Steve Pereira.

The workshop team includes representatives from multiple teams. These teams are responsible for infrastructure and operations.

Workshop participants: three team leads, two engineers, and one product architect.

It’s a reflection on a challenging workshop, to think on what can be improved for future workshops, to help myself and other facilitators.

This workshop is the first stage of a five-stage process described in the book Flow Engineering:

  1. Outcome Map
  2. Current State Value Stream Mapping
  3. Dependency Map
  4. Future State Value Stream Mapping
  5. Flow Roadmap

image1

The goal of this multi-stage process is to identify how we deliver value to the customer, clarify the most likely constraints causing delays and quality problems, and have the teams come up with minimum viable countermeasures to improve flow. We then follow up and measure to ensure we are improving.

image2

Outcome Mapping is the first stage of Flow Engineering. The primary goal for this session was to identify the area for improvement so we could focus our energy on impactful changes. The session was planned for 1 hour and 30 minutes.

If you want a primer about Flow Engineering, go here.

Outcome Discovery Stage

Writing the Cards

Activity: The team writes their thoughts on the sticky notes: context, goals, pains, questions, ideas.

image3

We jumped straight into writing sticky notes into the columns: context, goals, pains, questions, ideas. I asked the team to write any thoughts that come to mind in the different categories, such as pain points and opportunities for improvement.

I gave simple examples to explain what was meant in each column:

Context:

Pain:

Goal

Question

Idea

Giving adequate guidance for each column will improve the discussion and better steer the session

Including a discussion about current team goals and milestones before this activity could help align the day-to-day operational issues raised by the team with the broader organizational goals set by senior leadership.

> Steve: “This is what I press for in [the column] Context, which is also challenging for folks in the trenches to identify. Their context is often what’s on their mind today, and they have to be guided out to a broader scope and bigger picture.”

By connecting the team’s operational concerns with the company’s strategic objectives, we can prioritize the issues that will most effectively move the team and the organization closer to achieving their goals. This could help in discovering the best ways to address these issues and motivate the team by showing how their work directly contributes to the success of the team and the company.

I could feel some tension in the room at the start that I didn’t feel when I talked with each participant individually before the workshop. Perhaps I should spend more time breaking the ice in the beginning of the call, by giving an overview of the session, clarifying our goal, explaining the different stages, discussing the goals at each stage, talking about timeboxing and providing guidance on creating high-quality SMART goals could also help (more on this later). Coming in as an external facilitator, there might be a lot I don’t know about the team context, relationships, conflicts, frustrations. Some things that could be happening in a team i don’t have context about:

Splitting the cards into context, goals, pains, questions, and ideas was really effective in helping people express their thoughts. I did give them 1-2 extra minutes to finish writing their cards, but with six people, this might generate too many cards. At the time, I didn’t want to leave out the most important topics they might only write at the end.

I felt like I was flying blind because I didn’t know much about the context behind each card/thought. This workshop works like a funnel: we start with a lot of cards, and only those that the participants write. By talking about the notes and voting on the categories, the participants can themselves guide the workshop, and “funnel” our attention to progressively more specific topics, as we progress in the activities to the important topics, but in the end, with my limited understanding, I don’t have a lot of influence on what we end up focussing on, or on what we end up with as an outcome at the end. Of course, the goal of this article is to explore ways where I can be as effective as possible, provide the best environment as a facilitator for a high quality outcome to emerge, and also refine my expectations of myself and the group.

We could spend a lot more time going over the cards. People had a lot to say, mainly two of the six participants. As mentioned in the book, introducing an engagement rule at the start for keeping participation to 1-2 sentences could have helped us cover all the cards. It was challenging to keep the conversation moving because participants sometimes had a lot to say about a topic. While it’s valuable for them to discuss the cards, it came at the cost of time and conciseness. Encouraging participants to involve quieter members from the start would have been beneficial, as I had to do that throughout the workshop, and it would be nice to share that responsibility to reduce my own cognitive load so I’m able to be more serendipitous.

Splitting the Cards into Categories

Activity: Identify common topics and themes in the sticky notes, and group them together.

image4

Splitting into categories took longer than expected. I felt I had to jumpstart the effort myself before they engaged with the exercise. Providing an activity overview at the beginning, discussing timeboxing, defining clear roles for participants and the facilitator, and reiterating the goals of each timebox before the activity would, I think, help keep them engaged and reduce confusion.

Voting the Categories

Activity: Vote on the group/theme of cards that they think is most important to work on right now.

image5

I need to explain the micro logistics of voting better (placing the balls on the cards) to avoid confusion.

The team voted on a category that was not the one they complained about the most. I was so focused on keeping us moving through the activities and timekeeping that I didn’t stop to ask why. Here I think it would be beneficial to have some extra time to discuss the voting results, but keeping participation concise so we can hear from everyone. Keeping the same timeboxes but taking more time to explain the goal of each timebox clearly. For example, “At the end of this timebox, everyone should have expressed their understanding of the voting results, or expressed their view of the importance of the outcome. We should hear from everyone by all means necessary.” In this way, I’m sharing the responsibility of making sure we hear from everybody, and that we should keep to the timebox, so we can finish the exercise.

Outcome Definition

Activity: Based on the sticky notes from the most voted category, the team comes up with a goal representing a valuable target for the customer, business and workers. It’s helpful to aim for a SMART goal.

image6

It was not easy for the team to come up with an outcome. I can improve how I ask for outcomes, guiding the team to get to something close to a SMART outcome. I can try to guide them there slowly, with targeted questions:

The outcome that did come up from the most voted category seemed to be

The engagement seemed to drop at this point, maybe because the team didn’t think the outcome that was mentioned would move the needle, or there was a conflict in what they each thought was important.

Establishing engagement rules at the start, as mentioned in Flow Engineering (e.g., cameras on from the start, encourage everyone to ask for participation from quieter members), could help.

Its not trivial to transition from a group of cards, to a goal. In this case, the goal related to a problem that was included in the category, but there was no consensus on the importance of the goal. It was not enough to simply ask the team what could be alternative goals with high impact. I need to experiment with this transition and ways to help the team surface a significant goal. Talking about pre-existing goals from the start helps with this, as does helping the team formulate something similar to a SMART goal, but I also need to address the part about picking which goal to focus on among the many possible goals in the category. I think this is connected to how comfortable people are to speak up for what they think is important, so that’s also something I can influence. I wonder if I’m missing something else about this transition.

Talk about Benefits, Obstacles, and Next Steps

Activity: Armed with a target outcome, we spend time solidifying why we should address the goal, what could get in the way and what we do to get started.

image7

We didn’t spend much time refining the outcome and proceeded to discuss benefits, obstacles, and next steps. I hoped this activity would also help to refine the outcome and highlight the benefits. As we went through these, it wasn’t obvious that addressing this goal would be beneficial. More time spent picking the right category/goal and refining it would be beneficial before this stage, but in this specific workshop, at this point, we were disconnected from a relevant outcome for everybody. The outcome that surfaced was about addressing micro-scale issues in the release process, but without more clarity around the overall process, it’s difficult to understand the impact of these micro issues. We risk focusing on issues that cost resources to fix but don’t provide substantial ROI.

I believe it would help significantly to start from an outcome coming from leadership, that the business believes is the most impactful, and structure the workshop around that.

> Steve: “This seems to be the most sensible default. Rather than relying on the folks in the trenches to know the strategy and objectives, we should treat that as a bonus and replace a target from leadership if there can be a clearer, nearer term (3-6 months out) outcome available to the team.

We hope there’s more clarity and alignment in the teams than there typically is within a siloed or functional organization, but we should likely assume there isn’t.”

More Post Workshop Reflections

Even though we only spent 1h30min in the workshop, so the time cost for everyone was relatively low, I felt that it took a toll on the engagement of the participants, which influences the will to participate in more sessions.

Following up with people and learning more about their process, I realized that the area we could target for improvement and get the most benefit from might be the release process, as indicated by the voting. However, the thoughts shared in the cards were about micro-scale issues. If explored, these would likely reveal larger problems, related to system architecture and therefore team topologies. In a subsequent interview, we discussed that the team has multiple deployment processes they are responsible for, and one of them takes up most of their time. This seems like a good area to map. The teams had many ideas and issues but didn’t agree on the value of the goal, and there wasn’t clarity or consensus on the potential impact of the different issues. This uncertainty is not ideal, and the goal of the workshop is to get to the opposite: clarity.

I believe I should have started by talking with senior leadership to identify the outcome, then have the discussions around that target outcome from the start of the workshop.

>Steve: “This echoes what I’ve seen. Folks in the weeds talk about the weeds, and they’re fixated on the fires they can see from that vantage point. The problem is they’re very diverse problems that can be hard to connect back to strategy and objectives. Getting the leadership context and plugging it in from the start can help avoid that and steer away from rabbit holes. I think we may be missing a minimal assessment to reveal those hazards and provide guidance on how to navigate the first stages under various conditions.”

Next time it will help to talk with senior leadership to identify the outcome, by asking them questions like:

Live Q&A Sessions with Andrew Davis, Helen Beal and Steve Pereira

I was lucky enough to participate in two live Q&A sessions with Andrew Davis, Helen Beal and Steve Pereira around the time after I facilitated the workshop, so I had the chance to ask some questions, have them answered by world class experts and take some notes, which I leave below.

Session 1: LinkedIn Live with Andrew and Steve

Linkedin live

Here are questions that i asked

Main Points from the Live Session:

Follow-up Questions and Answers:

Resources to improve workshop facilitation

Session 2: LinkedIn Live with Helen and Steve

Linkedin live

Here are questions that i asked

Main points taken from this live with helen and steve

Conclusion

Here is a summary of the main improvement opportunities I identified for my workshop facilitation and planning, and possible countermeasures for those.

Issue identifiedProposed countermeasure
Lack of time to go over all of the cardsStart the session by giving an overview of the different activities. Explain timeboxing and the importance of fulfilling the task within the timebox. Encourage participants to ask for the opinion of quieter members. Provide rules of engagement.
Some participants taking up most of the conversation time
Lack of engagement with the workshop
Provide guidance on coming up with a SMART outcome Talk to senior leadership, discover their outcome, and use that as a starting point for the session.
Difficulty coming up with an outcome
Lack of connection with the outcome
Unclear, potentially low impacting outcome

> Steve: “There may be some missing guidance here as well around how to consider and scope an outcome. I do from time to time work with a team who have never really used the word outcome and couldn’t distinguish it from an output. There is something of an analogy needed about the three little pigs or something to set the tone around longer term and greater scope, in the context of the larger mission rather than random fires and grievances.”

Start from an outcome from leadership and connect it to the team context

First, discover senior leadership’s goal, then shape the workshop around it. As mentioned in the section “Engaging with a Mandated Outcome” from the Flow Engineering book, “the first stage of the outcome mapping is to critically assess the intended outcome and to surface fundamental doubts and questions from the beginning.” “Doubts and questions that are not articulated cannot be dealt with consciously but may manifest as resistance to change”.

Why start from an outcome?

How to guide teams to come up with an outcome?

I believe I will grow in this with practice. Here are some ideas to start from:

Rules of engagement

Getting participants to agree with these is important, and they address issues noted in this case study. For example “Coming to the session means that you agree to the following”

These are in line with the rules of engagement in the book image8

Acknowledgements

Thank you to Andrew Davis and Steve Pereira and for writing Flow Engineering and for answering my questions in their live sessions. Also, thank you to Helen Beal for hosting one of the lives. Additionally, thank you to Mike Osterling, Karen Martin for further developing my understanding of Value Stream Mapping, and to Patrick Debois, Gene Kim, Jez Humble, John Willis, the authors of The DevOps Handbook and Paula Thrasher for introducing me to Value Stream Mapping.

References

  1. Pereira, S., & Davis, A. (2024). Flow Engineering: From Value Stream Mapping to Effective Action. IT Revolution Press. ISBN: 9781950508457. Available from: https://itrevolution.com/articles/flow-engineering/
  2. Thrasher, P. (2020). Interactive Virtual Value Stream Mapping - Visualizing Flow in a Virtual World. Presented at the DevOps Enterprise Summit. Available from: https://videos.itrevolution.com/watch/466912411/
  3. Squirrel, D., & Fredrick, J. (2020). Agile Conversations: Transform Your Conversations, Transform Your Culture. IT Revolution. Available from: https://itrevolution.com/product/agile-conversations/
  4. Lipmanowicz, H., & McCandless, K. (2014). Liberating Structures: Simple Rules to Unleash A Culture of Innovation. Available from: https://www.liberatingstructures.com/
  5. Gifford, B., & Aherne, M. (2020). Workshop Survival Guide: How to Design and Facilitate Collaboration. Available from: https://www.workshopsurvival.com/
  6. Kim, G., Humble, J., Debois, P., & Willis, J. (2016). The DevOps Handbook: How to Create World-Class Agility, Reliability, & Security in Technology Organizations. IT Revolution Press. Available from: https://itrevolution.com/product/the-devops-handbook/
  7. Martin, K., & Osterling, M. (2013). Value Stream Mapping: How to Visualize Work and Align Leadership for Organizational Transformation. McGraw-Hill Education. ISBN: 9780071828918. Available from: https://www.mheducation.com/highered/product/value-stream-mapping-martin-osterling/M9780071828918.html

Workshop Participation Survey

An anonymous feedback survey was provided after the workshop. Here’s how I structured it:

Outcome Mapping with the DevOps Team

Thank you very much for taking the time to participate in the workshop. My goal is to help teams remove annoying things from their work and achieve their goals. Your direct and honest feedback is key so I can improve.

Overall Experience

On a scale from 1 to 5, how clear was the facilitator in communicating the goals and process of the workshop?

Facilitation

Were the objectives of the discussions clear to you during the workshop?

What did the facilitator do well, and what could be improved?

Engagement

Did you feel engaged throughout the workshop? Why or why not?

What could have been done to make this workshop more engaging/relevant for you?

Outcome

What topics do you feel deserve more attention going forward?

Improvement and Suggestions

What is one thing I can do differently to improve the next workshop?

Future Steps

What would you like to see as the next steps following this workshop?

Additional Comments

Is there anything else you would like to share about your experience in this workshop?