Validations and Feedback

‘How am I doing so far?’

What a loaded question… When we ask a question like this, are we seeking feedback, or are we seeking validation? Perhaps this is related to the fixed versus growth mindset.

With a fixed mindset, I would be seeking support to validate that I have been doing a great job and that I am considered a superstar.
With a growth mindset, I am truly open to feedback and want to understand how I could improve.

Of course, this then leads us to metrics and the question about how they could be used for feedback and how they might be used for validation.

Rainfall as a metric means that each day we check our rain gauge in the garden and note how many millimetres of rain have fallen in the last 24 hours. This is not feedback – I am fairly certain that the weather systems do not use measurements from our rain gauges to find ways to improve the weather. Instead it is a monitoring metric and we can use it to validate that the mud was slippery outside because we had a lot of rain.

Budgets and tracking the spend against budget looks like a great feedback metric – however, by itself, it is purely for validation. How many of us have been rightly happy that a piece of work we just completed met budget expectations? (Or rightly unhappy if we did not meet the budget expectations?) Remember those feelings for a moment – pride, relief, happiness – or regret, sadness, blame – these are not very helpful if we want to improve how we work and can distract us from seeing other opportunities if we are not careful.

Feedback is when we ask why and how. Back to the budget example, if we ask why we did not meet budget, without blame, we can find out a lot of useful information and opportunities to improve for next time. One of my favourite lean tools is the ‘5 whys‘ – it is fascinating what can be uncovered if we keep asking why (with polite respect, of course). In the case of budgets – it is just as useful to ask why we met as why we did not meet budget – learning what worked well is very useful for next time also.

Validation versus feedback might be one of the root issues with KPI’s – when we are set a target number, it is very tempting to do all that we can to meet that number and then breathe a sigh of relief when we finally meet or exceed it. How many of us have had truly useful conversations about how we did our work, what we learnt and how we can continue to improve in the era of KPI’s? Validation is a very tempting focus and can feel a lot like feedback – but it is hollow praise.

Metrics validation and feedbackIn summary,

  • Validation is when we use metrics to show that we did a good job
  • Feedback is when we ask why or how we got a result

Be careful which outcome we are seeking when selecting and discussing metrics.

Game play – a Management Insight

The whole idea of modern management can be distilled down to a single basic goal to maximise profit, the methods in which a company or business pursues this most basic goal can vary substantially, focusing upon different areas such as efficiency, staff satisfaction, R&D (Research and Development) and the list goes on.

For the purpose of this discussion the method chosen to achieve the basic goal of profit, is irrelevant.

I want to focus on a method of decision making that we are all familiar with, yet underutilise in our work lives. The process is that of game play and in particular the games of Yahtzee and Poker. The simple fact is that in our management we forget that there are not only outcomes but multiple outcomes from any given situation. These actual outcomes may or may not be desirable but they are equally valid regardless.

The reason I’ve chosen these two games to highlight my point is that although both games are games of chance, they are also games of skill, relying upon probable outcomes of particular desired patterns.

The game of Yahtzee is based upon poker with dice, the general game play is a simplified and modified poker variant. Yahtzee does not allow for the ability to bluff as in Poker which makes it less human but more instructive for my example.

YahtzeeAs you can see from the Yahtzee score card to maximise your score every box (category) must be filled with the highest possible score. This is where the management part comes in because when you get say three or four of a kind you need to allocate that roll of the dice, for the highest yield. You have to decide if the three 6’s you just rolled should be scored as either a three of a kind or three 6’s. The fact that three 6’s is the highest three of a kind possible is going to bias your decision, in this case, so you place them in the three of a kind category but if they were 4’s instead. Three 4’s is a good score for either category, and it depends where you are in the course of a game. You begin to see the point?

There are definite probabilities for each potential outcome and obviously the more difficult, lower probability, ones are the highest rewarding. However, if we focus on only getting the highest possible score for each scoring outcome we soon find ourselves in a losing situation. Each roll of the dice must be recorded and accounted for so aiming solely for Yahtzee (5 of a kind) will result in poor scores because that blind focus will mean other options will be ignored.

So what has this got to do with management, well projects can be considered outcomes just like a hand of cards or a roll of the dice in Yahtzee.

When we ask for a piece of work we expect to get it, yet in the real world we rarely get what we expect and in fact may not get it on time or at all. The delivery may be postponed because a part of the work has not been completed, requirements change over time. This can happen quite often and is one of the reasons we do risk analyses. Yet if we remove the original expectations, we may realise we have created a very worthwhile component which would not be delivered because it is not the expected/requested whole but could fit into and/or compliment another project. Just like in Yahtzee holding onto the original expectations may lead to a reduction in movement towards our goals.

When we manage we tend to focus upon expected outcomes and unexpected or below expected outcomes are disregarded or thrown out.

Both Poker and Yahtzee also share the fact that the initial hand or roll can then be cherry picked and tailored to achieve the best outcome available with what was given. Then the parts not suitable for the newly scoped goal are discarded and re-drawn or cast. This happens three times in Yahtzee and one, two or three times in Poker depending on how many draws are allowed.

Adaptive Management FlowThe simple fact is that maybe the three 1’s are better scored in the one’s category rather than the three of a kind box. The old make lemonade when life gives you lemons approach. We rarely get what we expect, so if we are open to see the possible benefits of a given outcome disregarding our expectations then maybe we will have more wins.

I suggest that we use the mind set of reiteration and liquid goals as seen in Yahtzee and Poker as a possible management methodology, examining components and maximising their utilisation.

Planning and Focus

Plans are great!

  • They help us focus and align to a goal
  • They demonstrate that we know what we are doing
  • They help us to break up work into manageable chunks so that we can deliver in stages or divide the work up between teams

But…

What if the goal is to achieve innovation or find new ideas?

Let’s take the 3 points above and see how they might be detrimental when we are dealing with complexity as defined in the Cynefin Framework.

Focus and alignment

When we focus we can miss things. Try staring at something nearby now – focus on its attributes and why it is there. Notice that while you are focussed, other things become less noticeable in our peripheral vision, hearing and other senses.

In an ordered environment, where things are knowable and there is a high amount of certainty, focus is a wonderful thing.

In an unordered environment of complexity or chaos it can destroy us in the worst case – and in the best case it will limit our ability to find new ideas and interesting things. We still need to have an idea of what we are looking for, setting some constraints or boundaries – but it is not a laser-like focus.

We know what we are doing

People who use traditional planning methods in complex environments do not know what they are doing. The planning needed in complexity is how to manage constraints, how to identify ‘good’ and ‘bad’ patterns and how to amplify or dampen them respectively. Risk and opportunity management methods are much more appropriate in complex environments and traditional planning is great in ordered domains.

PlanningPlanning

A person who knows what they are doing will use both types of planning/management, mixing and matching as they go.

Unfortunately, people are traditionally rewarded for showing how they executed to a plan and achieved an outcome. In this light, the outcomes achieved in a complex environment can only be described after they have happened. So it can look like random luck and is not generally well rewarded unless the outcome was an astonishing breakthrough or innovation of some sort. In these cases we will backfill the story in a retrospectively coherent way so that it seems the achievement was planned that way all along.

Retrospective CoherenceRetrospective Coherence

Divide up the work

Absolutely the best way to do something in the ordered domains. Break it up, build each part and then assemble it at the end.

When we try to do this in a complex environment we spend a lot of effort trying to get certainty. We also highly constrain the outcome to only that which we imagine is possible at that point in time.

So not only do we spend too much, we also sub-optimise our opportunities.

The better option is to explore the needs using test and learn approaches. Techniques for exploring the ‘fuzzy front end’ from design thinking work well as does prototyping, experimentation, articulation of assumptions and then validation or invalidation of them.

This is where we say that failure is good – let me explain that because many people have issues with such a statement.

When we are in a complex environment we do not know and cannot predict what will happen when we do action ‘XYZ’ and there are many actions that we could do. In order to find out what works and where the boundaries might be, we conduct experiments to see if there are stable and repeatable patterns. The experiments that work are what we are seeking and the ones that fail are outside of what we are seeking. Only by conducting experiments that fail can we be confident that we have found the edges of the space that we are exploring. If we don’t get failures, then we have not gone far enough and have missed opportunities for new ideas that might work.

BoundariesSo safe-to-fail experiments are important and the above helps to explain why we should be designing experiments that we expect to succeed and ones that we expect to fail – when do this we are guessing where the boundaries exist and the experiments confirm those boundaries.

Summary

These 3 examples demonstrate that our traditional planning and focus methods are really only suited to the ordered domains in the Cynefin Framework and that seemingly ‘opposite’ approaches are more effective in the complex domain.

We need more innovation and new ideas – we will not get them using standard planning and focus methods.

Plans are great – when we have certainty and predictability (and not when we want new ideas).

Not Perfect First Time

Not PerfectYou’ve spent the last 3 days putting together the deck for the workshop – speaking to the participants to get their input, collating, reviewing, updating and formatting.

Why?

So that we use the time in the workshop in the most efficient possible way.

Right?

Wrong.

A beautifully collated and presented deck is perfectly suited to use in a presentation – so we make the assumption that we should be creating one for a workshop as well.

What’s the Problem?

The problem is that the purpose of a workshop is to do some work – to identify issues, solve problems and get creative. If we start the session with a presentation deck, the participants will immediately focus on the presented content and not move very far from it. We might get some suggestions to fix the spelling, grammar or re-word a sentence – the focus will be on polishing the deck – not generating new content.

It can be a lot more useful to start with a ‘template’ type of deck which reflects the outcomes desired from the workshop and fill it in as the workshop progresses.

Even a template can act as a framing or anchoring bias and restrict the range of thinking – so if we are after innovation, it can be a problem.

So don’t try to get things perfect first time – allow for inputs, ideas and refinement – these things take time. If we do not allow the time, we miss opportunities for innovation and the quality that comes from stepping away and revisiting/reworking a piece later on.

Creativity

We can start with a goal in mind and then follow a process in order to reach that goal. I have been wondering if this approach sometimes stifles our creativity and what other approaches might also be valid.

 What if we started messing around and doing some random stuff?MessOnce we have done this for a while, we could stand back and look at our creation from different angles.

ArtPerhaps it will look like something useful and then we can decide what to do with it. Somewhere between random stuff and processes we can find creativity and innovation.

Comparative Context

Thank you to Steve Holt for the term ‘comparative context’ and Jay Johnson also for the twitter conversation that got me thinking about the difference between numbers and descriptive words when we are setting goals.

‘I want to stand close enough to an apple tree that I can smell the apples’ – in this case numbers are almost irrelevant because the words ‘close enough’ provide comparative context.

  • The apples must be ripe enough to smell – so it must be the right season
  • I must not have a bad cold – otherwise I will not smell anything
  • If it is raining, we will need cover, such as an umbrella

Imagine trying to describe this using numbers – we would need a lot more than this list.

  • Distance = 10cm
  • Rhino-virus level less than x%
  • Humidity less than 80%

Comparative ContextWhen we set goals, we often use a lot of numbers – perhaps we are trying to provide sufficient context. It is worth considering using descriptive words instead – a few words can provide a huge amount of meaning.

 Jay Johnson has posted his thoughts inspired by the conversation – Near, Far and Full Context

Goal Alignment

Ideally, a group put together from different areas would all be working on the same thing at any one time and towards the same goal. Often this is achieved in agile teams and works well.

Sometimes, there are other reasons to pull people together from different areas to work on several goals at once. Here is a way to get an idea about effective ways for groups like this to work together.

Examples of Goals

Ask the group to state their goals – in the example, the three goals are

  • Build Pyramids
  • Make Bricks
  • Destroy Pyramids

Put each goal across a page and also repeated down the left side – then ask for each pair of goals whether they align, conflict or are unconnected.

Goals GraphGoals Map

In this example

  • The goals of building pyramids and making bricks are aligned (and actually have a dependency).
  • The goals of building pyramids and destroying pyramids are conflicting
  • The goals of making bricks and destroying pyramids are unconnected

If we had a group put together with a goal map like this, we would need have discussions about the conflict and dependency so that we could work out the best way to start working together. In fact, we might ask why the people with the ‘destroy pyramids’ goal had been sent to this group in the first place.