Imagine actually being excited about an upcoming review meeting. You have three key metrics you are responsible for, and your overall numbers were pretty decent this quarter. However, you are most excited to present some big ideas about ways to scale and innovate.
Your part of the review meeting starts and you toss up the first chart to show past performance. As you start to move to the second slide, someone speaks up, “Can you explain what happened between the second and third week of August? It looks like there was a big dip there?”
And there goes the meeting.
You spend the next 25 minutes coming up with 20 ideas about why there was a dip one week.
Bad metric habits
Metrics are powerful tools for every business leader but most of us never got training on how to present or analyze data. As such, we’ve picked up a lot of bad behaviors around metrics:
Our charts tend to unnecessarily highlight natural fluctuations
We spend way too much time and energy trying to explain these fluctuations
People with no statistical training feel compelled to make a comment on data points that don’t need to be discussed
We set arbitrary targets for metrics without changing how we will achieve them
As soon as I finished my conversation with Mark Graban about Work Minus Overreacting to Metrics, I knew that I needed to understand his area of expertise better. Mark’s book Measures of Success lays out some foundational principles that are essential to anyone who needs to report data.
It’s not about the numbers
The main point of Mark’s book is that metrics can only tell the story of your system. If you have a metric you are chasing, you have to set up a system to try to achieve it. There are natural upper and lower limits to the metrics any system can produce, and the system will almost always fluctuate around an average.
When you plot all of these data points on a Process Behavior Chart, it becomes clear that certain data points are only noise–random blips within the natural limits that aren’t worth talking about. However, other points are real signals–clues to tell you that something has either changed in the system or its environment.
When given a target metric, most of us are focused on that number. However, Mark shares this powerful thought:
We don’t manage the metric; we manage the system that leads to the results, and we lead the people who help us improve the system.
Numbers don’t happen on their own. When we miss or achieve a target, it is rarely the result of trying harder; mostly it is a reflection of the quality of the system we have in place. So, the metric shows us if the system is adequate or not. If it isn’t, then we need to work with the people who are a part of it to improve it.
Don’t chase numbers, chase improvements
Unfortunately, most leaders are taught to just look at the number. When the goal is the number and not the improvement of the system, then Goodhart’s law comes into effect and people tend to either distort the number or distort the system to artificially affect the number.
How many times have you walked out of a planning meeting where someone sets a ridiculously high target metric and then doesn’t plan to change the system at all to achieve it? Mark says this is due to the misconception that you can easily squeeze better results out of any system just by trying harder, or hiring a different manager. Instead, if you want a higher target than your system is naturally capable of, you need to make significant changes to the system.
Worry about the signal, not the noise
A good chunk of this book shows that any system will have natural fluctuations–two data points are not a trend. When leaders don’t acknowledge these fluctuations, they end up spending a lot of time and energy trying to explain missed targets. Instead, when you accept that the numbers will go up and down, you can spend time focused on the real signals–signs that there has been a change in the system or its environment. When these signals happen, you can test it by making a change and see how it affects the metric.
Mark Graban makes process behavior charts easily accessible and shows how to create and plot them without needing a statistics degree.
Taking it deeper
Here are a few more of my takeaways from the book:
Weekly data points are much more helpful than monthly when it comes to analyzing signals and noise, and daily is better than weekly.
Be wary of using linear trend lines on your charts, because they suggest that your metrics will forever only go up and you are setting yourself up to have to explain what happens when they reach their limit.
It’s best to test out one idea at a time to improve a system so that you can know its effect, but you may not always have this luxury.
Mark brings up the idea of success theater, which is basically how we can bend and distort numbers to say what we want. In fact, when reviewing data in a meeting, I find that most of what is happening is everyone is deciding what kind of narrative to tell about the data. If the data looks bad, can you spin a story that excuses it and assume it will get better on its own? Can you cherry pick a few data points to spin a positive story? It’s this interpretation and narrative casting that seems much more powerful than the data itself.
I find that I’m always tweaking my systems, so how can I trust my metrics? Mark says that you need 20 data points to really be able to plot out a trend. I’ve rarely gone 20 weeks without making some kind of change. Perhaps this is a case for either collecting more frequent data or allowing a system to exist on its own for a longer time to test changes.
As much time as Mark Graban has spent with charts and data, he acknowledges that it’s really hard to know the causes of changes; we can usually only guess. This is a hard pill to swallow sometimes. Even if we know our metrics are accurate, we often have no idea what they mean. That doesn’t mean we shouldn’t pay attention, but it needs to be acknowledged.
Measures of Success
If you are in any kind of leadership role, you’ve probably got metrics that you need to track. Mark Graban’s book, Measures of Success, is a great tool to help you focus not on those metrics but on the systems that produce the numbers, and to acknowledge that there are natural fluctuations. Check out the book, and Mark’s blog.
Technology industry leaders face a lot of challenges when it comes to heading their digital transformation efforts. Where do you start? Are we too big? Are we too small? How much budget do I need? What do I do about low adoption rates?
But the one element that is most challenging of all is a community. Nearly every leader tasked with trying to figure out digital transformation is doing it alone.
There’s not a platform for like-minded people to discuss ideas and have a candid conversation about the challenges they face daily in their organizations. Leaders are left alone to fight their battles.
So, we launched a new initiative called WorkMinus Boardrooms–an event series that brings tech leaders together to talk about digital transformation, share struggles, learn from others, and encourage each other.
The event was set in a cozy, comfortable lounge where people could converse freely and discuss their ideas with everyone present in the room.
We invited 14 leaders from different industries and roles. There were C-level executives, IT directors, digital transformation heads, and financial advisors. Some of them had a successful digital transformation strategy in place while others were wondering if it is just a trend they had to wade through until the next big thing comes by.
The event kicked off with Neil sharing the backstory of WorkMinus–why we care about the future of work, where we are heading, and how to bend the trajectory back to a better place.
A case for digital transformation
Our main speaker was Mr Sukumar Rajagopal, former CIO of Cognizant Technology Solutions, a Fortune 200 company.
When he started his role in 2007, Sukumar was asked by the CFO, “Are you going to be about transformation or efficiency?”
Sukumar said, “Transformation.” The CFO said, “That’s fine, but you won’t get any more budget to do it.”
Sukumar used lean startup principles to get the project going and first focused on the company’s timesheets. He faced a lot of challenges with users not adopting his tools even though they were far superior to the other alternatives. He shared how eventually, the entire company of more than a hundred thousand people started using the digital tools they had created.
The amazing part was that by pursuing transformation, Sukumar led the company to greater efficiency too. Within the first three years of replacing manual processes, Cognizant saw a 5x increase in the organization’s productivity.
“When organizations transform, efficiency automatically follows”, he said.
A new digital workplace for the future
This event was sponsored by Kissflow, and the CEO Mr Suresh Sambandam spoke about how modern organizations can adopt a digital workplace platform that isn’t IT-driven, but people-driven. If the platform is built right, then the finance team can create their own timesheet app in a few hours rather than waiting for IT to take six months to produce it.
“A whole generation of employees that’s used to sophisticated software and world-class apps are entering the workforce today. They have certain expectations of the software they use at work. If you were to engage this workforce, you must be digital at your core,” he said.
Suresh quoted a report from Forrester that positioned a digital workplace as the operational system of record for a company. “Your ERP is your financial system of record. Your CRM is your customer system of record. But everything that happens operationally is scattered around dozens of apps instead of on a common platform.” Suresh said that an employee-first model is key to the digital transformation success of any organization.
When we established this event, we wanted to make sure there were no spectators, only participants. The issues we discuss aren’t able to be answered by a single person or at a single event. We want to start the conversation and get people listening to each other.
We passed around sticky notes and asked, “What are your biggest digital transformation fears?”
Here are some of the most common answers we got:
Employee readiness and changing their mindset
Using different tools for different business cases
The size of their organization
Measuring the success
Then we took one of the most burning questions that nearly everyone had about digital transformation:
How the heck do you measure the ROI?
Here’s the summary of the discussion:
It all begins with understanding what you want. Digitize data? Improve employee experience? Or, boost operational efficiency? Find focus areas like customers, infrastructure, and innovative disruption and start from there.
Depending on the functional area you choose, you can come up with metrics. Say, you choose customers, your KPIs would be the net promoter score or the churn rate or positive feedback.
Zoom back and look at your efforts holistically. See if it is affecting other areas, either positively or negatively.
Place high importance on stability while going through a transformation.
Swag in the bag
Once the discussion came to a halt, each guest received a bunch of WorkMinus goodies.
A networking session over brunch followed. The guests exchanged cards. Some continued their discussion while others bonded over the books they read and love.
And, that’s a wrap!
We broke away from all the norms with this event. We had an awesome bunch of people that sparked amazing ideas continuously for over two and a half hours.
We want to thank all our guests and our sponsor Kissflow for making this an absolutely delightful event.
Follow us on Twitter and LinkedIn (if you haven’t already) and watch this space for upcoming events. We have a lot more coming in the future. Stay tuned.