Optimizing for the Unexpected

Alexis:

Welcome to Le Podcast on Emerging Leadership. I'm your host, Alexis Monville. Today, we are joined once again by a very special guest, Gojko Adzic. Gojko is a renowned author, speaker, and recognized leader in the world of software development. He's been celebrated as one of the 2019 AWS Serverless Heroes, the winner of multiple prestigious awards, and the mind behind several influential books, including Impact Mapping and Specification by Example. In our last conversation, we dove deep into how to build a perfect product, how to avoid waste in software development, and explored the principles of impact mapping.

Today, we are excited to discuss his latest book, Lizard Optimization. We'll be unpacking the core ideas in the book, how they apply to modern software development, and what it means for leadership in an evolving technological landscape. Welcome back to Le Podcast on Emerging Leadership, Gojko. How do you typically introduce yourself to someone you just met?

Gojko:

Well, I say I'm Gojko, it's like Beyoncé. I don't know if it works really well — I've never been in a situation where it doesn't, because maybe people try to be polite. I'm a developer. I build my own products. Now I write books mostly as a way of doing a brain dump so I can leave more space for other things. I stole that one from Henrik Kniberg — he likes to do a brain dump to free up short-term memory. Upgrading RAM in my head would be really expensive. It's cheaper to write a book.

Alexis:

I love the way it's said. When you try to write something that could be read not only by you but also by other people, it's really good to help you structure your own ideas.

Gojko:

Yeah, and it gets you to clarify things that might not be perfectly clear. While I was writing this most recent ninth book, I was trying to hunt down some quotes in the exact way they were said. I realized that for years I'd been doing conference presentations and quoting people completely wrong. I misremembered the way I read them in books. The meaning was there — I didn't misremember the meaning — but really shame on me for misquoting people. So you get to consolidate your thinking and verify it's still correct.

Alexis:

You spoke briefly about your latest book — Lizard Optimization. I would probably not have picked that book on a shelf. I don't know anything about lizards. I'm not really keen to optimize any lizard.

Gojko:

That's your mistake as a leader. I think your job needs to be to watch out for lizards and support your product teams in optimizing for lizards. That's incredibly important.

Alexis:

So now you need to explain a little bit. What inspired you to write the book and what does that lizard mean?

Gojko:

What inspired me was a really crazy growth phase for one of my products where the usage increased by about 500 times in a space of 11 months. Things that were weird edge cases happening once every two years now started happening every day. The whole 11 months was a bit crazy and full of firefighting, but I learned a lot and wanted to pass it on.

Lizard optimization, in a sentence, is figuring out how people are misusing your product, then deciding whether you want to support that misuse in a more systematic way, or block it. Both are valuable.

One example I really like — not from my product, but I read it in Founders at Work by Jessica Livingston — was from a company started in the late 90s by very smart people who built incredibly efficient cryptography algorithms. They had a solution but no problem. Eventually somebody said: these algorithms are efficient, they can run on low-power devices, like a PalmPilot. What do you need encryption on a PalmPilot for? Well, encryption brings security — useful when you're transferring money. So they built a system where you take your PalmPilot out of your pocket, I take mine, we bump them together, and money goes from my PalmPilot to yours. Magical, but kind of insane.

They had trouble getting people to know about the mobile app, so they built a website to promote it — with a rough demo where you could use the website to transfer money to somebody's PalmPilot account. People started using the website not to transfer money to a PalmPilot, but just to transfer money — even without owning PalmPilot devices. Product management was furious; they fought users in forums. Then somebody looked at the numbers: 1.5 million active users on the website, 12,000 PalmPilot installations. They killed the PalmPilot app, and the web app became PayPal.

The really interesting lesson is that the product managers fought users for a very long time because the misuse wasn't consistent with their vision. They were true to the vision, not to solving the problem. People fall in love with the solution, not the problem. As a leader, helping your product people focus on solving the problem — not loving the solution — is really important. Noticing when people are misusing your product becomes critical for unlocking growth and understanding where the market wants the product to go. If PayPal had stayed on the PalmPilot, they would have had 12,000 users. They would never have made a decent company out of it.

Lizards in this terminology are people who do things you can't logically explain. It looks like it was done by somebody who's not rational. They're misusing the system — in a good way or a bad way. Figuring that out becomes critical for good product management.

Alexis:

Just noticing that something is going on is already important. When I read the book I thought, “I don't know if I would have noticed.” Take the example of the blank video — how would you even know?

Gojko:

That's an interesting thing. One of the products I built lets people upload different types of documents and create an audio file using text-to-speech. When users do something unexpected, like uploading an unsupported file type, they get a decent error message. I see a number of people every day try to upload MP3 files into a text-to-speech system. I don't understand why. There are people who try to upload Android package files every day. Occasionally, somebody does something potentially useful.

When the user gets an error message, I also get a log message that something unexpected happened. About a year ago I noticed a pattern: people were trying to upload subtitle files. Subtitles come with video files — they're text files, not images, music, or APKs. I thought: well, I didn't expect this extension, but why not? It took five minutes to enable it. Then I started getting complaints that the system also reads the timestamps. People said: “you uploaded a file with timestamps, what did you expect?” It reads the content. So I spent five more minutes to skip over the timestamps. Then people complained that it reads the text too slowly.

Talking to people, I realized the job to be done was: create an alternate audio track for a presentation video. Instead of creating lots of short clips and aligning them manually, they wanted the subtitle file to handle synchronization. It was a small change — about two days of work in total — and by far the most profitable thing I've ever done.

Later, these features were discovered by an American mega-church producing sermons in many languages, and by an enterprise software company doing instructional videos. If you're a video editor, you'd usually have to record voice clips and place them manually in the video. With this, you get it almost instantly — saves hours and hours. Although this feature is used by a tiny percentage of users, it contributes a decent percentage of revenue, and the most profitable customers are the ones using it.

That's the value of lizard optimization. I would never have guessed this without monitoring weird file types people upload. I wouldn't have done it through user research, because I wouldn't be researching that. Lizard optimization complements customer and user research and helps you discover the unknown unknowns. Customer research helps you with known unknowns — you know what people want, but not the details. This helps with the unknown unknowns. It can open a completely new market segment.

Alexis:

In user research you're coming with your own assumptions and trying to validate them. Here, it's way more powerful because you're on the lookout for what's actually going on, instead of brushing things off as “those users are completely idiot.” Maybe they have a brilliant thing you can solve in two days of work. Do you have a structure to help people understand how it works?

Gojko:

I've nailed it down to four steps that are easy to remember because they spell LZRD.

L — Learn how people are misusing your product.

Z — Zoom in on one behavior to change. You can't change everything. There's a range of weird, from “we'll never understand it” to “this kind of makes sense.” You need to find the signal in the noise.

R — Remove obstacles. The software is placing obstacles in front of users that prevent them doing what they want. Some obstacles need to be removed.

D — Detect unintended impacts. These users follow their own logic, not yours. Our assumptions about how to fix the problem aren't necessarily true. My first idea — just support the file — turned out fine, but the timestamps issue created an unintended impact and increased support load. Many times what I thought was a good idea didn't turn out to be spectacularly good.

Alexis:

Can you give me an example of an unintended impact?

Gojko:

In the European Union there are VAT numbers. With a digital product, if you sell to individuals you charge VAT in their country; if you sell to companies, you don't (reverse charge). Companies want to enter their VAT number. Domestically they enter just the number, but in foreign contexts they put the country prefix — FR12345 for France.

The payment processor I use is American, and their validation is iffy. Even if you select France and enter 12345 (obviously the French number), it tells you “invalid VAT number.” People then think it's the card number, not the VAT number, and try the card number, which fails. A ridiculous number of people from the EU end up selecting Russia as their country because Russian VAT numbers don't have a prefix. It's idiotic — I'm placing an obstacle in front of people trying to pay me.

I thought: let's solve this. I can't control the validation, but I can remove the field altogether and ask for the VAT number after payment, then prefix it with “FR” based on the country. I did that and measured payments. People paid me less.

Unintended impact. People who wanted to pay for their company went to the form, didn't see a VAT number field, got confused, and didn't pay. The number of payments dropped significantly. I had to go back and rework things. To me as a maker it sounded perfectly logical; to a certain type of user it didn't.

This is where I love Kat Holmes' book Mismatch. She rephrased the whole thing — it's not that users are stupid or smart, it's that there's a mismatch between the user's capability and the software. We need to understand it as a mismatch. Some people expect a VAT field that isn't there — mismatch of expectations. The UI is too complex for someone who isn't a trained developer — mismatch. Visual capability — someone with bad vision, or someone on a beach in direct sunlight without enough contrast. Identifying mismatches lets us decide whether and how to solve them.

Maybe I can't build an app that fully works for blind people, but I can make one that works well for an elderly person with bad vision — and that also helps people on the beach or in dark environments. Kat Holmes talks about how you don't need to follow each difficult edge case economically — you figure out how to solve those and at the same time improve the product for everybody.

Alexis:

You have a small population of users affected from one angle, but in reality it helps a large group of your users.

Gojko:

You make a better product. A couple of years ago we had a bug report for MindMup, one of my products — a collaborative diagramming and mind-mapping tool. The report said it doesn't work well on a refrigerator. Well, it doesn't work well on a microwave either; it's intended for computers, not kitchen utensils.

A woman who stayed home in the mornings to take care of her children — this was before COVID and work-from-home — had an Android screen on her fridge with a browser. Our software requires a large screen, so a phone wasn't an option, and keeping a laptop open in the kitchen while cooking can damage expensive equipment. The problem wasn't that it didn't run on a fridge — it was that the software was useless without a keyboard. We never thought about people using it without a keyboard or a pointer.

Instead of making it run on a fridge — which one user in 10 years complained about — we thought: maybe there's a whole class of people not at the keyboard. Maybe a class of people who just need to observe rather than participate (she wanted to observe what her colleagues were doing). We changed the floating toolbar with tiny buttons to a large toolbar with big buttons. We came up with a much better UX in general — it works better even with a laptop, keyboard, and mouse, because we challenged ourselves to improve it.

Alexis:

It's not only discovering new use cases or new personas — it's really improving the product overall.

Gojko:

Kat Holmes has this principle: solve for one, expand to many. Especially with lizard behavior — these are weird things, and solving them just for the 0.1% is never economically justifiable. A product manager looks and says: this is 0.1% of users, I have to spend time on what 80% expect. But it's not about helping that 0.1% — it's about using them as signals that your software is placing obstacles in front of people, then figuring out where other groups face similar obstacles.

Alexis:

I love it. How would you translate that beyond software development or product building? For an emerging leader — how does it apply in an organization or a team?

Gojko:

A related concept from outside software is desire lines. There's a story about a university that built a new campus and, instead of trying to figure out where to put walking paths, they planted grass and let students walk on it. They figured where the grass was stepped on and built pathways there. From an organizational perspective, we can ask: what do we want our employees to do? How can I as a leader support them in what they actually want to do, not what we think they want?

I remember working with a hedge fund / small investment bank — small in this case meaning about 3,000 people, with a couple hundred developers. Whatever we suggested wasn't improving productivity, because the bottleneck was somewhere else. We mapped where people felt they were wasting time. One thing that came up was waiting for virtual machines to start. They had a recent policy: for business continuity reasons, no data on physical machines — everybody had to use remote Citrix VMs. Everyone arrived at the same time, started logging in, and average wait times were around 40 minutes because they didn't have enough capacity.

Developers' time in central London is expensive. We added it up, went to the CIO, and said: with this much money you can buy a lot more hardware. He said of course we can, it's logical — but why are people waiting for VMs at all? It was a company-wide policy. He said: yes, everybody — meaning traders — not developers. Developers don't store data on their machines anyway, it's in version control. So there was a totally different desire line. The company was misusing its own people. When they said “everybody,” they didn't mean developers.

As a leader, it's important to figure out where misuse is happening, where people are trying to use the system in a different way, whether you want to support that, and whether the obstacles you're placing in front of people are intentional. Sometimes they are. Sometimes they should be removed — like that policy. If you have version control, you don't need a VM.

Alexis:

Lastly, what would be the one piece of advice you would give to your younger self?

Gojko:

In terms of product building: not to trust that the things I do actually have value. To try to validate it. I've spent far too long trusting that the things I do are good ideas — and very often they're not.

I'm not alone. I love Ron Kohavi's book Trustworthy Online Controlled Experiments. It has data from companies like Microsoft, Google, Slack, and Netflix. Between one tenth and one third of things they do actually deliver value.

Alexis:

After that, you need to be a little bit more humble.

Gojko:

It means these people, who are supposed to be industry leaders, are wrong about seven out of ten times. Their ideas don't improve the product in a measurable way. As a leader or as a product manager, think about what brings value to the market so you can capture some of that value back. If you're not delivering value to the market, you can't capture value from users.

The bad news for most listeners who've never thought about this: about eight out of ten things you do make no sense. The good news: eight out of ten things your competitors do also make no sense. If you can figure that out faster than the competition, you create a much better product. That's why these companies win — they can measure and stop bad ideas from progressing too far.

Alexis:

This is very insightful. Thank you for sharing.

Gojko:

Trustworthy Online Controlled Experiments. Wonderful book.

Alexis:

I will add the references in the companion blog post. Thank you very much for joining the podcast, Gojko.

Gojko:

Thank you!