Monday, September 30, 2013

Duke Medical Center offers online training module "Anatomy of an Error"

@Whatsthepont pointed me to this online training module from Duke University Medical Center. Entitled "Anatomy of an Error," the course intends to teach hospital employees about the types of error they can expect and show them how to reduce error and avoid catastrophic outcomes through human factors and well-designed processes.

This is a very important piece of evidence that error and mistake awareness is becoming more prevalent in complex environments, such as health care, where it is badly needed. The Duke module is a good step forward. That being said, I have some criticisms of the course, which I will cover in a future post. For the moment, let's share the Duke definition of an error, which is aligned with the definition of a mistake from the introduction to the book:

Dr. James Reason, a Professor of Psychology who has published extensively on the nature of human error, describes error as circumstances in which planned actions fail to achieve the desired outcome.

Our definition of a mistake is: an unexpected result proceeding from faulty judgment or inadequate knowledge.

The Duke module is a very useful introduction to error and how it works in complex environments. But don't view it as the complete story - there are some significant missing components that I will discuss this week.

Friday, September 27, 2013

Mistake Bank Bookshelf: Tim Harford’s “Adapt”

Tim has been a frequent inspiration to this site for the past few years, initially for his TED talk entitled “Trial, Error and the God Complex.” His 2011 book Adapt: Why Success Always Starts with Failure is this week’s selection.

His background is in economics; he writes the “Undercover Economist” column for the Financial Times, and in Adapt the 2008 financial crisis is an exception of the failure-begetting-success theme that he strikes throughout the book. He also takes up the on-the-ground action of the wars in Iraq and Afghanistan, the changing composition of the Fortune 500, and the invention of the toaster as examples of adaptation all around us.

The lens used to investigate these events is that of complex adaptive systems – the methods used by biological evolution. It’s a fascinating book and an important addition to the Bookshelf.

Here’s a section describing a famous project by the psychologist Philip Tetlock:

[Tetlock] rounded up nearly three hundred experts – by which he meant people whose job it was to comment or advise on political and economic trends. They were a formidable bunch: political scientists, economists, lawyers and diplomats. There were spooks and think-tankers, journalists and academics. Over half of them had PhDs; almost all had postgraduate degrees. And Tetlock’s method for evaluating the quality of their expert judgment was to pin the experts down: he asked them to make specific, quantifiable forecasts – answering 27,450 of his questions between them – and then waited to see whether their forecasts came true. They rarely did. The experts failed, and their failure to forecast the future is a symptom of their failure to understand fully the complexities of the present.

[Tim has a new book out called The Undercover Economist Strikes Back.]

Thursday, September 26, 2013

Songwriter Jason Isbell "follows his mistakes" to develop a personal style

From an article in CountryWeekly.com:

Combining his personal experiences with fictional ideas to make a more compelling story is something Jason has honed over years of trial and error. “I try to follow my mistakes. I think that’s really the only way to develop a personal style,” he suggests. “If you start out imitating people, the ways you mess that up are usually the things that become your own signatures. I try to follow that.” 




Wednesday, September 25, 2013

A bad color choice for the Honda Civic

This story is from Honda designer Margo Beylen, from the Honda "Power of Dreams: Failure" video. 

One of my worst ideas happened the very first week I worked at Honda. My boss said, "I'm going to Japan in a couple of weeks and I need proposals for new colors for all the 1996 models." This was Civic, Accord, Accord Wagon.

I completely panicked. I was asked to do more in my first week at Honda than I had done in 4 years at my previous job.

I had been really wanting to do an orange car. Really wanting to do orange. And decided: OK, 96 Civic, we're going to do orange. Without doing my homework, completely, I proposed this color. I get into this big room. Sales, engineering, design, the balance of powers, whoever makes this decision to go into mass production. They're all looking at this new person, this new designer, saying, "Well, maybe we should trust her." So they went ahead and put the orange into production. And the dealers flipped. It's like, "What is this hideous, hideous color?Stop production of this immediately!"

You take chances. Everybody in that room understood they were taking a chance, but they were willing to take that chance.

Tuesday, September 24, 2013

New 99u book out featuring an essay by yours truly

Today the new book from 99u is out. It's entitled Maximize Your Potential: Grow Your Expertise, Take Bold Risks & Build an Incredible Career. Many well-known writers and researchers are featured, including Gretchen Rubin, Teresa Amabile, Seth Godin and Joshua Foer. I contributed an essay entitled "Understanding Your Role in Risk." Here's a taste:

When evaluating whether to take a risk, it’s easy to see the early-stage pitfalls you could encounter. Imagining the realization of the opportunity is much harder. But not making the attempt guarantees you won’t realize the ultimate benefits. It is necessary to broaden the mind, see the bigger picture, and know that with determination obstacles will be overcome.

Former Coca-Cola president Don Keough ran into this trap when his German management team presented a plan to expand into East Germany after the fall of the Berlin Wall in 1989. Believing the budget was too high, Keough vetoed the proposal. Frustrated by Keough’s summary rejection, the management team threatened to resign. The head of German operations challenged Keough to look more closely:

“You don’t know the potential of East Germany. You’ve never been there. You rejected it out of hand without considering that this could be a great opportunity. At the very least, you should talk to them again. But I’d like to ask you to do more. Come with me to see East Germany for yourself, first-hand, and make up your own mind.” Keough ended up going with his team to East Germany and seeing for himself. After the trip, he said, “My mind was completely changed. We assembled everyone together, and I apologized for being so narrowly focused and so intransigent. Together, we made plans then and there to buy several plants in the east.”

Coke’s expansion into East Germany was not without missteps. Yet the opportunity was so great, and the team’s determination to succeed so strong, they were able to work through the disappointments and setbacks. East Germany became a fast-growing and profitable market for Coke.

To get the rest, you'll have to order the book.

Monday, September 23, 2013

Honda shares failure stories

A very cool video from Honda, "Failure: The Secret of Success." It focuses on their racing experiences, but also includes manufacturing and engineering anecdotes.



Hat tip to What's the Pont.

Friday, September 20, 2013

Paul Graham unveils communication around the Union Square Ventures mistake not to invest in AirBnb

Y Combinator partner Paul Graham has shared the entire email correspondence relating to Fred Wilson and USV considering an early investment in AirBnb, now likely a $1B+ valued enterprise. The email story is here, and it is absolutely fascinating to me because I loved Fred's recounting of his mistake in declining to invest those few years ago, which I wrote about here, and in the book.

I spent some time this week talking to startups as part of the 99u Pop-Up School, and this story is essential reading for anyone trying to launch something brand-new. My favorite quote is from one of Fred's emails:

It's interesting

Our two junior team members were enthusiastic

The three "old guys" didn't get it

Mistake Bank Bookshelf: "The Ultimate Book of Heroic Failures" - a minor book ennobled by "the world's worst soccer player"

In Ultimate Book of Heroic Failures Stephen Pile has written a funny, lighthearted look at failures. It's a compendium of third-person anecdotes about people doing things badly. Stories include "The Slowest Cross-Channel Swim," "The Worst Photography Exhibition," and "The Least Successful Concert." There's no analysis and the subjects themselves have no voice. As such, it's a very minor addition to the reading list on mistakes and failure. Rather than commanding a prominent place on the bookshelf, it's probably better suited for reading in the bathroom.

Not that there's anything wrong with that.

The book is not without insights, though. In the Epilogue, Leonardo Cunha Lima, director of the documentary short film Mauro Shampoo, about the so-called "worst soccer player in Brazil," discussed why he chose to make a football movie about a non-star:

Mauro’s story had something that I find particularly interesting when I see it in documentaries—a sense of the irony of life. After all, this is a film about a man that turned the otherwise catastrophic fact of his failure into a positive thing. And his ultimate goal of becoming famous was accomplished in a completely unexpected upside-down way. And of course there is the fact that I believe that Mauro’s story expresses a love for the game much greater then any love of winning. That is actually the beautiful legacy of the Ibis Sport Club, it is a living example of pure love of the game. In essence, Brazil is a country that would love football just as much if it had never won any game at the World Cup. The persistence of Ibis is the proof of this love....

He is peculiarly endowed with heroic underdog characteristics that make him an iconic urban legend and a symbol of Brazil. His personal story serves as the perfect counter balance for success stories like Pele, Zico and the Ronaldinhos out there. After all most Brazilians will become Mauro Shampoos, and only a few will ever see the glories of a Ronaldo.

So Mauro is the perfect role model for failing with dignity and pride, but to still succeed in those things that really matter in life: a loving family, his own business and the respect of his community. What more could a man wish for?

Mauro Shampoo (2006) from LCL on Vimeo.

[Note: the interview excerpt I included is expanded somewhat from the version in the "Ultimate Book." I liked some of Lima's words that Pile chose to leave out. Original interview source here.]

Thursday, September 19, 2013

Lenny Bruce's measure of success: 1/3 of the audience supporting him

“What I learned from Lenny Bruce was: You don’t need the entire audience…. If you’re too needy of that entire audience, you won’t find your own style. [When I saw Lenny Bruce, he] had only a third of the audience with him. And he didn’t mind that at all,” comedian David Steinberg on Fresh Air, 2007.


I recalled this quote when I was putting together the recent post on feedback. It seems sensible to try to steer to the middle of the road–to try to make everyone happy. (This is a mistake-avoidance strategy.) But it limits the value you have and dilutes your deepest advantages. To be all you can be, you have to follow your inner voice, say what you think, and accept the consequences that some people (maybe most people) won’t agree. This is something that contrarian entrepreneurs know deeply.

Wednesday, September 18, 2013

Negative feedback valuable? Let us count the ways

John Butman's latest post on the HBR Blog Network, "The Benefits of Negative Feedback" is thoughtful and provocative. Butman writes:

I recently gave a lunchtime “author’s talk” at Children’s Hospital in Boston and, although I thought the talk went well, somebody in the audience didn’t like it at all. On the evaluation form, the person in question wrote a single word in the comment box: CONFUSING.

Thank you, whoever you are. While everybody else gave me good marks and said nice things, which I appreciated, my critic forced me into self-examination....

An idea that advocates any kind of change is likely to receive some amount of negative response. When you’ve invested time, energy, and passion into your idea, this rejection can hurt. Your first impulse may be to lash back, to rebut the rebuttal. But a better response is to let the backlash unfold a bit: It is likely that negative feedback will be the most useful in further developing your idea.

Butman's assertions support some of the arguments I make in chapter 2 of the book. Let me share a few thoughts about feedback.

1. It's a very inexpensive way to learn. If something isn't going right, patterns in feedback will show patterns you can use to fix things before they get more serious. If Butman's talk was confusing to the audience, better to figure that out right away and fix it, rather than ignore the issue and find out by no longer getting speaking invitations.

2. It's difficult to be open to negative feedback. Because it's painful, we often tune it out or even discourage people from providing it. This can be costly - see item #1 above. Instead you should welcome feedback. My favorite example of inviting negative feedback came from advertising executive MP Mueller, who took an unhappy customer to lunch and stated, "Lay it on me."

3. Patterns are key. A single piece of negative feedback could be an anomaly. But a pattern of feedback, pointing to the same issue, is highly relevant and actionable. (Fred Wilson discussed this on his post on employee exit interviews.)

4. Some folks just won't be into what you're doing... and that's OK. Comedian David Steinberg, on NPR's Fresh Air program, recalled that Lenny Bruce felt he was successful if even 1/3 of the audience was appreciating his act. Whatever you're doing, especially if you have a provocative point of view, won't appeal to everyone. Be comfortable with that.

Tuesday, September 17, 2013

Autotuning Mr. Rogers - what can result from a culture of intelligent failure

I had the opportunity last week to talk to a group of project managers and customer support managers at a large tech company about learning from mistakes. They were most interested in the "contract for intelligent failure" that Rita McGrath proposed. This contract specifies, for a particular workgroup, the parameters around acceptable failure. It defines what acceptable failure is and creates a zone of safety for people trying new things.

Now I've just read "How I Got My Team To Fail More" by Jason Seiken on the HBR Blog Network.  Seiken, head of PBS' digital operations, was able to break through his unit's risk aversion and complacency - which caused growth to stagnate - by mandating that they fail more. Seiken recounted how the team learned how to work to this new objective:

In a beautifully ironic twist, the failure metric itself initially failed. We originally envisioned the metric as a formal KPI in each staffer’s annual performance review. But we soon realized we had created a contradiction: You can’t build a culture that values rapid iteration by simply changing an annual performance cycle. We needed daily reinforcement of the desirability of risk-taking and failing fast.

So instead of spending cycles working with HR to create a KPI measuring lack of failure, we focused on endlessly repeating the “must fail” message.

The change was rapid and profound. Some staff were uncomfortable with the new culture and left. Others began taking risks. The product manager working on our first augmented reality site for PBSKids.org ditched her plans for months of customer research and testing in favor of a 10-week sprint to launch. The site failed. The product manager? She received a spot bonus and her “smart failure” was listed as a top accomplishment in her glowing annual review....

Crucially, we redefined success. When our first foray into web-original video production, a safe, TV-type series called “The Parent Show,” launched to fairly good reviews, we resisted the temptation to declare victory. Instead, the team challenged itself to risk breaking the PBS mold by creating a truly YouTube-native show.

This led to the Mr. Rogers remix, “Garden of Your Mind,” which auto-tuned old clips so Mr. Rogers bursts into song. Within 48 hours, it rose to the top of the most viewed and most shared videos charts on YouTube.

Before the failure metric, the team would have considered a Fred Rogers music video to be risky at best, sacrilege at worst. Instead, the culture change triggered by the failure metric gave the team comfort that even if this blew up in their face, they would be protected.

Ordering people to fail more doesn't work unless they are given the tools to take intelligent risks. What would your "contract for intelligent failure" look like?

Here's an example of the residue of the failure-tolerant culture:

Monday, September 16, 2013

From HBR - how to recover from being fired

This post from John Beeson on the HBR Blog Network has some good advice on bouncing back from losing your job:

You've just received word that you've been fired. Or perhaps the company has gone through a re-structuring and eliminated your job — and you've been told that none of the managers you've worked with over the years have a position for you on their team. This comes as a shock to your system, especially if you've enjoyed a record of success up to this point in your career. While there are some practical things to attend to — negotiating your severance, signing up references, and agreeing with the company on a storyline about the reason for your exit — your most important action item is managing your own attitude to the situation....

As you dust yourself off, think through those parts of the situation you need to own. In a highly emotional state, it's too easy for you to curse the darkness: "I had a bad boss." "The place was rife with organizational politics." "My colleagues were non-cooperative and had it in for me." There may be some truth to this, but you also need to ask yourself, "What do I need to accept about the experience to avoid making the same mistakes so I can succeed in the future?"

Losing your job is a very particular type of failure. It's likely the most emotionally painful event of your career. Analyzing your role in the situation is not natural; you will want instead to lash out and place blame on others (as Beeson writes, this may very well be the case), or to fall into a funk of despair. But to bounce back, better to "own" the outcome and figure out what lessons you can take into the rest of your career.

Thursday, September 12, 2013

Malcolm Gladwell discusses something he'll never do again

A tiny but useful mistake story by Malcolm Gladwell, from the disclosure page on his website - a fascinating document in its own right. Here he discusses one of the principles he uses to balance his roles as New Yorker writer and paid public speaker:

Any speaking I do should be just that—speaking. The possibility for trouble is much greater when a writer steps outside the role of giving a set speech and becomes a consultant or advisor, or in some ways develops a continuing financial arrangement with a specific company or organization. Once, I was asked to be a consultant to a marketing firm. I said yes—briefly—then immediately resigned when I realized the implications of that act. I will not do that again. Anything outside of simple speaking is inappropriate.

"I will not do that again" is a bright signal that this is a mistake story that had a significant impact on the teller.

Wednesday, September 11, 2013

Ashley Good of AdmittingFailure.com discusses learning from failure

This is the keynote presentation Ashley Good gave at FailCon Oslo back in June. Ashley is a fellow traveler who helps companies and non-governmental organizations tell their failure stories so they can learn and improve. Her site AdmittingFailure.com includes stories and discussions of learning from failure. Her talk is a great reinforcement of many points we discuss at this site. Enjoy!



Money quote from a hockey coach to a player who was focused on playing perfectly: "You are not falling enough... practice is for finding your edge." Failure is a way to test our limits and understand what we have to do to get better.

One of Ashley's pieces of work is the annual Failure Report published by the Canadian NGO Engineers Without Borders. You can access its 2012 Failure Report here.

Monday, September 9, 2013

The positive outcome of a product development failure

Karsten Strauss, writing on Forbes.com, discussed the journey of Zach and Max Zitney, the undergraduate creators of MiRing, a Bluetooth-enabled ring that would vibrate when the wearer's phone rang or received a text message.

The product, despite placing third in an Ohio State University entrepreneur's competition, far undershot its goal on crowdfunding site Fundable.com - raising $8000 in pledges versus a goal of $150,000. A failure? Yes, but not an ending.

Max Zitney’s outlook is that the disappointing campaign saved the two a lot of grief because it showed them that the product, in its current configuration, was obviously not sparking a lot of enthusiasm.

But the Zitneys’ story doesn’t end by merely recognizing the silver lining. A UK-based company called NFC Ring, which has developed a similar idea, approached the brothers, offering to inspect their concept and possibly collaborate in the near future....

NFC Ring appreciated the Zitneys’ idea but said the technology is not yet in place to create a ring-size, Bluetooth-enabled vibrating notification device. Those features, however, are ones that the company would like to incorporate into its product . “They said that’s their ultimate goal,” said Zach Zitney, adding that NFC Ring intends to resume talks with the brothers when they’re ready to start integrating MiRing’s designs into their own technology. Given the fact that the Zitneys have a U.S. patent on such a gadget, they may have to.

“There are still options available,” Max Zitney said. “The crowdfunding just opened up the doors for us.”

So: failure, especially cheap and fast failure, does not mean that a project is finished. It may just be a step on the way. The Zitneys did not achieve their funding goal, but the crowdfunding project exposed them to new opportunities and proved to them that going it alone would not be a success - an example of an "intelligent failure."

Hat tip Roxanne Persaud.

Friday, September 6, 2013

Mistake Bank bookshelf: Paul Schoemaker's "Brilliant Mistakes" - mistakes as strategic assets

Kwame recently posted this comment: "No more mistake bank bookshelf? I am a huge fan of your work. supported kickstarter and would like to see more books that focus on mistakes." That was the impetus I needed to restart this segment. I have three mistake books in the reading queue, but for the moment I wanted to reshare a classic in the field, Paul Schoemaker's Brilliant Mistakes: Finding Success on the Far Side of Failure.

Schoemaker's book is a highly strategic look at mistakes. (He and a collaborator, Robert Gunther, originated the concept of "deliberate mistakes," which is to me the most counterintuitive and least-understood concept we've covered here) To Schoemaker, mistakes are tools for understanding the world, learning more quickly, and discovering deep insights.

I've had the opportunity to meet him and his book is a perfect reflection of his keen mind and curious soul. Brilliant Mistakes was my favorite book of 2011, and you should read it too.

From the book: Companies strive for error elimination, hiring advisers and relying on sophisticated management tools such as Six Sigma. It’s little wonder, then, that most decision-making books follow suit, encouraging you to focus narrowly on mistake avoidance today rather than provoking you to plan for the stream of decisions that you will face tomorrow.

Thursday, September 5, 2013

Improving large, distributed information systems by inducing failures

I've been thinking about this powerhouse paper from the Association for Computing Machinery's acmqueue site for the last week. It's called "The Antifragile Organization: Embracing Failure to Improve Resilience and Maximize Availability" by Ariel Tseitlin.

Taking its starting point from Nassim Taleb's arguments about antifragility (see references here and here), Tseitlin discusses ways of making large distributed information services antifragile - i.e., able to capitalize and improve themselves in the wake of disruption. His focus is on testing and simulation, the question of how to exercise highly complex systems to ensure they will not collapse as a result of stressors.

As Tseitlin observes, traditional scripted testing is utterly unsuited to this task - in systems of any size, it's impossible to build (or even imagine) the total number of test cases required to prove a system's robustness. Moreover, even the largest test system is a fraction of the size and complexity of the production environment.

Taking a radically different approach, companies like Amazon and Netflix are increasingly causing intentional disruption within their production systems to assess whether their resilience mechanisms are working as expected - and whether new vulnerabilities have emerged. Tseitlin describes several ways that this is done:

Once you have accepted the idea of inducing failure regularly, there are a few choices on how to proceed. One option is GameDays, a set of scheduled exercises where failure is manually introduced or simulated to mirror real-world failure, with the goal of both identifying the results and practicing the response—a fire drill of sorts. Used by the likes of Amazon and Google, GameDays are a great way to induce failure on a regular basis, validate assumptions about system behavior, and improve organizational response.

But what if you want a solution that is more scalable and automated—one that doesn't run once per quarter but rather once per week or even per day? You don't want failure to be a fire drill. You want it to be a nonevent—something that happens all the time in the background so that when a real failure occurs, it will simply blend in without any impact.

One way of achieving this is to engineer failure to occur in the live environment. This is how the idea for "monkeys" (autonomous agents really, but monkeys inspire the imagination) came to Netflix to wreak havoc and induce failure. Later the monkeys were grouped together and labeled the Simian Army.

Netflix's "monkeys" include, among others, a Chaos Monkey, which randomly terminates virtual instances in the production environment; and a Latency Monkey, which inserts delays into various components of the network.

These are run regularly and the Netflix team carefully measures to see whether the system adapts suitably to the disruption. The random and low-level nature of these tests helps avoid limits of human-scripted testing, and the fact they are running in the production environment means they are not limited by the size of a test environment.

Is it risky? Not if the system has been engineered not to collapse under stressors. Top quality distributed systems are built to isolate failures and degrade gracefully rather than result in catastrophic downtime. As such, it does require a system that has been in production long enough to develop stability - Twitter in its first two years would not be a good candidate for this, but Twitter at present would be.

Tseitlin concludes the paper by discussing how these exercises in resilience are building toward true antifragility, by using tools such as post-exercise blameless postmortems and requiring that developers also be operators, the better to anticipate code that might cause operational issues down the line.

Tuesday, September 3, 2013

Richard Diebenkorn: "Mistakes can't be erased but..."

Great quote from painter Richard Diebenkorn as quoted by Maria Popova in Brain Pickings. From Diebenkorn's handwritten "Notes to myself on beginning a painting":

Mistakes can't be erased but they move you from your present position.

Sunday, September 1, 2013

Marilyn vos Savant: Cheating "makes it appear you are more proficient than you really are"

Marilyn vos Savant's advice column in Parade Magazine has been a guilty pleasure of mine for years. But today's Q+A seemed apt for our site, and has echoes of an earlier post of ours, "No Mulligans."

Question: Moralizing aside, can you give me any persuasive reason why my friends and I shouldn’t cheat at school whenever we’re sure we’re not going to get caught?

Marilyn: Aside from what may happen if you get caught anyway? Possibly the best reason not to cheat is that doing so may make it appear that you are more proficient in a subject than you actually are. This will lead you into increasingly deeper water without knowing how to swim well enough. As you move into academic areas or courses that are over your head (because you didn’t master the foundations for whatever reason, such as not studying as much as you should have), you will find yourself in academic trouble that no amount of cheating can solve.