Duke Medical Center offers online training module "Anatomy of an Error"

@Whatsthepont pointed me to this online training module from Duke University Medical Center. Entitled "Anatomy of an Error," the course intends to teach hospital employees about the types of error they can expect and show them how to reduce error and avoid catastrophic outcomes through human factors and well-designed processes.

This is a very important piece of evidence that error and mistake awareness is becoming more prevalent in complex environments, such as health care, where it is badly needed. The Duke module is a good step forward. That being said, I have some criticisms of the course, which I will cover in a future post. For the moment, let's share the Duke definition of an error, which is aligned with the definition of a mistake from the introduction to the book:

Dr. James Reason, a Professor of Psychology who has published extensively on the nature of human error, describes error as circumstances in which planned actions fail to achieve the desired outcome.

Our definition of a mistake is: an unexpected result proceeding from faulty judgment or inadequate knowledge.

The Duke module is a very useful introduction to error and how it works in complex environments. But don't view it as the complete story - there are some significant missing components that I will discuss this week.

Mistake Bank Bookshelf: Tim Harford’s “Adapt”

Tim has been a frequent inspiration to this site for the past few years, initially for his TED talk entitled “Trial, Error and the God Complex.” His 2011 book Adapt: Why Success Always Starts with Failure is this week’s selection.

His background is in economics; he writes the “Undercover Economist” column for the Financial Times, and in Adapt the 2008 financial crisis is an exception of the failure-begetting-success theme that he strikes throughout the book. He also takes up the on-the-ground action of the wars in Iraq and Afghanistan, the changing composition of the Fortune 500, and the invention of the toaster as examples of adaptation all around us.

The lens used to investigate these events is that of complex adaptive systems – the methods used by biological evolution. It’s a fascinating book and an important addition to the Bookshelf.

Here’s a section describing a famous project by the psychologist Philip Tetlock:

[Tetlock] rounded up nearly three hundred experts – by which he meant people whose job it was to comment or advise on political and economic trends. They were a formidable bunch: political scientists, economists, lawyers and diplomats. There were spooks and think-tankers, journalists and academics. Over half of them had PhDs; almost all had postgraduate degrees. And Tetlock’s method for evaluating the quality of their expert judgment was to pin the experts down: he asked them to make specific, quantifiable forecasts – answering 27,450 of his questions between them – and then waited to see whether their forecasts came true. They rarely did. The experts failed, and their failure to forecast the future is a symptom of their failure to understand fully the complexities of the present.

[Tim has a new book out called The Undercover Economist Strikes Back.]

Songwriter Jason Isbell "follows his mistakes" to develop a personal style

From an article in CountryWeekly.com:

Combining his personal experiences with fictional ideas to make a more compelling story is something Jason has honed over years of trial and error. “I try to follow my mistakes. I think that’s really the only way to develop a personal style,” he suggests. “If you start out imitating people, the ways you mess that up are usually the things that become your own signatures. I try to follow that.” 

A bad color choice for the Honda Civic

This story is from Honda designer Margo Beylen, from the Honda "Power of Dreams: Failure" video. 

One of my worst ideas happened the very first week I worked at Honda. My boss said, "I'm going to Japan in a couple of weeks and I need proposals for new colors for all the 1996 models." This was Civic, Accord, Accord Wagon.

I completely panicked. I was asked to do more in my first week at Honda than I had done in 4 years at my previous job.

I had been really wanting to do an orange car. Really wanting to do orange. And decided: OK, 96 Civic, we're going to do orange. Without doing my homework, completely, I proposed this color. I get into this big room. Sales, engineering, design, the balance of powers, whoever makes this decision to go into mass production. They're all looking at this new person, this new designer, saying, "Well, maybe we should trust her." So they went ahead and put the orange into production. And the dealers flipped. It's like, "What is this hideous, hideous color?Stop production of this immediately!"

You take chances. Everybody in that room understood they were taking a chance, but they were willing to take that chance.

New 99u book out featuring an essay by yours truly

Today the new book from 99u is out. It's entitled Maximize Your Potential: Grow Your Expertise, Take Bold Risks & Build an Incredible Career. Many well-known writers and researchers are featured, including Gretchen Rubin, Teresa Amabile, Seth Godin and Joshua Foer. I contributed an essay entitled "Understanding Your Role in Risk." Here's a taste:

When evaluating whether to take a risk, it’s easy to see the early-stage pitfalls you could encounter. Imagining the realization of the opportunity is much harder. But not making the attempt guarantees you won’t realize the ultimate benefits. It is necessary to broaden the mind, see the bigger picture, and know that with determination obstacles will be overcome.

Former Coca-Cola president Don Keough ran into this trap when his German management team presented a plan to expand into East Germany after the fall of the Berlin Wall in 1989. Believing the budget was too high, Keough vetoed the proposal. Frustrated by Keough’s summary rejection, the management team threatened to resign. The head of German operations challenged Keough to look more closely:

“You don’t know the potential of East Germany. You’ve never been there. You rejected it out of hand without considering that this could be a great opportunity. At the very least, you should talk to them again. But I’d like to ask you to do more. Come with me to see East Germany for yourself, first-hand, and make up your own mind.” Keough ended up going with his team to East Germany and seeing for himself. After the trip, he said, “My mind was completely changed. We assembled everyone together, and I apologized for being so narrowly focused and so intransigent. Together, we made plans then and there to buy several plants in the east.”

Coke’s expansion into East Germany was not without missteps. Yet the opportunity was so great, and the team’s determination to succeed so strong, they were able to work through the disappointments and setbacks. East Germany became a fast-growing and profitable market for Coke.

To get the rest, you'll have to order the book.

Honda shares failure stories

A very cool video from Honda, "Failure: The Secret of Success." It focuses on their racing experiences, but also includes manufacturing and engineering anecdotes.



Hat tip to What's the Pont.

Paul Graham unveils communication around the Union Square Ventures mistake not to invest in AirBnb

Y Combinator partner Paul Graham has shared the entire email correspondence relating to Fred Wilson and USV considering an early investment in AirBnb, now likely a $1B+ valued enterprise. The email story is here, and it is absolutely fascinating to me because I loved Fred's recounting of his mistake in declining to invest those few years ago, which I wrote about here, and in the book.

I spent some time this week talking to startups as part of the 99u Pop-Up School, and this story is essential reading for anyone trying to launch something brand-new. My favorite quote is from one of Fred's emails:

It's interesting

Our two junior team members were enthusiastic

The three "old guys" didn't get it

Mistake Bank Bookshelf: "The Ultimate Book of Heroic Failures" - a minor book ennobled by "the world's worst soccer player"

In Ultimate Book of Heroic Failures Stephen Pile has written a funny, lighthearted look at failures. It's a compendium of third-person anecdotes about people doing things badly. Stories include "The Slowest Cross-Channel Swim," "The Worst Photography Exhibition," and "The Least Successful Concert." There's no analysis and the subjects themselves have no voice. As such, it's a very minor addition to the reading list on mistakes and failure. Rather than commanding a prominent place on the bookshelf, it's probably better suited for reading in the bathroom.

Not that there's anything wrong with that.

The book is not without insights, though. In the Epilogue, Leonardo Cunha Lima, director of the documentary short film Mauro Shampoo, about the so-called "worst soccer player in Brazil," discussed why he chose to make a football movie about a non-star:

Mauro’s story had something that I find particularly interesting when I see it in documentaries—a sense of the irony of life. After all, this is a film about a man that turned the otherwise catastrophic fact of his failure into a positive thing. And his ultimate goal of becoming famous was accomplished in a completely unexpected upside-down way. And of course there is the fact that I believe that Mauro’s story expresses a love for the game much greater then any love of winning. That is actually the beautiful legacy of the Ibis Sport Club, it is a living example of pure love of the game. In essence, Brazil is a country that would love football just as much if it had never won any game at the World Cup. The persistence of Ibis is the proof of this love....

He is peculiarly endowed with heroic underdog characteristics that make him an iconic urban legend and a symbol of Brazil. His personal story serves as the perfect counter balance for success stories like Pele, Zico and the Ronaldinhos out there. After all most Brazilians will become Mauro Shampoos, and only a few will ever see the glories of a Ronaldo.

So Mauro is the perfect role model for failing with dignity and pride, but to still succeed in those things that really matter in life: a loving family, his own business and the respect of his community. What more could a man wish for?

Mauro Shampoo (2006) from LCL on Vimeo.

[Note: the interview excerpt I included is expanded somewhat from the version in the "Ultimate Book." I liked some of Lima's words that Pile chose to leave out. Original interview source here.]

Lenny Bruce's measure of success: 1/3 of the audience supporting him

“What I learned from Lenny Bruce was: You don’t need the entire audience…. If you’re too needy of that entire audience, you won’t find your own style. [When I saw Lenny Bruce, he] had only a third of the audience with him. And he didn’t mind that at all,” comedian David Steinberg on Fresh Air, 2007.


I recalled this quote when I was putting together the recent post on feedback. It seems sensible to try to steer to the middle of the road–to try to make everyone happy. (This is a mistake-avoidance strategy.) But it limits the value you have and dilutes your deepest advantages. To be all you can be, you have to follow your inner voice, say what you think, and accept the consequences that some people (maybe most people) won’t agree. This is something that contrarian entrepreneurs know deeply.

Negative feedback valuable? Let us count the ways

John Butman's latest post on the HBR Blog Network, "The Benefits of Negative Feedback" is thoughtful and provocative. Butman writes:

I recently gave a lunchtime “author’s talk” at Children’s Hospital in Boston and, although I thought the talk went well, somebody in the audience didn’t like it at all. On the evaluation form, the person in question wrote a single word in the comment box: CONFUSING.

Thank you, whoever you are. While everybody else gave me good marks and said nice things, which I appreciated, my critic forced me into self-examination....

An idea that advocates any kind of change is likely to receive some amount of negative response. When you’ve invested time, energy, and passion into your idea, this rejection can hurt. Your first impulse may be to lash back, to rebut the rebuttal. But a better response is to let the backlash unfold a bit: It is likely that negative feedback will be the most useful in further developing your idea.

Butman's assertions support some of the arguments I make in chapter 2 of the book. Let me share a few thoughts about feedback.

1. It's a very inexpensive way to learn. If something isn't going right, patterns in feedback will show patterns you can use to fix things before they get more serious. If Butman's talk was confusing to the audience, better to figure that out right away and fix it, rather than ignore the issue and find out by no longer getting speaking invitations.

2. It's difficult to be open to negative feedback. Because it's painful, we often tune it out or even discourage people from providing it. This can be costly - see item #1 above. Instead you should welcome feedback. My favorite example of inviting negative feedback came from advertising executive MP Mueller, who took an unhappy customer to lunch and stated, "Lay it on me."

3. Patterns are key. A single piece of negative feedback could be an anomaly. But a pattern of feedback, pointing to the same issue, is highly relevant and actionable. (Fred Wilson discussed this on his post on employee exit interviews.)

4. Some folks just won't be into what you're doing... and that's OK. Comedian David Steinberg, on NPR's Fresh Air program, recalled that Lenny Bruce felt he was successful if even 1/3 of the audience was appreciating his act. Whatever you're doing, especially if you have a provocative point of view, won't appeal to everyone. Be comfortable with that.

Autotuning Mr. Rogers - what can result from a culture of intelligent failure

I had the opportunity last week to talk to a group of project managers and customer support managers at a large tech company about learning from mistakes. They were most interested in the "contract for intelligent failure" that Rita McGrath proposed. This contract specifies, for a particular workgroup, the parameters around acceptable failure. It defines what acceptable failure is and creates a zone of safety for people trying new things.

Now I've just read "How I Got My Team To Fail More" by Jason Seiken on the HBR Blog Network.  Seiken, head of PBS' digital operations, was able to break through his unit's risk aversion and complacency - which caused growth to stagnate - by mandating that they fail more. Seiken recounted how the team learned how to work to this new objective:

In a beautifully ironic twist, the failure metric itself initially failed. We originally envisioned the metric as a formal KPI in each staffer’s annual performance review. But we soon realized we had created a contradiction: You can’t build a culture that values rapid iteration by simply changing an annual performance cycle. We needed daily reinforcement of the desirability of risk-taking and failing fast.

So instead of spending cycles working with HR to create a KPI measuring lack of failure, we focused on endlessly repeating the “must fail” message.

The change was rapid and profound. Some staff were uncomfortable with the new culture and left. Others began taking risks. The product manager working on our first augmented reality site for PBSKids.org ditched her plans for months of customer research and testing in favor of a 10-week sprint to launch. The site failed. The product manager? She received a spot bonus and her “smart failure” was listed as a top accomplishment in her glowing annual review....

Crucially, we redefined success. When our first foray into web-original video production, a safe, TV-type series called “The Parent Show,” launched to fairly good reviews, we resisted the temptation to declare victory. Instead, the team challenged itself to risk breaking the PBS mold by creating a truly YouTube-native show.

This led to the Mr. Rogers remix, “Garden of Your Mind,” which auto-tuned old clips so Mr. Rogers bursts into song. Within 48 hours, it rose to the top of the most viewed and most shared videos charts on YouTube.

Before the failure metric, the team would have considered a Fred Rogers music video to be risky at best, sacrilege at worst. Instead, the culture change triggered by the failure metric gave the team comfort that even if this blew up in their face, they would be protected.

Ordering people to fail more doesn't work unless they are given the tools to take intelligent risks. What would your "contract for intelligent failure" look like?

Here's an example of the residue of the failure-tolerant culture:

From HBR - how to recover from being fired

This post from John Beeson on the HBR Blog Network has some good advice on bouncing back from losing your job:

You've just received word that you've been fired. Or perhaps the company has gone through a re-structuring and eliminated your job — and you've been told that none of the managers you've worked with over the years have a position for you on their team. This comes as a shock to your system, especially if you've enjoyed a record of success up to this point in your career. While there are some practical things to attend to — negotiating your severance, signing up references, and agreeing with the company on a storyline about the reason for your exit — your most important action item is managing your own attitude to the situation....

As you dust yourself off, think through those parts of the situation you need to own. In a highly emotional state, it's too easy for you to curse the darkness: "I had a bad boss." "The place was rife with organizational politics." "My colleagues were non-cooperative and had it in for me." There may be some truth to this, but you also need to ask yourself, "What do I need to accept about the experience to avoid making the same mistakes so I can succeed in the future?"

Losing your job is a very particular type of failure. It's likely the most emotionally painful event of your career. Analyzing your role in the situation is not natural; you will want instead to lash out and place blame on others (as Beeson writes, this may very well be the case), or to fall into a funk of despair. But to bounce back, better to "own" the outcome and figure out what lessons you can take into the rest of your career.

Malcolm Gladwell discusses something he'll never do again

A tiny but useful mistake story by Malcolm Gladwell, from the disclosure page on his website - a fascinating document in its own right. Here he discusses one of the principles he uses to balance his roles as New Yorker writer and paid public speaker:

Any speaking I do should be just that—speaking. The possibility for trouble is much greater when a writer steps outside the role of giving a set speech and becomes a consultant or advisor, or in some ways develops a continuing financial arrangement with a specific company or organization. Once, I was asked to be a consultant to a marketing firm. I said yes—briefly—then immediately resigned when I realized the implications of that act. I will not do that again. Anything outside of simple speaking is inappropriate.

"I will not do that again" is a bright signal that this is a mistake story that had a significant impact on the teller.

Ashley Good of AdmittingFailure.com discusses learning from failure

This is the keynote presentation Ashley Good gave at FailCon Oslo back in June. Ashley is a fellow traveler who helps companies and non-governmental organizations tell their failure stories so they can learn and improve. Her site AdmittingFailure.com includes stories and discussions of learning from failure. Her talk is a great reinforcement of many points we discuss at this site. Enjoy!



Money quote from a hockey coach to a player who was focused on playing perfectly: "You are not falling enough... practice is for finding your edge." Failure is a way to test our limits and understand what we have to do to get better.

One of Ashley's pieces of work is the annual Failure Report published by the Canadian NGO Engineers Without Borders. You can access its 2012 Failure Report here.

The positive outcome of a product development failure

Karsten Strauss, writing on Forbes.com, discussed the journey of Zach and Max Zitney, the undergraduate creators of MiRing, a Bluetooth-enabled ring that would vibrate when the wearer's phone rang or received a text message.

The product, despite placing third in an Ohio State University entrepreneur's competition, far undershot its goal on crowdfunding site Fundable.com - raising $8000 in pledges versus a goal of $150,000. A failure? Yes, but not an ending.

Max Zitney’s outlook is that the disappointing campaign saved the two a lot of grief because it showed them that the product, in its current configuration, was obviously not sparking a lot of enthusiasm.

But the Zitneys’ story doesn’t end by merely recognizing the silver lining. A UK-based company called NFC Ring, which has developed a similar idea, approached the brothers, offering to inspect their concept and possibly collaborate in the near future....

NFC Ring appreciated the Zitneys’ idea but said the technology is not yet in place to create a ring-size, Bluetooth-enabled vibrating notification device. Those features, however, are ones that the company would like to incorporate into its product . “They said that’s their ultimate goal,” said Zach Zitney, adding that NFC Ring intends to resume talks with the brothers when they’re ready to start integrating MiRing’s designs into their own technology. Given the fact that the Zitneys have a U.S. patent on such a gadget, they may have to.

“There are still options available,” Max Zitney said. “The crowdfunding just opened up the doors for us.”

So: failure, especially cheap and fast failure, does not mean that a project is finished. It may just be a step on the way. The Zitneys did not achieve their funding goal, but the crowdfunding project exposed them to new opportunities and proved to them that going it alone would not be a success - an example of an "intelligent failure."

Hat tip Roxanne Persaud.

Mistake Bank bookshelf: Paul Schoemaker's "Brilliant Mistakes" - mistakes as strategic assets

Kwame recently posted this comment: "No more mistake bank bookshelf? I am a huge fan of your work. supported kickstarter and would like to see more books that focus on mistakes." That was the impetus I needed to restart this segment. I have three mistake books in the reading queue, but for the moment I wanted to reshare a classic in the field, Paul Schoemaker's Brilliant Mistakes: Finding Success on the Far Side of Failure.

Schoemaker's book is a highly strategic look at mistakes. (He and a collaborator, Robert Gunther, originated the concept of "deliberate mistakes," which is to me the most counterintuitive and least-understood concept we've covered here) To Schoemaker, mistakes are tools for understanding the world, learning more quickly, and discovering deep insights.

I've had the opportunity to meet him and his book is a perfect reflection of his keen mind and curious soul. Brilliant Mistakes was my favorite book of 2011, and you should read it too.

From the book: Companies strive for error elimination, hiring advisers and relying on sophisticated management tools such as Six Sigma. It’s little wonder, then, that most decision-making books follow suit, encouraging you to focus narrowly on mistake avoidance today rather than provoking you to plan for the stream of decisions that you will face tomorrow.

Improving large, distributed information systems by inducing failures

I've been thinking about this powerhouse paper from the Association for Computing Machinery's acmqueue site for the last week. It's called "The Antifragile Organization: Embracing Failure to Improve Resilience and Maximize Availability" by Ariel Tseitlin.

Taking its starting point from Nassim Taleb's arguments about antifragility (see references here and here), Tseitlin discusses ways of making large distributed information services antifragile - i.e., able to capitalize and improve themselves in the wake of disruption. His focus is on testing and simulation, the question of how to exercise highly complex systems to ensure they will not collapse as a result of stressors.

As Tseitlin observes, traditional scripted testing is utterly unsuited to this task - in systems of any size, it's impossible to build (or even imagine) the total number of test cases required to prove a system's robustness. Moreover, even the largest test system is a fraction of the size and complexity of the production environment.

Taking a radically different approach, companies like Amazon and Netflix are increasingly causing intentional disruption within their production systems to assess whether their resilience mechanisms are working as expected - and whether new vulnerabilities have emerged. Tseitlin describes several ways that this is done:

Once you have accepted the idea of inducing failure regularly, there are a few choices on how to proceed. One option is GameDays, a set of scheduled exercises where failure is manually introduced or simulated to mirror real-world failure, with the goal of both identifying the results and practicing the response—a fire drill of sorts. Used by the likes of Amazon and Google, GameDays are a great way to induce failure on a regular basis, validate assumptions about system behavior, and improve organizational response.

But what if you want a solution that is more scalable and automated—one that doesn't run once per quarter but rather once per week or even per day? You don't want failure to be a fire drill. You want it to be a nonevent—something that happens all the time in the background so that when a real failure occurs, it will simply blend in without any impact.

One way of achieving this is to engineer failure to occur in the live environment. This is how the idea for "monkeys" (autonomous agents really, but monkeys inspire the imagination) came to Netflix to wreak havoc and induce failure. Later the monkeys were grouped together and labeled the Simian Army.

Netflix's "monkeys" include, among others, a Chaos Monkey, which randomly terminates virtual instances in the production environment; and a Latency Monkey, which inserts delays into various components of the network.

These are run regularly and the Netflix team carefully measures to see whether the system adapts suitably to the disruption. The random and low-level nature of these tests helps avoid limits of human-scripted testing, and the fact they are running in the production environment means they are not limited by the size of a test environment.

Is it risky? Not if the system has been engineered not to collapse under stressors. Top quality distributed systems are built to isolate failures and degrade gracefully rather than result in catastrophic downtime. As such, it does require a system that has been in production long enough to develop stability - Twitter in its first two years would not be a good candidate for this, but Twitter at present would be.

Tseitlin concludes the paper by discussing how these exercises in resilience are building toward true antifragility, by using tools such as post-exercise blameless postmortems and requiring that developers also be operators, the better to anticipate code that might cause operational issues down the line.

Richard Diebenkorn: "Mistakes can't be erased but..."

Great quote from painter Richard Diebenkorn as quoted by Maria Popova in Brain Pickings. From Diebenkorn's handwritten "Notes to myself on beginning a painting":

Mistakes can't be erased but they move you from your present position.

Marilyn vos Savant: Cheating "makes it appear you are more proficient than you really are"

Marilyn vos Savant's advice column in Parade Magazine has been a guilty pleasure of mine for years. But today's Q+A seemed apt for our site, and has echoes of an earlier post of ours, "No Mulligans."

Question: Moralizing aside, can you give me any persuasive reason why my friends and I shouldn’t cheat at school whenever we’re sure we’re not going to get caught?

Marilyn: Aside from what may happen if you get caught anyway? Possibly the best reason not to cheat is that doing so may make it appear that you are more proficient in a subject than you actually are. This will lead you into increasingly deeper water without knowing how to swim well enough. As you move into academic areas or courses that are over your head (because you didn’t master the foundations for whatever reason, such as not studying as much as you should have), you will find yourself in academic trouble that no amount of cheating can solve.

Idea to capitalize on mistakes and failure: a "brilliant mistakes" contest

I recently had the opportunity to spend some time with Paul Schoemaker, author of the book "Brilliant Mistakes: Finding Success on the Far Side of Failure" as well as co-author of the HBR Article that was one of the inspirations for this site ("The Wisdom of Deliberate Mistakes," with Robert Gunther). In our discussion I was reminded that, as part of the events surrounding the publication of "Brilliant Mistakes," the Wharton Digital Press created a Brilliant Mistakes contest, which asked entrants to describe "an error that they committed and recognized as a potential source of learning or innovation, and then leveraged as a new concept or idea that helped transform a project or organization."

The winning entry came from Dr. Stephen Salzman of UCLA Medical School, who did a cardiology experiment on athletes and found that his hypothesis was 180 degrees wrong, "forever changing his perspective, not only in the area of adrenaline, but throughout cardiology."

What do you think? Would contests like this within companies help them capitalize on mistakes and failure better? Why or why not?

VC Ed Sim: "Never give up but move on quickly"

The title is from a 2011 post from Ed's blog Beyond VC. At first (and maybe second) glance this is a paradox. At the heart of this paradox is one of the deeper secrets for learning from mistakes. Let's deconstruct it:

"Never give up..." This is a mantra repeated over and over again on this site, including Garr Reynolds's "Fall down 7 times, get up 8" and Angela Duckworth's discussions of grit. Persistence is essential to power past stumbles, failures, criticism, ridicule - all conditions that come before worthwhile successes.

...but move on quickly" - this refers to the ability to maturely sniff out a situation that is heading for failure, and not devote endless work cycles to trying to pull a success from the jaws of failure. Overall success requires diagnosing, with evidence, counsel and your own intuition, the smaller failures that come up along the way, and shifting your approach quickly after the diagnosis comes in.

"Never give up but move on quickly" is one of the Scylla-Charybdis balancing acts (per Albert Wenger) necessary to be a good entrepreneur or really senior leader of any kind.

20 tips to deal with failure from Leadership Freak

There's nothing in this Leadership Freak post ("When Projects Fail") that you wouldn't know if you had read all the 500 or so posts on this site. On the other hand, if you don't have time to read 500 posts today, Dan Rockwell does a fantastic job of summarizing 20 lessons to use when that project goes awry. I like these tips in particular:

6. Step up to excellence not down to mediocrity. Excuses are the path to ease, insignificance, and irrelevance. “How can we be better?”

7. Pick the scab. Dig into issues optimistically and respectfully. The goal is improving next time, not blaming and putting down.

15. Believe improvement is possible.

16. Don’t assume working harder will make things better. What will you do differently, next time?

18. Explain how you’ll do better next time. Be specific. Clarity is the mother of success.

19. Put it to bed.

20. Re-vitalize momentum. Examine what happened with what’s next in mind. Create a win and celebrate.

The lessons show the balancing act we perform when managing failure. Care, but not too much. Be brutally realistic, but don't lose hope. Learn your lessons, but move on.

If it were easy, it wouldn't be that interesting to study, would it?

VC Mark Suster recalls a painful sales loss, and what he learned from it

This is the second story from Mark Suster's 2010 post "Embrace Learning: It Will Make You Stronger." Suster recalls how one of his startup companies, a collaboration software maker, lost a deal it thought was all wrapped up. I a similar experience almost 20 years ago and many of the lessons are exactly the same.

In the 2003/04 timeframe I was living in the UK and running my first company. I had been competing to win a contract at Thames Water, the largest water company in the UK. They were looking for a collaboration tool to manage all of their large water development projects. The initial contract was worth about $500,000 and the whole value of the contract would have been worth a couple of million over the years. I was working hand-in-hand with my close friend and associate Stuart Lander who was running our UK office and with one of our local sales reps.

We had initially been told that we had no chance because they had previously purchased Documentum and it would mean changing the system entirely. They had a team trained up in Documentum and we certainly had enemies from the inside. But we worked the account tirelessly for months. We helped the write out their requirements for a system. We met everybody in the organization. We had every reference client we worked with call their senior team members (we had already won a major project at Scottish Water, Anglian Water and another at a large water company in Paris, France).

There were about 8 initial contenders for the work and in the end it came down to just 3 of us. As the founder & CEO I personally went and met with as many people at Thames Water as I could. We felt this was a marquee account and one that would help us take our collaboration tool global as Thames owned assets all over the world. Winning the contract meant that we would hit our quarterly revenue figure and be in good position for our annual sales target.

And then the news came. A woman named Trish Hannon called me with the good news that we had won the project. I was to tell noone until the contract was signed. I immediately put a full team on contract management and another on drawing up an implementation plan.

DefeatI later learned one of my biggest lesson in sales. You are most vulnerable right after you have won a deal. It is when you’re competitors have nothing to lose. It’s when the people who are part of the decision making process who don’t support the decision seek ways to undermine you. That is when you potentially become complacent.

Two weeks after winning the deal and well into implementation planning we released a new version of our software. We had made the decision that we would no longer be supporting IE v 5.5 (we would support 6.0 and 7.0, which was in beta). Even Microsoft publicly said that there were security flaws with 5.5 and that people should upgrade. But Thames Water was still on version 5.5. We assumed they would take our advice and upgrade. We had discussed this with Trish.

An internal resource inside Thames Water used our upgrade and lack of 5.5 support as a way to re-open the decision. How could a company like ours be so callous as not to support their software (even one more than 4 years out of date)? Did we really have good change management procedures if we were willing to launch products without backward compatibility. And so on. They decided to re-open the competition for 3-4 more weeks. I knew THEN that we had lost. We fought hard to stay in the game.

They hired a consultant to help them with the review. They stopped allowing us to contact them directly. The momentum had shifted. Something happened and it was clear to me that this IE issues was just a smoke screen. Somebody had gotten to somebody senior in the Thames organization. It just so happened that the consultant they hired to chose a software vendor worked for a company that had owned one of our competitors. It was a tiny little collaboration company that only had a presence in the UK (and therefore couldn’t meet their international needs). And surprise, surprise the decision came back 3 weeks later than none of the preferred 3 vendors had won but rather this tiny little competitor owned by the consulting company charged with doing the review.

Outrage. Scandal. Surely inside Thames they would see it for what it was. Given that it was a public tender the chairman of our board had encouraged us to think about launching a complaint with the UK government agency in charge of such reviews. We talked with lawyers. We felt totally deflated. We decided it wasn’t worth the fight. We licked our wounds and moved on.

I am still not over that loss. I sometimes call Stuart and we recount what happened. But that loss was really important in my career. It taught me a lot of lessons. We spent enough time dissecting it to really learn.

Here are my take aways from the loss:

1. In a sales campaign you always need to call as high as you can. If you don’t, your competitors will. If you don’t know them, somebody else does. They may not overturn decision, but they sometimes do.

2. No deal is ever done until the ink is dry and the money is in your bank account. Never take your win for granted.

3. You are most vulnerable right after it has been announced that you won (I will write a separate post on this). This is the most important lesson I learned from this experience. All other lessons were sort of obvious. This one was eye-opening.

4. We in the tech world extol the virtues of lots of product releases and rapid innovation. I hear Silicon Valley firms bragging all the time about how often they release software. In the consumer world, maybe. In the corporate world this strategy is flawed. Many large clients prefer stable technology and no changes – even sometimes when there are known security flaws. When I was at Salesforce we launched a new version of our UI. We gave users the choice to upgrade or keep with the classic UI. Years later 10% of users were still on classic. Go figure.

5. In every deal where you have serious competitors there is always somebody on the inside against you. You need to find out who that is and neutralize them.

6. No matter how much your customer tells you that they love you and that they favor you it is possible they are telling other people the same thing or some variation of this. (you would have thought I would have learned this lesson in high school ;-)

7. No matter how much large clients tell you they want transparency in pricing, they always seem to fall for the same old trick. Competitors price low to get in the door and then nail them with scope control, change orders, product extension costs and other hidden items. You can try to convince them of your “pay no more once you’ve signed up” model but they fall for the other guy’s pitch every time. Low numbers are sexy. I stopped trying to win this argument and chalk it up as some sort of human condition that I can’t change (like thinking that $14.99 = $14).

8. Time is the enemy of all deals. If you have a chance to close something – don’t let it drift.

9. Losing sucks. But at least it has made me a better competitor.

"You can't build a web system that will never break"

From a post on Fred Wilson's AVC blog:

Once you have a successful product in the market, you need to turn your attention to scaling it. The system you and your team built will break if you don't keep tweaking it as demand grows. Greg Pass, who was VP Engineering at Twitter during the period where Twitter really scaled, talks about instrumenting your service so you can see when its reaching a breaking point, and then fixing the bottleneck before the system breaks. He taught me that you can't build something that will never break. You have to constantly be rebuilding parts of the system and you need to have the data and processes to know which parts to focus on at what time.

VC Mark Suster learns from a failed pursuit of a venture investment

Mark Suster's "Both Sides of the Table" blog is a great resource for startups and managers. I found a 2010 post in which he outlined the positive value of losing - its ability to teach deep lessons. Here's one of the two stories he tells in the post:

Last year I lost a deal in a company that I wanted to invest in and that I thought I should have won. I was angry – mostly at myself. Rather than blame the team that I thought should have chosen me, I became reflective. They were in LA and I was in LA. They had a prominent NorCal investor already so I thought a SoCal lead would make sense – that I could help them in a more hands-on way. They had agreed!

I had lost a previous deal where the team said they liked me but didn’t know my partners well enough so I promised myself never to let that happen again.

So I organized a team dinner with all four of my partners and all three of their founders. I wanted to be sure that they knew how much all of our partners loved what they were doing with their company. I wanted to be sure that they felt they knew all of my partners well so they could see why I joined up with them in the first place – they are smart guys who have a 20-year track record of winning. 15 companies North of $1 billion exit. And they are normal, down-to-earth people as well.

After dinner on a Thursday night I thought we had the deal and that the team knew how hard I would work on their behalf if I were chosen. By Monday morning after their board meeting in NorCal I didn’t get a return phone call. I knew what this meant. Good news always comes quickly, bad news takes time to simmer. By the time I got through to the guys on Tuesday we had lost.

I knew that the, “I’m really sorry” message was coming. I embraced it with honor and didn’t give them a hard time. But I obviously asked, “Why did it happen? I need to learn for next time.”

There were two main reasons that I could distill from their kind words of solace: 1) the existing NorCal investor didn’t know me well enough & 2) the new NorCal investor had a good knowledge of and presence in China, which they believed would be critical.

I decided to put both of those issues to bed in 2010. I came several times to NorCal (where I grew up, actually) and went and met several partners from each Silicon Valley firm. I didn’t want this to happen again – that people didn’t know me. I also made several trips to New York & Boston. Next year I’m going to spend time in Seattle and Boulder in addition. I realized that it is not enough to know one partner per firm and it is not enough for only the management team to like you. VCs have a seat at the table in deciding future investors.

I also spent two weeks in China and vowed to make it back frequently. China is indelibly an important part of the future of the global technology system. Although I had lived and worked in more than 10 countries – it wasn’t good enough. I didn’t know the one that mattered most to their future. And for my own good I vowed to have relationships in China and knowledge of the local markets. I’m not looking to invest there – I’m looking to understand the trends, the people, the innovation, the regions and how China can become an integral part of any of my portfolio companies as they scale.