JEFF GOTHELF: Good morning.
Thank you guys for being here.
What I want to talk about–we have an hour together– actually, a little less.
We're starting afew minutes late.
So I want to get right to it.
What we're goingto talk about today is how to build successfulteams inside digital product organizations likeGoogle and others, and how to do so focusingreally on a variety of different things, includingevidence-based decision making, customer centricity,and really great design.
And to start off, I don't knowhow much you know about me, but I want to share aquick personal story to start off the conversation.
19 years ago, I graduatedfrom undergraduate school, and my first jobwas with the circus.
I ran away.
I graduated on Saturday.
Monday morning Ijoined the circus, and spent the next six months onthe road with this circus right here– the Clyde BeattyCole Brothers Circus, which at the time was the largestthree-ring tented circus.
I spent six months onthe road with them, traveling up and down.
It was an I-95 Circusis what they called it.
And I saw the circus.
We worked every day– no daysoff– two shows every day, three shows on Saturday.
So over the courseof six months, I saw the circus 400times in a row– note for note, word for word.
And needless to say,my children will never be going to the circus,at least not with me.
The interesting part, too, isI was driving home actually the other day.
I was in the city.
I live in New Jersey.
And there it was.
I hadn't seen it inprobably in 19 years.
And it was set up on Route 80just outside of New Jersey.
And it just looks alittle, slightly different, but it smelled the same.
I went by there to see ifanybody I recognized was there.
And over the course of the timethat I spent in the circus, I met a lot ofinteresting people.
One of the interesting peoplethat I met during that time was Steven Tyler.
That's Steven Tyler right there.
And that's actually menext to him 19 years ago.
Time's been a little rough tomy hairline in that time frame.
That was really interesting.
He would come to the circus,and we all got to meet him.
He's done well.
He's preserved himself ratherwell over the last 20 years.
He's got that secret formula.
Another reallyinteresting person that I got to meet whileI was there was this guy.
That's the Human Cannonball.
And in case youcan't see him, that's him at the top beingloaded into the cannon.
And in the circus, therewas a lot of downtime.
We worked two shows everyday, but once the last show was finished, youdidn't have anything to do until the next daywhen the next show started.
And so there's alot of downtime, really no internet in 1995,certainly not mobile internet.
And so you get to talking.
And one of the questionsthat I found fascinating and I wanted to askwas, how does one become the Human Cannonball? What's the careerpath to ending up in this particularposition at some point? So I asked him, andhe told me the story.
And the story that hetold me is fascinating.
And as these storiestend to do, they start with the previousHuman Cannonball.
That's how thesestories have to start.
And so the previousHuman Cannonball had this job before this guy.
And the way thatthis trick works is it's not a real cannonin the sense of there's no fire or anything.
There's a spring-loadedmechanism in there.
The guy slides down.
The Ringmaster hits a button.
There's a puff of smoke,and then he shoots out.
It pushes him out ofthe cannon, and he lands on the other side of thetent in a net of some kind.
Now the way that theyknew where to set up the net with the previousHuman Cannonball was they had a dummy– a mannequinthat weighed the same as the previousHuman Cannonball.
And what they woulddo is, they would drive this redtruck into the tent.
They would aim the cannon.
They would loadthe mannequin in.
They would fire it, and they'dkind of see where it landed.
And they would putthe net up there.
And we did thatevery other night, because we changedvenues every two days.
One night, we were runninglate– or the previous Human Cannonball– I wasn't there thatyear– they were running late.
And they got to thenext place late, and they didn't have time totest where to put the net up.
And so they got the truck inthe tent, and they aimed it, but they left thedummy outside overnight and it rained that night.
And the next day,assuming everything was exactly the same, theprevious Human Cannonball and his team loaded thedummy into the cannon.
They fired it.
They saw where it landed.
They put the net up.
And that afternoon, infront of 4,000 children, the previous HumanCannonball waved goodbye to all those kids,loaded himself into the cannon.
They fired him off, andhe sails way past the net.
All right, tragic story.
He doesn't die.
He ends up severelyinjured, as you could imagine,from such a flight.
And he goes back toFlorida where all circuses come from– in caseyou didn't know that.
That's where they're based.
But that's alsowhere they come from.
And he sees his pool guy.
His pool guy is this guy.
And he says, hey, would youlike a job upgrade from pool guy to Human Cannonball? You only have to workfour minutes a day, and here are the risks, though.
And the guy took thegig, and so that's how this guy ended upas the Human Cannonball.
Why am I telling you thisstory, besides it just kind of being a fun story totell and a fascinating look into this weird subculture thatI was a part of very briefly? The reason is this– theHuman Cannonball's team– the previous HumanCannonball's team– made the same assumptions every day.
Everything's exactlythe same, so we should work exactlythe same way, and assuming that theresults that we get will be exactly the same.
But as it turns out, theassumptions that they did make were wrong on that one day,with very tragic consequences.
And the same thing is truein the world of software these days.
The way that we'vebuilt software in the past, and the way thatwe're building software today, and where it'sheaded are changing.
And the assumptions thatwe're basing the way that we build digitalproducts don't work in today's realities in theway that we're moving forward.
And the reasonfor that– and you guys should know thisreally well here, obviously, is that software's eating theworld– every business that's of scale in the 21st century,every business that seeks to scale in the 21stcentury, at its core is a software business.
It's a digital business.
And this became reallyevident recently with the leaked New YorkTimes Innovation Report.
I don't know if you guyshad a chance to read this, but this is afascinating document.
It's an internal auditthat "The New York Times" ran for six months to seewhy Buzzfeed and the Verge and Vox and Vice Mediawere eating their lunch.
And at the heart ofthis– at the root of all this 90-pagedocument– was the fact that "The New YorkTimes" saw itself as a journalisticorganization that happened to deliver contentthrough a digital channel, amongst other channels.
Whereas the recommendationwas to flip that on its head, and they needed to think ofthemselves as a digital company that happened to doreally great journalism.
And that's reallywhere the disconnect is for this company, anda lot of the companies that I work for.
And we're seeing thisacross the board.
We're seeing it atFedEx, for example.
What business is FedEx in? You could say it's logistics,but in reality, it's the software of logistics.
It's the running of logistics.
And I think in yearspast, we've seen this.
As software comes into playinto existing industries, the typical reactionfrom those industries is to push back, right? It's to say, this is not theway that we'd like to work.
Software is notgoing to disrupt us.
And there's no waythat we're going to sell our music inthis particular way, or use thisparticular technology.
In case you're curious, that'sLars Ulrich from Metallica, who was reallyloud, and very, very present in the fightagainst Napster when Napster first came out.
There was no waythat he was going to allow Metallica'smusic to be sold that way.
Surprisingly enough, Metallicais now available on Spotify.
So software inevitablyeats the world.
We're seeing this again inthe world of cars, with Tesla.
Tesla's trying to selldirectly to consumers.
And what is the automotivedealership lobby trying to do? Litigate them out of existence.
They're suing themover and over again, so that they can'tbypass the dealership and change the way thatwe sell, and upgrade, and service our cars.
Netflix is anothergreat example, right? No one ever thoughtwhen Netflix came around that we'd re-think theway that we consume media, much less produce it.
And yet today, try tofind a Blockbuster that's still in business, right? Now what's interestingis that when you're making productsin the physical world, and you've gotthis assembly-line model of production,it works well when you know exactly what theend state is going to be, how it's going to be used, andwhat it's going to look like.
And that's great if you'remaking cars, or TVs, or even cell phones.
But what we've done iswe've taken this model, and we've appliedit in many companies directly to software,where there's become this kind ofassembly-line model of software development.
Someone has to define it, andthen someone has to design it, and then someone has to developit, and test it, and deploy it.
And what's interestingis that in software, we don't know what the endstate is going to be.
In fact, I wouldargue that there is no end state to software.
And I would alsoargue that we really don't know exactly howour customers are going to use the things thatwe put in front of them.
And so using this modelstarts to break down.
And it starts to break down forthis reason– I think again, you guys know this betterthan most companies– that software hasbecome continuous.
There is no end state.
We just simply keep improvingit, optimizing it, iterating it, moving it forward.
There is no finished product.
There is no big release.
There's justcontinuous deployment and continuous releaseto our customers.
And then ultimately, from thatcomes continuous learning.
What I think it'sfascinating is this– and I'm curious howthis compares here– this is how often Amazonpushes code to production.
Some customersomewhere sees new bits every 11.
6 seconds at Amazon.
That's staggering.
That's amazing, right? Five times a minute, theyare taking tiny risks.
They're trying new things,and then they're listening.
They're sitting back,and they're listening.
They're sensing theresponse from the market.
Did they like it? Did it change their behavior? Did it get them to dosomething differently? If it did, terrific.
Let's scale that out.
Let's optimize it.
Let's design it better.
Let's make it easier to use.
And if it didn't, let'slearn why and roll it back.
Now the impact, of course,is that the learning is huge.
If you get it wrong and you rollit back, the failure is tiny.
And so taking thesetiny risks allows them to develop this sensinglayer with their customers.
They're continuallyhaving a conversation with their customers,so that they're able to sense veryquickly, and then in theory, they canrespond equally as quickly.
This is working well.
Let's scale it out.
This is not working well.
Let's find out why, andthen let's try it again a different way.
Now the interesting thing isthat insight has two sides.
We can't simply justmeasure, and assume that we know why something isworking or not working, right? That's the quantitativeside of things.
Insight also has aqualitative side.
We have to understand why peopleare actually behaving this way.
What are they doing isthe quantitative side, and why they're doing itis the qualitative side.
And together, we canbuild a 360-degree view that allows us to reactvery, very quickly.
By comparison, this is where Iused to work a dozen years ago.
Our feedback cycle–just to give you a sense– Amazon gets feedback,let's say five times a minute, potentially.
Our feedback loop at AOL12 years ago was 12 months.
We would work for sixmonths to get every bit of code and designperfect, because we had to print 20million of these, and then send them outto people in the mail.
And that's how peopleconsumed software.
So six months of work to getit done, ship it in the mail, wait for people to installit, see how they use it, six months to collect data,and then to ship again.
Right? It's a 12-month-feedbackloop, whereas today, five times a minute youcan get that feedback.
Back then, it madesense to try to get it as perfect as possible,because again, you're making 20 million copiesof this particular disc.
Today we can update thatmuch more efficiently.
And because of that, theseindustrial-era management tactics don't work withcontinuous software.
We can't manage an assemblyline and optimize a production process, becausewe don't know what that end state needs to be.
We have to continuallyfigure it out, sense it, and respond to it.
And so as makers ofdigital products, as makers of software workingin this continuous world, there's a few questions that weneed to answer for ourselves.
The first is this-.
What does this continuousworld mean for the way that we build productsand organizations? It's not enough to just changethe way that we build products.
We have to build thecompanies and the teams around those products thatcan work in this reality, and that can do their bestwithin this new world.
Right? The second question is this.
How can we take advantageof all this information? We've got all thisinformation coming in, quantitative data afterquantitative data, assuming you're doing theresearch underneath it as well.
You've got qualitativeand quantitative.
How do we actually build ateam and an organization that can take advantageof that information? And then lastly, how canwe maximize the creativity, the learning, andthen ultimately, the productivity ofour teams in an effort to do their best possiblework, to build the best possible productsfor our customers? And so in orderto do so, in order to survive in thiscontinuous world, we have to build aculture of learning.
That's the key here.
We have to change the mindset.
It's not enough tojust deliver software.
We have to actually deliverlearning on a continual basis.
That's what theAmazon teams get to do with 11.
6 seconds of release.
They're building aculture of learning.
My friend Dave Graywrote a book about a year or two ago called "TheConnected Company," and it's a terrific book, andI highly recommend you read it.
Now in that book, Dave talksabout two different types of corporate strategy.
The first isdeliberate strategy.
Deliberate strategy comesfrom the industrial-era mode of thinking, where it'sthe executive suite that has all the ideas.
That's where the creativity is.
We'll come up with everythingthat you need to do, and then we'll giveit to you guys, and you can goforth and execute.
And that's the reality.
And what happens in thatis really interesting.
I'll show it to you in thisreally short video that I've got in here, aboutwhat actually happens when you take deliberatestrategy to an extreme.
And that's Johnny Depp as WillyWonka illustrating my point for me.
[VIDEO PLAYBACK] [WHIRRING] [DING] -Do you mean that's it? -Do you even know what it is? -It's gum.
-Yeah.
It's a stick of the mostamazing and sensational gum in the whole universe.
Know why? Know why? Because this gum is afull, three-course dinner all by itself.
-Why would anyone want that? -It will be the end of allkitchens and all cooking.
Just a little strip of Wonka'sMagic Chewing Gum, and that is all you will ever need atbreakfast, lunch, and dinner.
This piece of gum happens tobe tomato soup, roast beef, and blueberry pie.
[END PLAYBACK] JEFF GOTHELF: In mostcases, deliberate strategy yields a situation where you endup with a customer asking you, why would anyone ever want that? Which is the worstpossible thing you could hear whenyou actually put your product incustomers' hands.
The second worstthing you could do is exactly what hedoes there, which is fall back onyour marketing notes to justify all theassumptions that went into making that product.
It will be the end of allcooking and all kitchens, as if people don'twant to cook, and don't want to spend timein their kitchens.
And again, at thebeginning when you set out to solve a businessproblem or a customer need, we have no idea what that endstate is going to look like.
I think this is aperfectly good example.
You could walk past thesetwo hippies 40 years ago, and never think, never knowwhat the end state was actually going to be for these twoparticular individuals.
Right? And again, that's the same thingwith the products that we make.
And so what's the alternative? The alternative, of course,is emergent strategy.
It's different and it's organic.
It lets companies learnand continually develop new strategies based on thisongoing culture of hypothesis and experimentation.
It takes a bottom-upapproach that says, the people who are closest tothe customer, the people who are making theproduct, probably have the best ideas abouthow to solve this.
We need to build a team thatallows them to try this out.
And we need to build anecosystem around them that allows them to run theseexperiments while not crashing the system as a whole, so thatwe can keep doing business.
That's where that emergentstrategy comes from.
And again, we hear this froma variety of executives– Amazon– you need to set upand organize so that you can do as many experiments perunit of time as possible.
Eric Schmidt calls them at-bats.
He's talked about itat Google, about how we've got to take as manyswings as possible at something to make sure thatwe get it right.
And so in order tobuild a culture that supports this way ofthinking, of learning, of experimentation,we have to think about the structure for it.
And the structurefor it is the team.
The atomic object of a cultureof learning is the team.
Right? That's the smalleststep we want to go.
We don't have aparticular rock star that can answer allof our questions.
We want to build these teams.
And the question becomes, then,how do we build these teams? What makes up aninnovative team, and a team that can take advantage ofthis culture of learning? And we'll talk aboutthree different things.
The first is theanatomy of the team.
What's the makeup of the team? The second is, howdo we task the team? How do we tell theteam what to do? And then lastly, howshould the team work? Should they work inan agile process, in a waterfall process,a lean process, whatever you want to call it.
OK? Let's start with theanatomy of the team.
What is the makeupof the team itself? And to do so, I'm going to startwith a series of anti-patterns, how not to buildinnovative teams.
The first is this.
Putting people in silos breaksup the communication cycle.
It isolates teammembers, and forces them to communicate withwritten documentation.
Everybody feelslike a line worker.
I'm going to do my thing, andhand it off to the next person.
And because of that, no one hasownership of the whole thing.
Everybody feels likea service provider.
Right? It just passesthrough my department.
I add my coat of paint on myassembly line, and it moves on.
No one has a sense of the whole.
What are we building? Who are we building it for? What's the problem thatwe're trying to solve? What does successactually look like? How does this fit intocorporate strategy? Why are we even working on this? And worse, there's nocollaboration that takes place.
And collaboration really isthe secret sauce of innovation, right? That's where learning comes.
That's where innovationcomes from, the building off of each other's ideas.
And I know this firsthand.
Actually, just downthe street from here, I used to run a UX team at acompany called The Ladders.
I worked there for aboutfour years starting in 2008.
And my job was to come in thereand build a design practice.
And the first thing thatI did was I set up a silo.
That was our silo.
And then every time somebodyneeded work from us, they would come to us.
And I played the trafficcop in the center.
Every time– we need some designwork– I provided some insight.
Product management said,hey, we need some work.
I said terrific.
Bob's got some bandwidth.
Let's give it to Bob.
Engineering said, we've got alittle bit of work over here.
Terrific, Bob's gotsome more bandwidth.
Let's give it to Bob.
Marketing says, hey, weneed the website spruced up.
Terrific, Bob's got afraction of bandwidth left.
Let's give it to Bob.
Now Bob's supporting threedifferent projects a day, and every day he's comingin deciding which two people he's going to piss off today.
Right? Because you can't havethree number one priorities.
You've got to pick onething and move on to it.
As soon as you build that silo,people do just enough work to move on to the nextthing, and the next thing, and the next thing.
And just because designis in the center here, doesn't mean that thisis unique to design.
Engineering could be the centerhere, or product management, or marketing, and so forth.
Right? We've got to get peopledistributed differently.
And so the questionthen becomes, what team structurefacilitates this culture of innovation and learning? Four qualities.
The first is this.
Small.
Keep your teams small, sixto eight people, roughly.
Again, I quote Jeff Bezos a lot.
I like what he's doing.
What he calls a two-pizza team,a two-American pizza team.
It's worth quantifying.
That if you can't feed your teamwith two pizzas, it's too big, right? Six to eight people.
That way you knowwho everybody is.
You know who to go towhen you need something, and there's amazingaccountability.
There's nowhere to hide.
If there's six people ona team and someone's not doing their work,you know very well who's not doing theirwork right away.
Second, take those six to eightpeople and co-locate them.
Put them together in oneplace, not in the same campus, but in the same area so they'resitting next to each other, so they can talk toeach other, so that they can reduce the feedback loopfrom each other as quickly as possible.
And the reality is,I know for you guys, you've got multiple offices.
Where I work, we'vegot multiple offices.
If you have to workwith a distributed team and you want to buildthis hypothesis-based, experiment-driven approachto building products, you at least have to beawake at the same time.
You've got to at least havesome overlap during the day.
I have colleagues in Singapore.
They're fantastic people.
I never get to work with them.
It's a 12-hour time differencebetween here and there.
So we've got small, co-located,and next– dedicated.
Everybody's working onone project, this project specifically, right? That's what we wantpeople working on.
As soon as we steal somebody offto work to fix a bug somewhere, or to work on anexecutive pet project, they're not workingon our project.
And we're a small team, andso we're waiting for them to come back, and we can'tmove until they come back.
And the contextswitching is costly.
It takes time to rampdown and to ramp up to each one of these ideas.
Small, co-located, dedicated,and lastly, self sufficient.
The team should be able todo whatever it needs to do.
If it needs to write code,someone can write code.
If it needs to design,someone can design.
Research, someone can research.
Product managementdecisions, somebody needs to be there to make that.
It doesn't necessarily meanthat you need representatives from all of those disciplines.
You just need thecompetencies on the team, so that the team can makethose decisions at the pace that the information comes in.
Remember, we're seeinginsight and information come in super fast.
The team has to be capable andempowered to react to that.
So those are the fourqualities of a team that values learning and innovation.
So then the questionbecomes, how do we tell the team what to do? How do we task the team? This is certainly aviable option up here.
You just yell atthem for awhile.
And I've had bosses whohave tried this strategy.
It works temporarily, untilsomeone quits and finds a better job.
But in reality, what aresome ways to task a team? And once again, I'llshare some anti-patterns.
My first and myfavorite anti-pattern is the roadmap, theproduct roadmap.
I don't know if youguys make those or not.
I've made a lot ofthem in my life.
And look, they'recompelling documents.
Right? They tell a great story.
We're here, and we'regoing over there, and there are five stepsbetween here and there.
And this is what's betweenus and the deadline.
And this is when we'll launch.
The end comes atthis particular date.
And so we convey thisidea that we fixed scope, and that we fixedthe deadline as well.
Now I don't know aboutyou, but in the 17 years that I've been buildingsoftware products, every time we fixed scope andfixed time, one of three things has happened.
We've moved the deadline,we've reduced scope, or we've worked in crunchmode for the last three weeks of the project– 80 hourweeks– to get it done.
And then we allresign at the end and go find jobs somewhere else.
In reality, roadmaps look like this, because again, there's somuch unknown and so much complexity that goesinto software development that we simply can'tpredict exactly where these challengeswill come up.
Something that was easyturns out to be hard, right? There's a new technologythat's entered the marketplace, and it's radicallyshifting the way that we actuallydeliver our service.
We have to be able tobounce around like this.
And that's the reality.
We can't simply predictexactly when we'll be done, and exactly what it will looklike when we get done with it.
This is how Gilt Group doesit from a recent blog post.
They say they don'tmaintain a roadmap document.
Their road map is simply thesum of active initiatives.
Initiatives are business goalsthat they're trying to achieve.
And every so often, theyrevisit the prioritization of those business goals.
Are we making progress? Is it still worth itfor us to continue pushing in thisparticular direction? Right? No specifics about exactlywhat will launch and when.
And we've seen this fromthe authors of "The Agile Manifesto.
" This is one of them.
This is Kent Beck and hetweeted this a while back.
And he said, look,product roadmaps should be lists of questions,not lists of features.
Because when you havethese product run-ups that are lists of features, we'reincentivizing our teams to deliver output,to deliver features.
And when we delivernothing but features, we end up with product bloat.
Look, this is the MicrosoftWord version of a guitar, right here.
Right? 95% of this guitaris useless to 95% of the people who actuallypick this thing up.
This thing was specificallycreated for Pat Metheny, and really only hecan really do justice to the entirety of this product.
The rest of us could probablymake out a few of the notes on that long neckright in there.
But this is what weget in product suite after product suite, when weincentivize our teams to launch features, to generate output.
And that's what roadmaps do.
They drive us there, right? At the end of the day,shipping a product is not a measure of success.
Too soon? Sorry.
Simply launching the productis not a measure of success.
The reality is, did wechange customer behavior in a positive way? Again, as we pushfeatures in, features out, all we end up with isfeature bloat, and products that should do things very,very simply and very cleanly.
Now interestingly enough,most companies currently manage to output to features,to getting out the door, because it's easy.
It's binary.
Did you ship it? Yes we did.
Terrific.
You get to keep your job,get a raise, get a bonus, whatever it is.
Did you not ship it? Terrific.
You get fired or reprimandedor whatever it is.
Instead, we shouldbe tasking our teams to achieve business outcomes.
Business outcomes area measurable change in customer behavior.
It's an objectivemeasure of success.
Did we get customersto click on more ads? Did we get customersto come back more often to the Search Results page? Did we get them tocome back less often, because we're givingthem the right results? Those are measurable changesin customer behavior.
And that's how weshould task our teams.
We give them a problem to solve,not a solution to implement.
Now the interestingpart is this gets messy.
It's a big can of worms.
It's not a binary thing.
If you task your teamsto get more people to sign in successfullyinto your product by 50%, and over the courseof six months, they increase it by32%, did they fail? Did they succeed? It's not as clear.
It's messy, so it's moredifficult to manage, and many companiesstay away from this.
Instead, we haveto get granular.
We have to give teams theseoutcomes that they can directly attribute to the workthat they're doing.
Right? Decrease shopping cartabandonment by 15%.
Did the work that wedid actually do that? Right? Increase the number of timesa customer visits each month.
That's the kind ofsuccess criteria we want to give people, andthen let the teams figure out how to actuallyachieve that goal.
Interestingly enough, I was ona project over at The Ladders just like this, justto give you an example.
I was tasked withmoving a metric called Job Seeker Response Rate.
Job Seeker ResponseRate was the number of times a job seeker respondedto an employer contact through our system.
The team that we put togetherwas a small, co-located, cross-functional team, and theirtask was to move this number– which was our current Job SeekerResponse Rate, 14%– to 75%.
That was it.
No further instructionswere given to that team.
Your job is to getthat number to 75%.
Go figure it out.
Right? And so we got together.
And we brainstormed.
We did all the things that makeup brainstorming and innovation activities– Post-Itnotes, and white boards, and all of those things.
And we tried a lot ofthings, and we experimented.
And at the end ofthree months, we had managed to getthat number up to 38%.
And we went back up infront of the organization, and we said look, youfunded us for three months.
We've moved thenumber from 14 to 38%.
Here's what we've learned.
If you fund us again foranother three months, this is what we're going to do.
And the organizationsaid, terrific, go ahead.
Do more Post-Itnotes, which we did.
We actually shipped code,believe it or not– features, and designs, and so forth.
And at the end of sixmonths, we got to 68%.
We didn't hit 75.
We hit 68.
The company said, you know what? That's good enough.
It's close enough.
We are going to moveyou to the next project.
And we built whatever featuresmoved the needle appropriately, and could scalewith the business that we were moving forth.
And what that allowedus to do, by having those objectivemeasures of success, we could make decisions based onobjectivity, on observing what and why customers are doing.
And when you do that, youcollect a bunch of data.
And when you've gotenough information, you've got threedecisions to make, right? The first, if the data says,hey, this is a bad idea, you should kill this idea.
This is a bad idea.
Don't do this.
OK? Kill the idea if the datasays it's a bad idea.
Maybe you get some data backthat says, hey, you know what? This is a pretty goodidea, but the execution is not moving theneedle fast enough.
That's when you pivot.
You maintain your strategy.
You change your tactics.
When you find somethingthat works well, that's when you double down.
That's when you scale it.
That's when you optimize it.
That's when you giveit to more people, because it's moving theneedle in the right direction, but only based on thoseobjective observations that you're making as youslowly increase it now.
Now look, thereeasy parts of this, and there are hardparts of this.
Here's the easypart– measuring.
Measuring is easy.
It's just instrumentingthe code in such a way that you're actuallycollecting this data.
Talking to customers is easy.
Right? It gives you thequalitative insight into why customers arebehaving in certain ways.
Right? And then reflecting,coming together into teams and saying here'swhat we've learned.
This is what's happening.
This is whatcustomers are doing.
Here's why they're doing it.
These are the easy parts.
Here's the hard part.
Changing course,actually saying, you know what, yeah,we've gone down this path for three months.
The numbers arecoming back and saying this is the wrongpath to go down.
I know, I know, butthe executive really wants this to launch.
Or wow, we spent threemonths working on it.
What are we going todo, just throw it away? Yeah, that's whatyou're going to do.
You're going to throw itaway, because it's not meeting yourbusiness objectives.
The hard part is having theorganizational and the team maturity to say, thisis the wrong direction.
Let's change course.
And let's move onto something else.
And again, from this blogpost over at the Gilt Group– I thought this was reallyinteresting– they define two types of KPIs– keyperformance indicators.
They've got high-levelstrategic ones that the company cares aboutfor an extended period of time.
And then they give teamsthese tactical KPIs to achieve that theyknow are leading indicators of thesestrategic KPIs.
And the teams work on thatuntil they hit those numbers, and then they move on.
That's the tactical, that's thebusiness objective to learn.
And we have to buildan organization that takes advantageof the small team, and that allows them to comeup for funding periodically.
It's almost likea start-up, right? You've funded us forthis amount of time.
You've given us this objective.
After that amount oftime, you can say, look, this is what we've learned.
Will you fund us again? And the organization gets tomake an evidence-based decision about whether or not it'sworthwhile to actually put more money into thisparticular path.
The last thing I wantto talk about is this– how should the team work? We talked about themakeup of the team.
We talked about howwe task the team.
And the reality is,what should the process be– the final things.
How should the process be? How should the team work? And then once again,we're going to start with a few anti-patterns,how the team should not work.
So first and foremost, a lack ofcross-functional collaboration really starts to breakdown a culture of learning and a culture of innovation.
This is a bit of an excuse toput more circus photos up here, but just to give you a sense,those are the elephants, obviously, on the end.
The women in the middle walkedon large white balls up ramps.
That's what they did.
And that's Charlie.
He served our meals,three meals a day, if you were brave enoughto eat food from Charlie.
And most days, we were.
Some days were a littlesketchier than others.
Now look, there was nocross-functional collaboration in the circus.
In fact, the siloswere so deep there were interdepartmental silos.
The clowns fought each otherover who originated a gag.
I mean, can you imagineif the Human Cannonball– and we had a tiger trainer.
We had 10 tigers.
If the Human Cannonball and thetiger trainer collaborated– the guy's alreadyflying across the tent.
If there were 10 tigersunderneath there, what's the difference,ultimately? But at the end of the day, theexcitement of that act is 10x.
None of that everhappened, because everybody felt like they would be givingup something unique if they let go of their originalityand collaborated with somebody else.
Another anti-pattern isa fixation on job titles.
We get so hung up on whatit says on our job title.
Oh, you're the designer.
There's no way you can code.
You're the engineer.
There's no way youcan design, right? And thinking about thatgreatly limits the contribution that our small teammembers can make to the process oflearning and innovation.
To drive the point home,I want to share with you this photo right here.
Does anybody knowwho this guy is? Any guesses? Take a guess.
AUDIENCE: Nigel from Spinal Tap? JEFF GOTHELF: It's notNigel from Spinal Tap, but that's a great guess.
AUDIENCE: Gene Simmons? JEFF GOTHELF: It'snot Gene Simmons, but you guys are in theright era, for sure.
AUDIENCE: Randy Rose.
JEFF GOTHELF: It'snot Randy Rose.
That's a really good one.
I never heard that one before.
That's a good one.
All right, I'll give it to you.
This is who this is.
This is Jeff "Skunk" Baxter.
Jeff "Skunk" Baxter was afounding member of Steely Dan, and he was a Doobie Brotherfor 35 years– legendary American rock-and-rollguitarist.
Any guesses onwhat he does today, other than playing nostalgiaconcerts at state fairs? AUDIENCE: Human Cannonball.
JEFF GOTHELF: Human Cannonball.
That's a good one.
UX designer? No, he's not a UX designer.
This is what he does today.
Right? I'll let you readit for just a sec.
Mr.
Baxter is a consultantto the US Government on missile defense.
That guy.
For better or forworse, that's the guy.
OK? Point here is, if heshowed you his resume, and it said, listen– foundingmember of Steely Dan, Doobie Brother for 35 years,would you let him within a mile of a conversationon missile defense? And of course the answer is no.
We would never let himdo that, because we get hung up on that job title.
Again, people have secondarycompetencies and passions, and they're reallygood at other things.
Let's not limittheir contribution, especially in the earlystages of a product, during product discoveryand conception.
Inevitably, peoplewill fall back to their core competencies.
Engineers will write codeand designers will design.
But especially in theearly stages of a product, let people contribute inwhatever way that they can.
Other anti-patterns.
A fear of failure, right? My ass is on the line, right? Right? If I don't get this right,something's going to happen.
I'm going to get fired.
I'm going to get demoted.
And we create thesecultures, but we never consider the scopeof the failure.
Think about that Amazonscope of failure.
If they get 11.
6 secondsworth of a release wrong, that's terrific.
They've learned something.
And they can roll it back, andthey can move that forward.
If we take small risks, thenthe failure brings learning.
That's the experimentation.
And that's what we should value.
We shouldn't make peopleafraid of actually getting something wrong.
That should be, ultimately,encouraged, right? Another is arbitrary deadlines.
If you owe Paulie money,you got to pay Paulie.
Right? That's kind of how it works out.
Arbitrary deadlinesare typically just motivational tools.
There's usually noreal reason to get something done bya certain date.
And we talked earlier aboutwhat happens when you fix date and you fix scope.
If you fix thedate, that's fine, but let the teamsbuild the best product that they can by thatdeadline, and then let them continue optimizingit after the deadline passes.
What ends up happening here,is the teams end up building this CYA culture, where we dojust enough to get things done, and then to move thingsforward to the next thing.
We never do our best work.
We never take risks.
We never innovate, becausewe're afraid that we're not going to hit thedeadline, or that we're going to get fired,and so forth.
And so the thingsto think about is, how does a culture oflearning change the way that a team works? Ultimately– So we talkedabout who's on the team and how we task theteam, but how do we actually get themto do their best work? And there's a couple ofdifferent ways to do this.
First and foremost,people take smaller risks.
Right? That's the first tack.
This could essentiallybe the– you could argue this is the MVPfor the GoPro camera, right? Let's get that out there.
Let's try it andsee if it works.
Let's see what kind ofproduct we get out of it.
There's a key principlein lean thinking that says that we're alwaysmoving from doubt to certainty.
That between us and theend state of perfection, which we neverattain, there's a fog.
And we can only see two orthree steps into that fog.
And so to mitigate the risk ofrunning in the wrong direction, we take small steps.
We take small risks.
We run experiments.
We test our hypotheses.
And if the resultsfrom that experiment come back andconfirm our thinking, we keep going in that direction.
But if the results come back andthey say that was the wrong way to go, we roll back, or wepick a different direction.
Because we don't wantto run head-first into the fog assuming thatit's the right way to go.
We take smaller risks.
This was awesome.
I ran into this.
This is Map My Run.
You guys know this app? I was running in LA this summerwhile I was working out there.
Early in the morning, you cansee the dates, the time stamp, so you know I'm not lying.
6:54 AM I wasalready out running.
And I got to theend of Venice Pier.
Venice Pier is beautiful.
It goes a half mile into theocean, and the sun is rising, and the palm trees are swaying,and the surfers are out.
And so I paused to takea break and kind of take in the scenery.
And I stopped thetracking of my run, and this beautiful modalwindow comes up and says, hey, would you like to take photosand add it to your run? And at that moment, I wasfeeling particularly inspired.
Adrenaline's flowing,and I'm sweating, and I'm breathing heavy.
And I said hell yeah.
I also liked the fact thatthey used the term MVP, coming from the lean world.
They mean most valuableplayer, of course.
But I said, terrific.
Go MVP.
And I tap Go MVP, and they'relike, all right, great.
Give us 30 bucks.
And you can totally do that.
And I was, like wah wah.
No way am I making a rationalpurchase decision at 7 o'clock in the morning, wheezing, can'tbreathe and eyes are burning.
But, so the end of this workflowis kind of crappy, right? But the beginning–none of these features here ever have to exist.
Right? All you have to dois put up that modal and count the taps onGo MVP and No Thanks.
And when you getenough that say yes, I like this– that'swhen you start investing in actualfeature development.
Right? That's a small experiment.
That's a smallerrisk that you can take to see if there's any valuein actually investing further in building thisparticular product.
Right? Because the agileworld has come up from an engineering perspective.
The 17 writers of "The AgileManifesto" were engineers.
And typically agile–in most organizations that I deal with– comes fromthe engineering department.
It's very rare thatthe designers are like, let's do agile.
And the engineers are like,no, we don't want to do it.
Let's do waterfall.
Right? What's happened is that we builtthese amazing organizations, these amazing softwareengineering organizations, that are incentivized forthe efficient delivery of high-quality code.
But what the agileprocess doesn't have– as my friend Bill Scottlikes to say– is a brain.
It lacks the capacity todetermine what to build, and how to implement it,and how to design it, and what copy toput on top of it.
And this experimental,hypothesis-driven approach, where we take small risksand then we learn from it, allows us to drive theagile process in such a way that we get great code andgreat design and great copy that customersactually want to use, because we've measuredtheir behavior.
That's our definitionof success, is the change incustomer behavior.
And we have to rememberthat works as designed is simply not goodenough anymore.
So one more.
I just want to prove to youhow healthy and athletic I am, so I'm going to use onemore athletic story here.
But this is a picture of my gym.
I live in New Jersey.
This is a picture of my gym.
This is my favoritefeature at this gym.
It's called the cardiotheater, and essentially, it is a movie theater withcardio machines in it.
That's it.
So instead of putting yourheadphones on and watching a movie on a smallscreen, you got on the ellipticalor the treadmill, and they blast the movieout at you in the morning.
I like it.
I love that feature, in fact.
I usually go in the mornings,so I arrive around 5:30.
4:30 in the morning, theguy comes and opens the gym.
He turns the lights on.
He pushes Play on the movie.
He sees that there'sa picture and he hears that there's sound.
As far as he's concerned,works as designed.
And then he goes backto the front desk.
I show up at 5:30, alongwith half the population of northern New Jersey.
For some reason, everyone'sawake at that hour.
And as soon as you get more thantwo people back here running on a treadmill, you can'thear the movie anymore.
Unusable, undesirable,works as designed.
I have to get offthe treadmill and go to the front every single day,and tell the guy to turn it up because we can't hear the movie.
So we have to ensure thatjust because a feature works as designed, people actuallywant to use it and can use it.
And the way we do that is withclear definition of success.
Can we change customerbehavior in the right way? And we can measure that.
And we use that as the milemarker for whether or not we're building the right things.
Now again, we want to promotecompetencies over roles.
We want people to do whateverit is that they're good at to help us move forward faster.
And then lastly andmost importantly, we want to promoteteam self organization.
Give teams a littlestructure, and then let them figure out howbest to work together, and how best to achievethe goals that you've set out for them.
This is a quote fromGeoff Nicholson, who invented the Post-It note.
He said that 3M wanted toproductize this thing with Six Sigma as soon as hehad thought of it.
And he had no proofthat customers actually wanted this thing yet.
We don't want to imposea process on the team just because we think it'sgood for the production of code or the production of features.
Right? Let the teams figure outexactly what they need to build, and when they need to scale it,and how best to work together.
Last thing I want to say.
This is hard work.
All of this, everything I justtold you to do is difficult.
It's process change.
It's culture change.
Why should you do it? Why? Why invest in it? If things are goingOK, why do it? First and foremost, you're goingto make your customers happy, or you're going to build stuffthey actually want and can use, which means they'llcome back and use it, and they'll tell theirfriends, and they'll share it, and you'll have more customers.
You'll reduce waste by buildingmore successful products and not working for along time on products that don't stand a chance of success.
We'll focus our people andour time more effectively.
And then lastly, thisincreases employee morale.
It allows people to feel likethey're part of a greater whole, there's a biggermission, their ideas count.
They're loyal.
They're passionateabout the ideas, because they areultimately their ideas that solve these problems.
And that's terrificfor your hiring brand and for yourretention over time.
At the end of day, letme leave you with this.
You have to transform froma culture of delivery, focused on gettingfeatures out the door, to a culture of learning.
Thank you verymuch for listening.
I'd love to take your questions.
[APPLAUSE] Questions.
I know we have mics over there.
Yes, sir.
AUDIENCE: You give somegreat examples of companies that do this, they iterate fast.
They learn quickly.
They try a lot of experiments,like Amazon, Facebook, even Google.
But there's a company thatwrites a lot of software that doesn't seem to dothis, and that's Apple.
Nevertheless, they'vebeen very successful.
So can you explain whythey managed to get things right without seeming to try allof these constant experiments? JEFF GOTHELF: I would arguethat they do experiment, and that they certainly do asignificant amount of research.
I just don't think thatwe see a lot of it.
And I don't think that– they'recertainly not public with it like the rest of the companiesthat you just mentioned, like Google andFacebook, and so forth.
I think what Apple really excelsat is understanding ecosystems.
And they do this byobservation and by research and by learning.
And then when they come,they take ambitious gambles at design-driven innovationto solve the problems that they're viewingin these ecosystems.
Now you could say look,hey, you know what? They don't test, and theydon't talk to customers.
I think they dotalk to customers, and I think they absolutelybuild prototypes and run experiments.
They seem to lose them in barsin Palo Alto fairly regularly, right? But also take a lookat the iteration after iteration afteriteration of the hardware and the software productsthat you're using for them, and you can see thatthose essentially end up being experiments.
I mean the original iPhonefeels like a rock compared to an iPhone 6 these days.
And I think that there'sexperimentation and tons of learning there.
Incidentally, so Ithink that's one thing.
Related is there's aChinese company called [? Chaomi ?] thatmanufactures cellphone, mobile phones, and tablets.
And they're competing directly.
They're trying to competewith Apple, anyway.
They do iterativedesign on their hardware and their software in one-weekiterations, which is amazing.
Every Tuesday theyship 100,000 phones.
People snap them up, and they'reon the phone with these people, and they're measuringusage to understand how well the productsare meeting their needs.
And so that's what I'veseen as evidence of that.
Now the last thingI'll say about this, because this questioncomes up all the time, is you can't dismiss somelevel of design genius.
Right? You have to give some creditto Steve Jobs and the very few people in the worldwho are like him, like Jony Ive and so forth,who have this design genius.
I don't think youcan dismiss that as part of the componentof their success.
But I think the designgenius, the creativity, comes in in solvingreal problems.
And they see those realproblems by observation and understanding ecosystems.
That's what I think.
Yeah? AUDIENCE: I waswondering if you could talk a little bit about thestructuring of small teams in big organizationssuch as here, where there are literallyhundreds of engineers and tens of product managers,just in one product.
So do you see these teamsforming and reforming? Do you see these teams havecross product, cross function? So how do you imagine them work? And then related to that, whatis the decision-making process like in that structure? JEFF GOTHELF: Yeah, it's agreat, really great questions.
I don't claim tohave all the answers.
I can share with you a coupleof anecdotes from my experience, and something interesting thatI heard last week on my travels.
So the team structure itself,typically designer, product manager, and engineers, solike, one, one, and then two to four engineers as faras each one of those pods.
And I think you'll see thatrepeated in some of the other, like the Spotifywhitepaper that was out.
I think what's interestingis if you've got– so taking the– if you'vegot a big problem and you've got a set ofengineers or a set of teams that are working towardssolving this particular problem, I think you give thatproblem to all those teams.
And I heard Mary Poppendieckspeak last night.
Mary Poppendieck is– she'sone of the legends of the Agile world.
She's been around35, 40 years, trying to make this stuff workin large companies.
And she was talking aboutgiving these pods, essentially, all the same problem,and incentivizing them the same way, so everyonehas the same incentives to achieve the samesuccess criteria.
And then they have to builddependencies on each other.
They have to worktogether in order to achieve thatoverarching goal.
So that's one way tokind of unite them in a similar direction.
I think the questionthen becomes, how do you make decisions? I've seen it workone of two ways.
And again, that'sjust my experience.
It's not necessarily the answer.
Either there is a product lead,typically a product manager.
Although it could bethe design leader.
There's a team leadwho makes a decision.
And where I've seen it work,actually, more effectively is that there's a triumvirateof the product lead, the tech lead, andthe design lead, who kind of gather all theseideas from the team, and then make a decisiontogether, and then bring that back to the teams.
I've seen that work well, also.
Yes, sir? AUDIENCE: I'm Interestedin constraints, things like legal or regulatoryor just bureaucratic, and where they live.
So these teamsare generally kind of constraints oninnovation, as well, or least boundariesof innovation.
Should the team itselfown that constraint and possibly bedemoralized by it? Or should it beexternal, in which case it becomes kind of an enemyor a systematic problem? JEFF GOTHELF: I think you haveto work within your reality, right? So if you work inhealth care, there are going to belegal constraints and personalinformation constraints, privacy constraints, thatyou have to work with, and same thing in financialservices and so forth.
I don't think thatyou can ignore them.
And I think you risk damage.
Forget legal damagefor a second, but I think yourisk brand damage if you choose to flaunt thosein favor of experimentation and hypothesis.
Now that being said, therigidity of those constraints is debatable incertain situations, and I think that if you canhave a realistic conversation with legal department,for example, or the brandingdepartment, or IT, or whoever isconstraining your ability to learn the thingsthat you need to learn, I'm confident thatyou can typically find ways to get around that.
I'll give you an example.
We did a lot of experimentationwith a big financial services company downtown from here.
Again, 100-year-old brand, lotsof financial data– it's risky.
Options are– you can testoff-brand, for example, so it has nothing to dowith the official brand.
That's one test.
You can opt people intoa beta-tester pool.
So you're self-selectinga little bit.
There's some bias there.
You get the more tech-savvyindividuals, and so forth.
But at least they've agreed totake part in these experiments.
But there are ways to get aroundthis where you can negotiate with the people whoare constraining you to get somekind of learning.
I think that youhave to own it, and I don't think you can disregardit, because ultimately whatever you come up with has tolive in that world as well.
And I think that ifyou disregard it, you're building something that'sprobably destined to fail.
Yes, sir.
AUDIENCE: I also have aquestion about the small teams.
So the first questionis, what's the best way, in your experience,to incentivize people to do their bestwithin the small teams? And second question is, whenyou have the small teams, and you want to encourage themto kind of all collaborate and work together on something,but then in that environment, how do you assigncredit correctly so that people, again, haveincentives to kind of– JEFF GOTHELF: Well, youassume that taking credit is incentive.
And I suppose thatthat could be.
The way– so again,based on my experience, the way that I've seenthe best incentive has been to let peoplefigure out the solution.
There's an amazinglevel– there's an exponentially bigger levelof passion and commitment to the success of aninitiative if the team is working on an ideathey came up with.
The difference between tellinga team to build something and telling them to figureout how to solve this problem is the incentive.
That's what getsthem to motivate.
That's what getsthem moved forward.
And the interesting thing–if you buy my concept that the atomic object ofinnovation is the team, then the team wins togetheror the team loses together.
There's no, the teamdid great design, but the engineers messed it up.
Right? There's none of that.
It doesn't happen.
The product is the product.
You can't separate greatengineering from great design.
And we managed toachieve our business goal and thecustomer's need, and it seems to be a winning product.
That to me has been thegreatest source of incentive to get these teams towork well together.
OK? Anything else? Thank you guys very much forspending your lunch with me today.
That was a lot of fun.
[APPLAUSE].
Source: Youtube
