How Google Grows…and Grows…

Below is a great article on Google, taken verbatim from an article in FastCompany Magazine. [ From: Issue 69 | March 2003 | Page 74 | By: Keith H. Hammonds ]

If you’re a Google engineer, you know that those nine words comprised about 120 bytes of data, enough to slow download time for users with modems by 20 to 50 milliseconds. You can estimate the stress that 120 bytes, times millions of searches per minute, put on Google’s 10,000 servers. On the other hand, you can also measure precisely how many visitors took the tour, how many of those downloaded the Google Toolbar, and how many clicked through for the first time to Google News.

This is what it’s like inside Google. It is a joint founded by geeks and run by geeks. It is a collection of 650 really smart people who are almost frighteningly single-minded. “These are people who think they are creating something that’s the best in the world,” says Peter Norvig, a Google engineering director. “And that product is changing people’s lives.”

Geeks are different from the rest of us, so it’s no surprise that they’ve created a different sort of company. Google is, in fact, their dream house. It also happens to be among the best-run companies in the technology sector. At a moment when much of business has resigned itself to the pursuit of sameness and safety, Google proposes an almost joyous antidote to mediocrity, a model for smart innovation in challenging times.

Google’s tale is a familiar one: Two Stanford doctoral students, Sergey Brin and Larry Page, developed a set of algorithms that in 1998 sparked a holy-shit leap in Web-search performance. Basically, they turned search into a popularity contest. In addition to gauging a phrase’s appearance on a Web page, as other engines did, it assessed relevance by counting the number and importance of other pages that linked to that page.

Since then, newer search products such as Teoma and Fast have essentially matched Google’s advance. But Google remains the undisputed search heavyweight. Google says it processes more than 150 million searches a day — and the true number is probably much higher than that. Google’s revenue model is notoriously tough to deconstruct: Analysts guess that its revenue last year was anywhere from $60 million to $300 million. But they also guess that Google made quite a bit of money.

As a result, there is constant, hopeful speculation among financiers around an initial public offering, a deal that could be this decade’s equivalent of the 1995 Netscape IPO. A few years back, such a deal might have valued Google at $3 billion or more. Even today, a Google offering might fetch $1 billion.

For now, though, most of the cars in the lot outside Google’s modest offices in a Mountain View, California office park are beat-up Volvos and Subarus, not Porsches. And while Googlers may relish their shot at impossible wealth, they appear driven more by the quest for impossible perfection. They want to build something that searches every bit of information on the Web. More important, they want to deliver exactly what the user is looking for, every time. They know that this won’t ever happen, and yet they keep at it. They also pursue a seemingly gratuitous quest for speed: Four years ago, the average search took approximately 3 seconds. Now it’s down to about 0.2 seconds. And since 0.2 is more than zero, it’s not quite fast enough.

Google understands that its two most important assets are the attention and trust of its users. If it takes too long to deliver results or an additional word of text on the home page is too distracting, Google risks losing people’s attention. If the search results are lousy, or if they are compromised by advertising, it risks losing people’s trust. Attention and trust are sacrosanct.

Google also understands the capacity of the Web to leverage expertise. Its product-engineering effort is more like an ongoing, all-hands discussion. The site features about 10 technologies in development, many of which may never be products per se. They are there because Google wants to see how people react. It wants feedback and ideas. Having people in on the game who know a lot of stuff tells you earlier whether good ideas are good ideas that will actually work.

But what is most striking about Google is its internal consistency. It is a beautifully considered machine, each piece seemingly true to all the rest. The appearance of advertising on a page, for example, follows the same rules that dictate search results or even new-product innovation. Those rules are simple, governed by supply, demand, and democracy — which is more or less the logic of the Internet too.

Like its search engine, Google is a company overbuilt to be stronger than it has to be. Its extravagance of talent allows it crucial flexibility — the ability to experiment, to try many things at once. “Flexibility is expensive,” says Craig Silverstein, a 30-year-old engineer who dropped his pursuit of a Stanford PhD to become Google’s first employee. “But we think that flexibility gives you a better product. Are we right? I think we’re right. More important, that’s the sort of company I want to work for.”

And the sort of company that every company can learn from. What follows, then, is our effort to “google” Google: to search for the growth secrets of one of the world’s most exciting growth companies. Like the logic of the search-engine itself, our search was deep and democratic. We didn’t focus on Google’s big three: CEO Eric Schmidt and founders Brin and Page. Instead, we went into the ranks and talked with the project managers and engineers who make Google tick. Here’s what we learned.

Rule Number One: The User Is in Charge

“There are people searching the Web for ‘spiritual enlightenment.’ ” Peter Norvig says this with such utter solemnity that it’s impossible to tell for sure whether he gets the irony. Then again, Norvig is the guy who authored a hilarious PowerPoint translation of Lincoln’s Gettysburg Address (available at www.norvig.com), a geek classic. So maybe he’s having fun.

But he’s also making a point. When someone enters a query on Google for “spiritual enlightenment,” it’s not clear what he’s seeking. The concept of spiritual enlightenment means something different from what the two words mean individually. Google has to navigate varying levels of literality to guess at what the user really wants.

This is where Googlers live, amid semantic, visual, and technical esoterica. Norvig is Google’s director of search quality, charged with continuously improving people’s search results. Google tracks the outcome of a huge sample of the queries that we throw at it. What percentage of users click on the first result that Google delivers? How many users click on something from the first page? Norvig’s team members scour the data, looking for trouble spots. Then they tweak the engine.

The cardinal rule at Google is, If you can do something that will improve the user’s experience, do it. It is a mandate in part born of paranoia: There’s always a chance that the Google destroyer is being pieced together by two more guys in a garage. By some estimates, Google accounts for three-quarters of all Web searches. But because it’s not perfect, being dominant isn’t good enough. And the maniacal attack on imperfection reflects a genuine belief in the primacy of the customer.

That’s why Google must correctly interpret searches by Turks and Finns, whose queries resemble complete sentences, and in Japanese, where words run together without spaces. It has to understand not only the meanings of individual words but also the relationships of those words to other words and the characteristics of those words as objects on a Web page. (A page that displays a search word in boldface or in the upper-right-hand corner, for example, will likely rank higher than a page with the same words displayed less prominently.)

It’s why the difference between 0.3 seconds and 0.2 seconds is pretty profound. Most searches on Google actually take less than 0.2 seconds. That extra tenth of a second is all about the outliers: queries crammed with unrelated words or with words that are close in meaning. The outliers can take half a second to resolve — and Google believes that users’ productivity begins to wane after 0.2 seconds. So its engineers find ways to store ever-more-arcane Web-text snippets on its servers, saving the engine the time it takes to seek out phrases when a query is made.

And it’s why, most of the time, the Google home page contains exactly 37 words. “We count bytes,” says Google Fellow Urs Holzle, who is on leave from the University of California at Santa Barbara. “We count them because our users have modems, so it costs them to download our pages.”

Just as important, every new word, button, or feature amounts to an assault on the user’s attention. “We still have only one product,” Holzle says. “That’s search. People come to Google to search the Web, and the main purpose of the page is to make sure that you’re not distracted from that search. We don’t show people things that they aren’t interested in, because in the long term, that will kill your business.”

Google doesn’t market itself in the traditional sense. Instead, it observes, and it listens. It obsesses over search-traffic figures, and it reads its email. In fact, 10 full-time employees do nothing but read emails from users, distributing them to the appropriate colleagues or responding to them themselves. “Nearly everyone has access to user feedback,” says Monika Henzinger, Google’s director of research. “We all know what the problem areas are, where users are complaining.”

The upshot is that Google enjoys a unique understanding of its users — and a unique loyalty. It has managed a remarkable feat: appealing to tech-savvy Web addicts without alienating neophytes who type in “amazon.com” to find . . . Amazon.com. (Yes, people really do that. Google doesn’t know why.)

“Google knows how to make geeks feel good about being geeks,” says Cory Doctorow, prominent geek, blogger, and technology propagandist. Google has done that from the beginning, when Brin and Page basically laid open their stunning new technology in a 1998 conference paper. They invited in the geeks in and made them feel as if they were in on something special.

But they didn’t forget to make everyone else feel special too. They still do, by focusing relentlessly on the quality of the experience. Make it easy. Make it fast. Make it work. And attack everything that gets in the way of perfection.

Rule Number Two: The World Is Your R&D Lab

Paul Bausch is a 29-year-old Web developer in Corvallis, Oregon. He works with ASP, SQL Server, Visual Basic, XML, and a host of other geek-only technologies. He helped create Blogger, a widely used program that helps people set up their own Web log. And in a way that’s intentionally imprecise, he’s part of Google’s research effort.

“Isn’t this great?” exclaims Nelson Minar, a senior Google engineer. Minar and I are fooling with Bausch’s quirky creation called Google Smackdown, where you can compare the volume of Google citations for any two competing queries. (The New York Yankees slam the New York Mets; war conquers peace.) Google loosed Smackdown and other eccentric Web novelties when it released a developer’s kit last spring that lets anyone integrate Google’s search engine into their own application. The download is simple, and the license is free for the taking.

Here’s the scary bit: Basically, those developers can do whatever they want. The only control that Google exerts is a cap of 1,000 queries per day per license to guard against an onslaught that might bring down its servers. In most cases, Minar and his colleagues have no idea how people use the code. “It’s kind of frustrating,” he concedes. “We would love to see what they’re doing.”

Most companies would sooner let temps into the executive washroom than let customers — much less customers who can hack — anywhere near their core intellectual property. Google, though, grasps the power of an engaged community. The developer’s kit is a classic Trojan-horse strategy, putting Google’s engine in places that the company might not have imagined. More important, Bausch says, opening up the technology kimono “turns the world into Google’s development team.”

Sites like Smackdown, while basically toys, “are an inkling of what Google could be used for,” Minar says. “We can’t predict what will happen. But we can predict that there will be an effect on our technology and on the way the world views us.” And more likely than not, it will be something pretty cool.

Rule Number Three: Failures Are Good. Good Failures Are Better.

In Google Labs, just two clicks away from its home page, anyone can test-drive Google Viewer, sort of a motion-picture version of your search results, or Voice Search, a tool that lets you phone in a query and then see your results online. Is either ready for prime time? Not really. (Try them out. On Voice Search, you’re as likely to get someone else’s results as your own.)

But that’s the point. The Labs reflect a shared ethos between Google and its users that allows for public experimentation — and for failure. People understand that not everything Google puts on view will work perfectly. They also understand that they are part of the process: They are free to tell Google what’s great, what’s not, and what might work better.

“Unlike most other companies,” observes Matthew Berk, a senior analyst at Jupiter Research, Google has said, ‘We’re going to try things, and some aren’t going to work. That’s okay. If it doesn’t work, we’ll move on.’ ”

In the search business, failure is inevitable. It comes with the territory. A Web search, even Google’s, doesn’t always give you exactly what you want. It is imperfect, and that imperfection both allows and requires failure. Failure is good.

But good failures are even better. Good failures have two defining characteristics. First, says Urs Holzle, “you know why you failed, and you have something you can apply to the next project.” When Google experimented with thumbnail pictures of actual Web pages next to results, it saw the effect that graphical images had on download times. That’s one reason why there are so few images anywhere on Google, even in ads.

But good failures also are fast. “Fail,” Holzle says. “But fail early.” Fail before you invest more than you have to or before you needlessly compromise your brand with a shoddy product.

Rule Number Four: Great People Can Manage Themselves

Google spends more time on hiring than on anything else. It knows this because, like any bunch of obsessive engineers, it keeps track. It says that it gets 1,500 résumés a day from wanna-be Googlers. Between screening, interviewing, and assessing, it invested 87 Google people-hours in each of the 300 or so people that it hired in 2002.

Google hires two sorts of engineers, both aimed at encouraging the art of fast failure. First, it looks for young risk takers. “We look for smart,” says Wayne Rosing, who heads Google’s engineering ranks. “Smart as in, do they do something weird outside of work, something off the beaten path? That translates into people who have no fear of trying difficult projects and going outside the bounds of what they know.”

But Google also hires stars, PhDs from top computer-science programs and research labs. “It has continually managed to hire 90% of the best search-engine people in the world,” says Brian Davison, a Lehigh University assistant professor and a top search expert himself. The PhDs are Google’s id. They are the people who know enough to shoot holes in ideas before they go too far — to make the failures happen faster.

The challenge is negotiating the tension between risk and caution. When Rosing started at Google in 2001, “we had management in engineering. And the structure was tending to tell people, No, you can’t do that.” So Google got rid of the managers. Now most engineers work in teams of three, with project leadership rotating among team members. If something isn’t right, even if it’s in a product that has already gone public, teams fix it without asking anyone.

“For a while,” Rosing says, “I had 160 direct reports. No managers. It worked because the teams knew what they had to do. That set a cultural bit in people’s heads: You are the boss. Don’t wait to take the hill. Don’t wait to be managed.”

And if you fail, fine. On to the next idea. “There’s faith here in the ability of smart, well-motivated people to do the right thing,” Rosing says. “Anything that gets in the way of that is evil.”

Rule Number Five: If Users Come, So Will the Money

Google has no strategic-planning department. CEO Eric Schmidt hasn’t decreed which technologies his engineers should dabble in or which products they must deliver. Innovation at Google is as democratic as the search technology itself. The more popular an idea, the more traction it wins, and the better its chances.

Here’s how one Google service came into the world. In December 2001, researcher Krishna Bharat posted an internal email inviting Googlers to check out his first crack at a dynamic news service. Although Google offered a basic headline service at the time, news was not a corporate mandate. This was simply Bharat’s idea. As a respected PhD hired away from Compaq and a member of the company’s 10-person research lab, coming up with new ideas is basically Bharat’s job.

For an early prototype, it was quite a piece of work. Bharat had built an engine that crawled 20 news sources once an hour, automatically delivering the most recent stories on in-demand topics — something like a virtual wire editor. And within Google, it got a lot of attention. Importantly, it attracted the attention of Marissa Mayer, a young engineer turned project manager.

Mayer connected Bharat with an engineering team. And within a month and a half, Google had posted on its public site a beefed-up version of the text-based demo, which is now called Google News and which features 155 sources and a search function. Within three weeks of going public, the service was getting 70,000 users a day.

One reason Google puts its innovations on public display is to identify failures quickly. Another reason is to find winners. For Bharat and Mayer, those 70,000 users provided ammunition to build a case for News within Google. “A public trial helps you go fast,” Mayer says. “If it works, it builds internal passion and fervor. It gets people thinking about the problem.”

Soon, Mayer had marshaled a handful of engineers to bulk up News. They expanded its reach to more than 4,000 sources, updated continuously instead of hourly. They created an engine that was robust enough to support five times the anticipated early volume. And they prettied it up, designing an interface that displayed hundreds of headlines and photos but that was still easy to navigate. By September, the new News was up.

Is Google News an actual product? Not exactly. Its home page is still labeled Beta, as are all but a few of Google’s offerings. It may become a Google fixture, it may disappear, or it may recede into Google Labs. Mayer is still studying the traffic, and the engineers are still tweaking, reacting to users’ emails.

The company’s organic approach to invention bugs some onlookers. “Google is a great innovator,” says Danny Sullivan, editor of Search Engine Watch and an influential commentator. “They keep rolling out great things. But Google News was an engineer deciding he wanted a news engine. Now Google has this product, and it doesn’t know how to make money off of it.”

Sullivan is onto something important: At some point, all of this great stuff has to turn a profit. That was the one great moral of the dotcom blowout: “Monetizing eyeballs” turned out to mean “throwing money down a sinkhole.” When Mayer argues that “the traffic will let us know” whether News is a success, she’s echoing a long line of now-unemployed executives who thought that they had tamed the business cycle.

But at Google, building and then following the traffic makes perfect sense. It’s central to the company’s culture and its operating logic. Consider this: For the first 18 months of its existence, Google didn’t make a penny from its basic Web-search service. Only then did it make the transition from great technology to great technology with a critical mass of users.

And Google was able to package that traffic in ways that seem both ingenious and completely synchronous. The search service itself remained free. But Google has, for example, sold untold numbers of ads pegged to specific search keywords. (Not surprisingly, Fast Company slips in a paid ad to the side of your results whenever your query includes fast company.)

Advertisers don’t just pay a set rate, or even a cost per thousand viewers. They bid on the search term. The more an advertiser is willing to pay, the higher its ad will be positioned. But if the ad doesn’t get clicks, its rank will decline over time, regardless of how much has been bid. If an ad is persistently irrelevant, Google will remove it: It’s not working for the advertiser, it’s not serving users, and it’s taking up server capacity.

This is how it is at Google. Google News attracted eyeballs among Bharat’s employees, so it made the leap to the public domain. If enough users like it, it will have real power with advertisers. And traffic for advertisers will beget even more traffic for advertisers.

So yes, Mayer has a revenue strategy. She’s had one since January 2002, before the first version of News went public. She won’t say what it is, but if News can build enough traffic, Google almost surely will seek advertising. It will probably resell the service to portals and other commercial sites, just as it does with its core Web search. (Every time you see the Google logo on a corporate site, the company is likely paying at least $25,000 a year for a Google server.) “But we’re not in a hurry,” Mayer says. “We’re focused on making News a great experience. Until we figure out whether the product has traction, there’s no rush to execute the revenue plan.”

Could it be any simpler? Build great products, and see if people use them. If they do, then you have created value. And if you’ve truly done that, then you have a business. Says Mayer: “Our motto here is, There’s no such thing as success-failure on the Net.” In other words, if users win, then Google wins. Long live democracy.

Sidebar: Just how big is Google?

That’s hard to say. Officially, Google says that it processes more than 150 million searches a day, but the true number is probably much higher. According to Nielsen/NetRatings, 67.6 million people worldwide visited Google an average of 6.2 times last December. Analysts guess that last year’s revenue was between $60 million and $300 million.

Sidebar: A Gaggle of Google Games

While tens of millions of people like Google, a disconcertingly large minority are obsessed with it. Since 1999, techies have invested many hours and much creativity into devising a wide range of Google-based parlor games and curiosities. Here’s a sampling, courtesy of Google and Cameron Marlow at MIT’s Media Lab.

Googlewhack Find two words which, when combined in a Google query, deliver one and only one result. www.googlewhack.com claims that it has recorded 120,000 whacks since January 2002. Among recent entries to its “Whack Stack” are prevarication pileups and hiccupping flubber. (A Fast Company original: defamatory meerkats.)

Googlebomb Geek terrorism. Taking advantage of a Google loophole, Googlebombers gang up to mass-hyperlink a target page with a specific (usually derogatory) phrase. Google picks up on the links, even if the phrase isn’t on the page itself. The legendary first, incited by Adam Mathes in April 2001, tagged Mathes’s friend Andy Pressman’s site with the words “talentless hack.” For a while, it stuck.

Googleshare The invention of blogger Steven Berlin Johnson. Search Google for one word. Then search those results for the name of a person. Divide the number of results delivered for your second search by those from the first to get that person’s “semantic mindshare” of the word.

Googlism Type in your name, someone else’s name, or a date, place, or thing at www.googlism.com. The application, written by a team at Domain Active in Australia, uses Google to deliver Web-based definitions of your phrase. Bill Gates, for example, is “the anti-Christ,” “a thief,” “a hero,” and “a wanker.”

Google Smackdown Two queries. One search engine. A “terabyte tug-of-war,” as its creator, Paul Bausch, calls it. Just plug in two competing words or phrases at www.onfocus.com/googlesmack/down.asp, and see which delivers more Google results. (Google, with 17.5 million, suffers a rare embarrassment at the hands of God, with 42.6 million.)

Sidebar: How does Google keep innovating?

One big factor is the company’s willingness to fail. Google engineers are free to experiment with new features and new services and free to do so in public. The company frequently posts early versions of new features on the site and waits for its users to react. “We can’t predict exactly what will happen,” says senior engineer Nelson Minar.

Keith H. Hammonds ([email protected]) is a Fast Company senior editor based in New York.