NYTimes: Regulate Your Search Results

I was travelling yesterday and didn't read the papers much, but just saw this today.

Stunning.

Quote:

---

July 14, 2010
The Google Algorithm
Google handles nearly two-thirds of Internet search queries worldwide. Analysts reckon that most Web sites rely on the search engine for half of their traffic. When Google engineers tweak its supersecret algorithm — as they do hundreds of times a year — they can break the business of a Web site that is pushed down the rankings.

When Google was a pure search engine, it was easy to appear agnostic about search results, with no reason to play favorites with one Web site or another. But as Google has branched out into online services from maps and videos to comparison shopping, it has acquired pecuniary incentives to favor its own over rivals.

Google argues that its behavior is kept in check by competitors like Yahoo or Bing. But Google has become the default search engine for many Internet users. Competitors are a click away, but a case is building for some sort of oversight of the gatekeeper of the Internet.

In the past few months, Google has come under investigation by antitrust regulators in Europe. Rivals have accused Google of placing the Web sites of affiliates like Google Maps or YouTube at the top of Internet searches and relegating competitors to obscurity down the list. In the United States, Google said it expects antitrust regulators to scrutinize its $700 million purchase of the flight information software firm ITA, with which it plans to enter the online travel search market occupied by Expedia, Orbitz, Bing and others.

The accusations in Europe may or may not have merit. Google says it only tweaks its algorithm to improve its searches. Some Web sites that have accused Google of unfair placing are merely collections of links with next to no original content of their own, precisely the kind of sites that Google’s search algorithm screens out to better answer queries. Antitrust regulators in the United States could well let Google buy ITA because it does not now provide online travel services.

Still, the potential impact of Google’s algorithm on the Internet economy is such that it is worth exploring ways to ensure that the editorial policy guiding Google’s tweaks is solely intended to improve the quality of the results and not to help Google’s other businesses.

Some early suggestions for how to accomplish this include having Google explain with some specified level of detail the editorial policy that guides its tweaks. Another would be to give some government commission the power to look at those tweaks.

Google provides an incredibly valuable service, and the government must be careful not to stifle its ability to innovate. Forcing it to publish the algorithm or the method it uses to evaluate it would allow every Web site to game the rules in order to climb up the rankings — destroying its value as a search engine. Requiring each algorithm tweak to be approved by regulators could drastically slow down its improvements. Forbidding Google to favor its own services — such as when it offers a Google Map to queries about addresses — might reduce the value of its searches.

With these caveats in mind, if Google is to continue to be the main map to the information highway, it concerns us all that it leads us fairly to where we want to go.

http://www.nytimes.com/2010/07/15/opinion/15thu3.html

---

If you're interested, a thorough response is here: http://searchengineland.com/regulati...rk-times-46521

I just find this whole idea stunning, almost a caricature of aloof "there should be a law" regulation-felation. That they use the phrase "editorial policy" several times speaks volumes.

Google has its own response:

http://googlepublicpolicy.blogspot.c...s-best-in.html





I don't think I, personally, have based my decision to only use google for searches on some researched idea of what constitutes the "best" search-results for me. I'm pretty sure it's been in part due to google's clean and unobtrusive design and in part due to my desire to run with the herd (ie. I feel it's safer for me to just use the same search engine as almost everyone else). How many people actually change their behaviour in the way that Marissa Mayer implies?

With that said, I would like to see data on what happens when a site is pushed down on the search results rankings, and I would like to see which sites have been getting pushed down. Google has apparently found that half a second's delay in page loading speed dramatically influences user behaviour, and I wonder how being pushed down three places may influence user behaviour.

Is this about search engines in general or Google specifically?

I'm less concerned about external forces than their internal ones:

Quote:

---

Here's the main reason for the earnings letdown: Google is spending more to maintain its commanding lead in Internet search while it also tries to diversify by developing products in other promising niches such as online video, mobile devices and computer operating systems. To help achieve its goals, the company added nearly 1,200 employees in the second quarter to end June with more than 21,800 workers.

Google, based in Mountain View, has hired nearly 2,000 workers through the first half of the year, putting it on pace to add the most people to its payroll since 2006 when it ushered in 6,100 new employees in 12 months.

---

Continued @ http://www.nytimes.com/aponline/2010...Google.html?hp

For such a mammoth company that doesn't pay a dividend.....? Curious how many share-holders are employees themselves, holding paper.

I'm surprised by how many articles/blog-posts on the Foundem-case seem to be based on the following ideas:

1. Google can do no wrong, so obviously Foundem sucks and is at fault. Maybe if they sucked less they would do better.

2. What the hell is "Foundem" anyway??! Never even heard of it.

Yeeeeeeah, I'm not really buying either of those arguments. Investigation and a ruling first pls. From what I gather Foundem isn't entirely unknown in the UK at least, and the same goes for Ciao, another company that's joined Foundem in accusing Google for being naughty.

Google can do no wrong, yeah that's sort of conventional wisdom in the US. :confused:

Enterprises can actually pay Google to show up first on their search engine, regardless of real "number of clicks". Then it becomes a self-fulfilling prophecy, in a way. (Sites who pay to be up top get more traffic by default.) That alone makes me take Google's claims less seriously. It's not really all about random efficiency and consequential algorithms, but teh evil regulators are getting in their way?

This is only my personal opinion as faceless web surfer, but I think this sort of thing should be investigated properly based on real cases and on evidence, rather than approached purely through armchair reasoning based on principles and on speculation.

But re. principles, I'm wondering how different it'd be from Microsoft/IE for Google to post its shopping services over all others, regardless of whether or not it suppresses results from other services.

This is very close to the economic and legal issue that Microsoft and its Internet Exploder had in terms of market tying/bundling, except the liberulz are not taking a hard-ass approach on this one.

Quote:

---

Some early suggestions for how to accomplish this include having Google explain with some specified level of detail the editorial policy that guides its tweaks. Another would be to give some government commission the power to look at those tweaks.

---

No. This is what anti-trust regulation is for.

In the past, companies have been fined for tying product prices and bundling their products.
http://www.justice.gov/atr/public/he.../chapter_5.htm


If there is a concern that Google is using its undisputed market power in search engines to influence its sales in acquired divisions, new laws should be made to prevent google from aquiring these divisions, or to allow fines if an official antitrust action/investigation is brought. Not oversight.

Market oversight should exist when the consumer could get badly hurt from a bad product (you can't just ask for a refund for lead-paint toys.. your F- children are already watching South Park reruns all day!) or when the market is controlled by a legal monopoly, as is the case with some utilities, such as water, or often electricity.

We are all armchair-ing. Or regurgitating other "expert analyses". Aren't we?

I've never liked monopolies. Or corporate bullies. Partly why I never owned a pc that was forced to run Microsoft shit, or cell phones that would only operate on mandated platforms or specific providers. Doesn't mean I'm an Apple fanatic either.....I use bing more often than google to search for things.

I bought a new mac because I like macs, but it came loaded with Google Maps and some other stuff that I just won't use.....nothing wrong with partnerships or bundling in principle, but I hate it when using a product assumes things about my preferences. Or worse, when they use one of my choices and 'force' me into using more of their products.

Many of the euro-inspired legal battles are actually good for everyone in the long haul. It's almost amazing that the US bristles at the thought of real competition, since we love to opine about freeeee and open markets, but don't seem very adept at recognizing monopolies when behemoths can buy dominance.

:confused:

Aggie, how do you propose

Quote:

---

new laws should be made to prevent google from aquiring these divisions, or to allow fines if an official antitrust action/investigation is brought.

---

without some oversight in the first place?

Quote:

---

If there is a concern that Google is using its undisputed market power in search engines to influence its sales in acquired divisions, new laws should be made to prevent google from aquiring these divisions, or to allow fines if an official antitrust action/investigation is brought. Not oversight.

---

Why? Its a search engine. And the barriers to entry in the search engine field are pretty small. In no way is Google a monopoly. That is is like suing Wal-Mart for having an oil change service at their stores...

Quote:

---

Originally Posted by Lewkowski

Why? Its a search engine. And the barriers to entry in the search engine field are pretty small. In no way is Google a monopoly. That is is like suing Wal-Mart for having an oil change service at their stores...

---

Or is it like buying a Chevy that will accept only GM parts? That's not really a "monopoly" in the true sense, but it definitely presents some obstacles.

Why should the government give preference of one search engine over another?

Quote:

---

With these caveats in mind, if Google is to continue to be the main map to the information highway, it concerns us all that it leads us fairly to where we want to go.

---

Why is Google the designated main map to the information highway?

Quote:

---

Why should the government give preference of one search engine over another?

---

The government shouldn't... it shouldn't involve itself with regulating internet search engines.

Quote:

---

Originally Posted by Lewkowski

The government shouldn't... it shouldn't involve itself with regulating internet search engines.

---

You think anti-trust laws are just gummint meddling?

Quote:

---

In the past few months, Google has come under investigation by antitrust regulators in Europe. Rivals have accused Google of placing the Web sites of affiliates like Google Maps or YouTube at the top of Internet searches and relegating competitors to obscurity down the list. In the United States, Google said it expects antitrust regulators to scrutinize its $700 million purchase of the flight information software firm ITA, with which it plans to enter the online travel search market occupied by Expedia, Orbitz, Bing and others.

---

EDIT: Should I point out that Google is no longer "just" an internet search engine? Or wouldn't that matter to you.......

I was going to say that as long as Google Scholar remains impartial, I'm good, but then I realized that I mostly search for articles whose names I already know. If I had to look up something completely unrelated to everything I'm already somewhat familiar with I wouldn't rely on Google alone, I don't think. Hmm.

Quote:

---

Originally Posted by Aimless

I don't think I, personally, have based my decision to only use google for searches on some researched idea of what constitutes the "best" search-results for me. I'm pretty sure it's been in part due to google's clean and unobtrusive design and in part due to my desire to run with the herd (ie. I feel it's safer for me to just use the same search engine as almost everyone else). How many people actually change their behaviour in the way that Marissa Mayer implies?

---

Probably a lot. Just one example comes to mind here at 7:30AM. Most computers come with Internet Explorer and most people use IE. IE has Bing as the default search engine which can be changed, but I doubt the average user really knows how. Yet Google's alleged "market share" hasn't dropped precipitously over the years, probably because people actively go to Google.com to do a search. It's the reverse of what that Google exec is talking about, but I don't see why the active changing of behavior wouldn't happen.

Or why it's especially relevant when it comes to how a free tool to organize the Web actually organizes the Web.

Quote:

---

Originally Posted by Aimless

I'm surprised by how many articles/blog-posts on the Foundem-case seem to be based on the following ideas:

1. Google can do no wrong, so obviously Foundem sucks and is at fault. Maybe if they sucked less they would do better.

2. What the hell is "Foundem" anyway??! Never even heard of it.

Yeeeeeeah, I'm not really buying either of those arguments. Investigation and a ruling first pls. From what I gather Foundem isn't entirely unknown in the UK at least, and the same goes for Ciao, another company that's joined Foundem in accusing Google for being naughty.

---

I remember some inane article about that Foundem case a while ago. Just take a look, it's a shitty aggregator of links. You search for a product and they give you links, but it seems they recommend things based on commercial pay-for-inclusion.

The proliferation of those sites is exactly why people like search engines -- they (at least in theory) should rank crappy sites down. There aren't too many keywords that should rank Foundem highly unless it's really a quality site. If Foundem was the best place to search for a camera deal, a lot of people would link to Foundem with words like "Camera deal" and Foundem would rank highly with that kind of search. Instead, no one links to Foundem and their rank justifiably sucks for "camera deal".

But Foundem wants all that sweet free traffic, so they are trying to appeal to regulators to get some kind of special exception and pretending there is an "editorial policy" here besides an algorithm with 200 variables that all point to Foundem being a stupid site.

Quote:

---

Originally Posted by GGT

Enterprises can actually pay Google to show up first on their search engine, regardless of real "number of clicks".

---

Um, no that is factually untrue. Do you seriously believe this? :confused:

Quote:

---

Originally Posted by GGT

I bought a new mac because I like macs, but it came loaded with Google Maps and some other stuff that I just won't use.....nothing wrong with partnerships or bundling in principle, but I hate it when using a product assumes things about my preferences. Or worse, when they use one of my choices and 'force' me into using more of their products.

---

Your computer didn't come loaded with Google Maps. It came with a link to Google Maps in Safari bookmarks. There's a big difference between pre-installed software and a link to a free service with many competitors. :confused:

Quote:

---

Originally Posted by Dreadnaught

I was travelling yesterday and didn't read the papers much, but just saw this today.

Stunning.



If you're interested, a thorough response is here: http://searchengineland.com/regulati...rk-times-46521

I just find this whole idea stunning, almost a caricature of aloof "there should be a law" regulation-felation. That they use the phrase "editorial policy" several times speaks volumes.

---

I happen to agree with you, but the language you use is what people are talking about when they say "you've changed." You used to be a reasoned, even-handed poster, and that was why people admired you. But sometime after you finished college you became a knee-jerk soldier in the culture wars. I'm not the only one who noticed this, but I'm sure you'll just disregard me.

Oh, and stop using "felation." You do it incessantly. Cain is not somebody you should emulate, Dread.

Foundem, like Ciao, aggregates price-data as well as other information in various categories and allows for price comparisons eg. for cameras that meet certain requirements specified by the user. It's a useful service, and I'm not sure why it differs so much from Google's own shopping/price comparison services that it deserves to go from a high ranking to a low ranking and then back up again (which is what they claim has happened for no defensible reason). At least for people in the UK, foundem might actually be useful for finding the best prices.

I personally use pricerunner.se and prisjakt.se very often because they're similar services that are more useful to me than is google, and I see pricerunner-links pop up when I search for pricing info for various products in Swedish. I don't see how Foundem can be very different from those.

Quote:

---

Originally Posted by Aimless

With that said, I would like to see data on what happens when a site is pushed down on the search results rankings, and I would like to see which sites have been getting pushed down. Google has apparently found that half a second's delay in page loading speed dramatically influences user behaviour, and I wonder how being pushed down three places may influence user behaviour.

---

I know the data is out there, it comes across Reddit every few months. The change in traffic is insane if you're not in the top 5 listings for a blind search. 2nd page and beyond will net almost nothing. Google changes its procedures for the same reason state lotteries change their balls. People can pick up on the trends. Its why 4chan is so good at google bombing. Hell, Reddit just did it a few weeks ago to help out one of the members get some bullcrap removed that was negatively effecting his job searching. How long did failure link to Bush's bib page? People game Google all the time, and then they get pissed off when they get busted.
I think it will become less of an issue as the internet continues to grew into everyone'e lives and they end up with a core collection of known site addresses. Aside from spell checking I rarely use plain google anymore.

Google also understands they won't be #1 forever. Its one reason they've expanded into everything they can get their hands on.
Who remembers Altavista?

Quote:

---

Originally Posted by GGT

We are all armchair-ing. Or regurgitating other "expert analyses". Aren't we?

I've never liked monopolies. Or corporate bullies. Partly why I never owned a pc that was forced to run Microsoft shit, or cell phones that would only operate on mandated platforms or specific providers. Doesn't mean I'm an Apple fanatic either.....I use bing more often than google to search for things.

I bought a new mac because I like macs, but it came loaded with Google Maps and some other stuff that I just won't use.....nothing wrong with partnerships or bundling in principle, but I hate it when using a product assumes things about my preferences. Or worse, when they use one of my choices and 'force' me into using more of their products.

Many of the euro-inspired legal battles are actually good for everyone in the long haul. It's almost amazing that the US bristles at the thought of real competition, since we love to opine about freeeee and open markets, but don't seem very adept at recognizing monopolies when behemoths can buy dominance.

:confused:

---

You do know that Apple is even worse than Microsoft? Only difference: They're not as big now.

Khendra - actually, Apple is bigger than MS by market cap as of about 6 weeks ago.

On the broader issue, I think it's ridiculous. Why should Google's results be 'fair' in some government-regulated sense of the word? They could be much more egregious than currently (right now, sponsored links are clearly delineated, they make no secrets about favoring Google products above other companies, competitors aren't intentionally delisted, etc.), but even if they were I don't get the problem.

The cost of switching search providers is essentially zero (*provided the search is just as good*) so I don't see the case for getting upset over their relative control over the market. Over the last dozen or so years I've used a bunch of different search engines for different purposes - remember back when Altavista had the best media search, or meta-search engines were popular? Currently, I use a few neat features on Bing (especially their travel algorithms which are pretty nice and a good adjunct to Kayak), and generally stay away from Google Scholar, preferring proprietary engines for articles (e.g. Pubmed, SciFinder, etc.). For shopping, I never use Google - I generally use something like Shopzilla. I know a lot of more serious types who don't touch Google News with a ten foot pole, preferring something like Lexis Nexis.

The point is that Google is strong in a few categories because they are the best - Maps, general search, Mail, News (for some), image/video search. But in other categories they simply aren't strong, and aren't the go-to site for people. Why would this be, if Google were really pushing all of their products at the expense of competitors? Because people follow quality.

Quote:

---

Originally Posted by ']['ear

I happen to agree with you, but the language you use is what people are talking about when they say "you've changed." You used to be a reasoned, even-handed poster, and that was why people admired you. But sometime after you finished college you became a knee-jerk soldier in the culture wars. I'm not the only one who noticed this, but I'm sure you'll just disregard me.

Oh, and stop using "felation." You do it incessantly. Cain is not somebody you should emulate, Dread.

---

I used it twice this week, both cases with you and in this case because it sorta rhymes with "regulation". Never used it before. It's amusing that you're so hyper-sensitive about Cain that you see his ghost in someone absentmindedly using a single word when blasting out a post before going to bed.

But did you seriously just say that I'm a "soldier in the culture wars"? I mean...are you fucking kidding me? I can't take you seriously when you say nonsense like that.

Quote:

---

Originally Posted by Aimless

Foundem, like Ciao, aggregates price-data as well as other information in various categories and allows for price comparisons eg. for cameras that meet certain requirements specified by the user. It's a useful service, and I'm not sure why it differs so much from Google's own shopping/price comparison services that it deserves to go from a high ranking to a low ranking and then back up again (which is what they claim has happened for no defensible reason). At least for people in the UK, foundem might actually be useful for finding the best prices.

I personally use pricerunner.se and prisjakt.se very often because they're similar services that are more useful to me than is google, and I see pricerunner-links pop up when I search for pricing info for various products in Swedish. I don't see how Foundem can be very different from those.

---

Fundamentally, a search engine is a ranking. In some respects, every search is a zero-sum game between the sites eligible to appear in the ranking for a given search.

If Foundem was especially useful, it would probably have a better rank for search terms like "camera discounts", etc. One of the big factors in a Google ranking is the quantity and quality of sites linking to a given Web page. Foundem seems to lack that; people aren't asking about cameras on forums and folks are linking them to Foundem.

I'd also like to point out, once again, that for anyone familiar with SEO Foundem is still a crappy site. Here's one take on it: http://econsultancy.com/blog/4456-fo...dy-in-seo-fail