Duncan Riley has been doing a good job of trying to help the online newspapers stop naughty Google from stealing their precious content. As Duncan and others have pointed out regularly, preventing Google from profiting from their news is quite simple, it’s called Robots.txt and it’ll stop Google and other search engines dead in their tracks.
That’s right. Google will not index sites that have a valid Robots.txt file. What’s more, creating a Robots.txt file is very simple. So simple in fact that Ropert Murdoch, or indeed the most challenged of computer users could create one and upload it to their website.
But of course, the newspapers won’t do this. If they were to block Google then they would lose a significant portion of traffic. Yes, you see, these newspapers have a dirty little secret. Google is actually good for them. It drives traffic to their websites, which means they can serve up more ads, and make more money.
The newspapers appear to want to be paid some sort of royalty-free for allowing their news to be indexed by Google, and then they want to continue to make the money from ads which they already get from Google traffic.
Can’t. Have. Cake. Eat. Too.
The irony is of course that if Google actually went ahead and decided not to index content from these websites — to guard against any potential legal action, or to simply make a point — the newspapers would be up in arms about how evil Google is trying to suppress the free press… or something along those lines.