All kinds of stupid
|
| Author | Content |
|---|---|
| jacog Nov 24, 2009 5:29 AM EDT |
It doesn't any sort of economic/marketing genius to figure out who will lose here in the end. I bet a few shareholders are calling up Microsoft, "Steve! Warble Tarble Farble !?" |
| tjhanson9 Nov 24, 2009 8:05 AM EDT |
Yup, Good luck with that, Rupee and Stevie. There exist thousands of sites only too happy to get some attention from an aggregator like Google. |
| bigg Nov 24, 2009 8:44 AM EDT |
I wasn't aware that a news aggregator needed permission to link to stories. |
| montezuma Nov 24, 2009 10:09 AM EDT |
Bigg,
I believe there is a simple device that allows one to ban the google bots from indexing your site. This whole thing smacks of desperation on both the part of MS and News Corp. MS need an angle to get the stagnating Bing up a notch or two against google search and Murdoch sees the Internet as a long term threat to his revenue stream from media outlets. Personally I would bet a lot of money on google winning this battle. There is so much choice out there in news coverage and Murdoch only controls a small fraction of it. He would need to grab a lot more of it for this to be a real threat. |
| bigg Nov 24, 2009 10:28 AM EDT |
> I believe there is a simple device that allows one to ban the google bots from indexing your site. I think that is up to Google. I don't think that legally anything can be done, unless News Corp blocks links from Google. |
| jacog Nov 24, 2009 10:38 AM EDT |
If I were Google, I'd just say "Remove all links to quality news like Fox? Sure! Heck why not!" Just give them what they want, and then wait patietly for the numpties in question to realise how much it hurts them. |
| Bob_Robertson Nov 24, 2009 11:04 AM EDT |
> "Remove all links to quality news like Fox? Sure! Heck why not!" Yep, that's exactly what Google said. "de-indexed on request". I think it's neat that Rupie has somehow figured out how to get Microsoft to pay THEM for blocking Google. Funny, funny, funny. What would be very, very interesting would be to see honest traffic stats before/after Rupie's Minions are removed from Google. |
| tuxchick Nov 24, 2009 11:10 AM EDT |
Bots are supposed to honor whatever you put in your robots.txt file, and if you say "Google begone" then Google begones. And no, nobody needs permission to link to your site, even though twits of various stripes keep trying to make that a requirement. A few years ago this doof kept giving brand x a hard time for linking to his sites (he was an editor or some such), claiming copyright violations and wanting money, so we stopped. Traffic dropped and eventually he was replaced. This story is so many kinds of stupid. So many duhs, so little time. |
| nicsmr Nov 24, 2009 11:24 AM EDT |
> "Remove all links to quality news like Fox? ..." An oximoron... |
| TxtEdMacs Nov 24, 2009 11:43 AM EDT |
[very serious] Bigg, your are wrong. Indeed I was toying with the idea of offering my services to the News Corp to block nearly all* indexing at a low fee. Just rewrite the robots.txt and remove the current permissions to all (the News Corp uses an asterisks) and disallow either all or just name the few you want excluded. Thus, stopping indexing of all sites with the changed file. Simple? Technically, you are correct the robots file can be ignored, however, there could be legal consequences, hence, the more ethical indexers will abide the wishes of the site owner.[/very serious]** YBT * See second paragraph. ** The world is really off kilter when I am forced to write serious content. |
| montezuma Nov 24, 2009 1:47 PM EDT |
> I think it's neat that Rupie has somehow figured out how to get Microsoft to pay THEM for blocking Google. Funny, funny, funny. I agree Bob. Rupert is very clever at shaking people down while they think they just got a good deal. It is the key to his success in the newspaper industry and dealing with politicians. |
| hughesjr Nov 24, 2009 2:13 PM EDT |
Well, it makes sense from this standpoint. Google is making money by providing the links to the user and putting ads on that page ... it is News Corp's content ... they want some of Google's money made from their content. I don't agree with it, but it is a valid point of view. If you want to provide the New York Times in the lobby of your business, you have to BUY it. They don't send a copy to you for free, just so that many people will read it. Same basic principle. This is different in the sense that they are sending you the customer, to your site ... and you can get them to see your ads. But, hey, if they can also get Google to pay them too, then more power to them. It is a legitimate request. |
| tuxchick Nov 24, 2009 2:33 PM EDT |
OMG so fox news will be confined in the Bing ghetto? What a loss to the world!! |
| gus3 Nov 24, 2009 2:47 PM EDT |
Quoting:Google is making money by providing the links to the user and putting ads on that page ... it is News Corp's content ... they want some of Google's money made from their content.Aha, the light just went on. However, due to a caffeine shortage, it isn't at full luminosity, so bear with me. Google isn't making money aggregating and linking News Corp content. They are making money by associating other content with it: their own ads, not under control (or, better put, "in the revenue stream") of News Corp. No wonder Rupert's in a huff. Thanks, hughesjr, for finally arranging the pieces for me. |
| Steven_Rosenber Nov 24, 2009 3:12 PM EDT |
It's pretty easy to tell Google not to index certain content on your Web server. I've had situations where that form of hiding is already on the server and I've had to turn it off. In some instances of Wordpress, for instance, the configuration tells Google not to index images on the site, so the site will be in "regular" Google but not in a Google Images search. This is all tied in to the so-called "pay wall" that many news organizations are considering for their online content. The Wall Street Journal has had such a pay wall for years. Even though Murdoch reportedly considered opening up WSJ when he bought it, that publication has stuck with its pay model. But Murdoch isn't the only one considering a pay wall for at least some online news content. The problem is that Google indeed brings traffic to Web sites, especially news sites, but while that aggregation has made Google very wealthy, the providers of that content are struggling to make any money of their own. Online ad inventory is pretty much limitless, and that has driven prices way down. Bottom line: Most news-producing entities are willing to give up significant audience shares because they're not making money with the model they have, and things like being first, building the brand, etc., just hasn't brought in the numbers. They're willing to try anything, and this is the next thing to be tried. So the majority of these news organizations are not making sufficient amounts of money in the present, and it's not looking like this will turn around in the future. Hence talk of a "pay wall" as both a last-ditch effort to make money online as well as give the existing print product of some of these publications some kind of exclusivity. Murdoch can presumably give up 60 percent of his search traffic if for the remaining 40 percent he can share the search-ad revenue with Microsoft and by taking his content out of Google presumably force it to either lose that money-generating content or come to terms that are more favorable to News Corp. It's the non-secret of Web advertising: Google makes money; it's working in volume/scale that most other Web entities just don't have. |
| Sander_Marechal Nov 24, 2009 6:34 PM EDT |
Quoting:Technically, you are correct the robots file can be ignored, however, there could be legal consequences Microsoft/bing bots regularly ignore robots.txt, fake Bing referrals and do other unethical things. Usually it's blamed on "technical errors" or "quality control" I have blocked all known Microsoft bot IP addresses from my site. It doesn't matter anyway. Before I did that, I got maybe 5 visits a month from Bing (out of ~15.000 unique visits to my site a month). More info: http://www.webmasterworld.com/msn_microsoft_search/3461188.h... (on robots.txt) http://www.the-art-of-web.com/system/logs-bing/ (on referrer spam) If you have a website you can set up a bot trap. E.g. Take an URL on your website that is inaccessible, is not linked to and doesn't even exist, such as http://www.example.org/some-dummy-url. Then, explicitly block that URL in robots.txt. Monitor your weblogs for bots visiting that IP. After all, the only place that URL appears is in robots.txt and it will only get hits from bad bots that explicitly visit URLs forbidden in robots.txt. |
| tuxchick Nov 24, 2009 6:45 PM EDT |
Excellent trick, Sander! I like it :D |
| jdixon Nov 24, 2009 7:04 PM EDT |
> it is News Corp's content ... they want some of Google's money made from their content. And the ads are Google's content, as is the search algorithm which provides links matching your search criteria. News Corp. has no claim on anything. Google is providing direct links to the News Corp. stories, not copying the stories themselves. > ...but it is a valid point of view. It's the point of view of some one whose greed outweighs their common sense. Without the search engine, no on will find your article in the first place. > It is a legitimate request. It's legitimate in the same sense that a panhandler is legitimate. > Murdoch can presumably give up 60 percent of his search traffic if for the remaining 40 percent he can share the search-ad revenue with Microsoft... But can Microsoft make money on the deal? If not, then Murdoch will eventually be out of luck. I strongly suspect the answer to that is no. > ...and by taking his content out of Google presumably force it to either lose that money-generating content or come to terms that are more favorable to News Corp. And I think it's quite clear which one Google is going to choose. There will be plenty of external links to News Corp. stories which won't block Google, and there's not much Murdoch and do about those. > ...Microsoft/bing bots regularly ignore robots.txt, Why am I not surprised? The words Microsoft and ethics have always been contradictory. |
| softwarejanitor Nov 24, 2009 7:11 PM EDT |
@Sander_Marechal Just one more reason to Boycott Bing... Imagine that... Microsoft doing unethical things. Who'd have thought? |
| bigg Nov 24, 2009 7:22 PM EDT |
About making money on someone else's information. If that's a valid argument, when will News Corp start compensating everyone in their news stories? And if selling information is wrong, should it be illegal for someone on the street in Chicago to sell directions to the Sears Tower? |
| flufferbeer Nov 24, 2009 7:23 PM EDT |
@Sander n' jdixon,
> ...Microsoft/bing bots regularly ignore robots.txt
I agree!
I think that once this robots.txt issue is more widely publicised or is brought to the attention of the M$ higher-ups, the Micro$uck$ tech folks'll pay attention to robots.txt. They'll first fully-adhere to the standard, then they'll enact some other robots.txt replacement (e.g., anyone remember the LMHOSTS text file as a replacement for the hosts text file for their proprietary NetBIOS ID on Losedow$ ?). Once the M$ borg wheedle their own replacement for robots.txt into a hard-and-fast standard, then its bye-bye robots.txt! Pity on News Corp, and its staff for losing out in the end whatever happens. OTOH, I don't care too much for what happens to M$ from this deal. 2c |
| hkwint Nov 24, 2009 7:30 PM EDT |
Quoting:Microsoft/bing bots regularly ignore robots.txt Following that comment and also the "IP bocking remarks" which were also made on my local forum, a discussion of "P2P distributed web crawlers" emerged. Turned out it already exists since 2000 on grub.org. Who'd have thought? Conclusion: If it worked, we don't need no stinking company to crawl the web, in pretty much the same way we don't need no stinking corporation to write an OS (it helps though, I'd have to admit). |
| Sander_Marechal Nov 24, 2009 7:35 PM EDT |
Quoting:I think that once this robots.txt issue is more widely publicised or is brought to the attention of the M$ higher-ups, the Micro$uck$ tech folks'll pay attention to robots.txt. Nope. especially the robots.txt issue is already very old and widely-known. MS has been at it back even before Live.com when it was called MSN Search. The referrer spam is a newer issue. MS claims to do it to ensure that websites show the same content to their search bots as they do to normal visitors. That it happens to inflate (in my case by several thousand percent) the stats for Bing is purely "accidental" ...[insert roll-eyes emoticon] PS: Google has been checking the same thing for years... without inflating stats. |
| flufferbeer Nov 25, 2009 1:40 AM EDT |
> Nope. especially the robots.txt issue is already very
> old and widely-known. MS has been at it back
> even before Live.com when it was called MSN Search. @Sander, Yep to your nope. Where I'll disagree with you is M$'s "sudden" change to a new standard once they see everyone else uses it. Another example I easily recall of this particular thing is M$'s getting mostly rid of their own NetBIOS/NetBeui and finally going straight to all-out use of TCP/IP. Sure NetBeui isn't routable, but even so, M$ kept this NetB* LAN-segmented peer-to-peer protocol around far longer than it was worth in the face of the burgeoning Internet. Where I'll agree with you, is that if M$ cannot change or be the leader in a technology, then it is prone to downright THWART it. This whole news story gets to this key desire of the Borg -- that last E of EEE ! fb |
| Bob_Robertson Nov 25, 2009 9:23 AM EDT |
> OMG so fox news will be confined in the Bing ghetto? What a loss to the world!! One can only hope that CNN and the New York Times follow suit. I like bias right out front where I can see it, so I can balance the materials and the source myself. What I loath are "news" sources which try to hide their bias, claiming to be "fair" and "inclusive" while actually being neither. |
| dinotrac Nov 25, 2009 9:56 AM EDT |
Bob! Shame on you! CNN and the New York Times not completely and utterly impartial? Next you'll be saying the same of the Boston Globe (part of the Times rotten empire) and MSNBC might tilt a bit in their coverage. That liberal bias thing is cr@p. Just ask any Times, Globe, MSNBC, or CNN reporter. You can even ask their Republican reporter (the one shared jointly by all of them so they can be impartial without actually having to have a Republican on staff). |
| jdixon Nov 25, 2009 10:31 AM EDT |
> That liberal bias thing is cr@p. Just ask any Times, Globe, MSNBC, or CNN reporter. Soon to be verified by an in depth PBS study. |
Posting in this forum is limited to members of the group: [ForumMods, SITEADMINS, MEMBERS.]
Becoming a member of LXer is easy and free. Join Us!