redho home | products | services

Programming Forums


Community for Java, PHP, Perl C, ASP and Python programmers

Ask your IT question here

Programming Forums
FAQFAQ MemberlistArchive Log inLog in RegisterRegister

how dogpile is legal


Goto page 1, 2  Next
 
Post new topic   Reply to topic    Programming Forums -> General Web Design discussion
View previous topic :: View next topic  
Author Message
Mupolan
Enthusiastic Coder


Joined: 30 May 2006
Posts: 101
Hey, does anyone know how dogpile is legal, since google doesn't allow itself to be spidered?

Reply with quote
 
autos06
Enthusiastic Coder


Joined: 13 May 2006
Posts: 123
Location: New York
Mupolan, it just does a HTTP request to google?
Not that hard to pretend to be a browser :d
And holy crap http://www.dogpile.com/info.dogpl/search/web/foo%252Bbar/1/-/1/-/-/-/1/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/417/top/-/-/-/1
Little much with the URL parameters Neutral
Reply with quote
 
Mupolan
Enthusiastic Coder


Joined: 30 May 2006
Posts: 101
Google doesn't allow itself to be spidred
Could't they stop dogpile?
Reply with quote
 
autos06
Enthusiastic Coder


Joined: 13 May 2006
Posts: 123
Location: New York
What do you mean "doesn't allow itself to be spidered"
Reply with quote
 
Mupolan
Enthusiastic Coder


Joined: 30 May 2006
Posts: 101
Robots.txt
Reply with quote
 
autos06
Enthusiastic Coder


Joined: 13 May 2006
Posts: 123
Location: New York
So spoof the useragent?
Reply with quote
 
Mupolan
Enthusiastic Coder


Joined: 30 May 2006
Posts: 101
They could sue dogpile
Reply with quote
 
autos06
Enthusiastic Coder


Joined: 13 May 2006
Posts: 123
Location: New York
How so?
Reply with quote
 
Mupolan
Enthusiastic Coder


Joined: 30 May 2006
Posts: 101
With lawyers
Reply with quote
 
autos06
Enthusiastic Coder


Joined: 13 May 2006
Posts: 123
Location: New York
:d
They put their content on the internet, public domain
Reply with quote
 
Page 1 of 2 Goto page 1, 2  Next
Post new topic   Reply to topic    Programming Forums -> General Web Design discussion


Dubai Forums - Expat Help | Vegan Chat | Java Programming | Free 3D tutorials and 3d textures | Paris Forum | EU Forum
Free Dubai Classifieds | Free London Classifieds | Jobs in London

High Quality, Custom 3d animation and Web Design solutions Royal Quality Web Hosting Services Vegetarian and Animal Rights news

Powered by phpBB © 2001, 2005 phpBB Group