redho home | products | services
Web Design Forums

Web Design Forums  


Web Hosting, Web Design, Software and Web Development Forums  
 FAQFAQ   MemberlistArchive  Log inLog in   RegisterRegister 
         

how dogpile is legal


Goto page 1, 2  Next
 
Post new topic   Reply to topic    Web Design Forums -> General Web Design discussion
View previous topic :: View next topic  
Author Message
Mupolan



Joined: 30 May 2006
Posts: 101
Hey, does anyone know how dogpile is legal, since google doesn't allow itself to be spidered?
  Reply with quote


autos06



Joined: 13 May 2006
Posts: 123
Location: New York
Mupolan, it just does a HTTP request to google?
Not that hard to pretend to be a browser :d
And holy crap http://www.dogpile.com/info.dogpl/search/web/foo%252Bbar/1/-/1/-/-/-/1/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/417/top/-/-/-/1
Little much with the URL parameters Neutral
  Reply with quote
Mupolan



Joined: 30 May 2006
Posts: 101
Google doesn't allow itself to be spidred
Could't they stop dogpile?
  Reply with quote
autos06



Joined: 13 May 2006
Posts: 123
Location: New York
What do you mean "doesn't allow itself to be spidered"
  Reply with quote
Mupolan



Joined: 30 May 2006
Posts: 101
Robots.txt
  Reply with quote
autos06



Joined: 13 May 2006
Posts: 123
Location: New York
So spoof the useragent?
  Reply with quote
Mupolan



Joined: 30 May 2006
Posts: 101
They could sue dogpile
  Reply with quote
autos06



Joined: 13 May 2006
Posts: 123
Location: New York
How so?
  Reply with quote
Mupolan



Joined: 30 May 2006
Posts: 101
With lawyers
  Reply with quote
autos06



Joined: 13 May 2006
Posts: 123
Location: New York
:d
They put their content on the internet, public domain
  Reply with quote
Page 1 of 2 Goto page 1, 2  Next
Post new topic   Reply to topic    Web Design Forums -> General Web Design discussion


Dubai Forums - Expat Help | Vegan Chat | Java Programming | Free 3D tutorials and 3d textures | Paris Forum | EU Forum
Free Dubai Classifieds | Free London Classifieds | Jobs in London

High Quality, Custom 3d animation and Web Design solutions Royal Quality Web Hosting Services Vegetarian and Animal Rights news

Powered by phpBB © 2001, 2005 phpBB Group