> Home > Downloads > Search Engine Wrapper

Search Engine Wrapper

This package provides a Java wrapper framework for unifying programmatic access to search engines. A convenience class is also included for downloading the files at the URLs in the search engine results. This package contains an API as well as a command-line application.

The search engine wrapper has the following key features. Firstly, as search engines place restrictions on the number of results they return per request, the search engine wrapper automatically issues as many requests as needed to obtain the required number of results for a particular query string. Secondly, when transient errors such as network errors and server reboots occur, the search engine wrapper will retry search engine queries as many times as needed to obtain the results. The waiting time between successive retries is determined by an exponential backoff mechanism. Thirdly, for search engines that requires a key to function, the search engine wrapper can manage a pool of keys to be rotated.

Currently, this package contains clients for the following search engine APIs:

The search engine wrapper framework is designed such that additional clients for other search engines can easily be integrated into this framework.

Oracle Java 1.6 is required to use this package.

License

The search engine wrapper is open source. Releases made on or after 2013 April 23 are licensed under the Apache License, Version 2.0. Releases made before 2013 April 23 are licensed under the GNU General Public License, either Version 3 or (at your option) any later version.

Status

This package is under general release.

Please note that the quality of the results returned by the search engine providers are not within our control.

Users may encounter problems when parsing XML documents. This is likely due to having a buggy XML parser installed or included in the classpath. A possible workaround is to set the system property javax.xml.parsers.DocumentBuilderFactory to "com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl".

Downloads

Release: 2013 June 18

This release incorporates the following changes:

  1. Adds support for Facebook Search.

Release: 2013 May 27

This release incorporates the following changes:

  1. The Twitter Search client is improved with various bug fixes and enhancements. In particular, if a Twitter status text is truncated and it is a retweet, then an attempt is made to reconstruct the complete status text from the original tweet.
  2. This release includes updated components.

Release: 2013 April 26

This release incorporates the following changes:

  1. Adds support for DuckDuckGo Search and Wikipedia Search.

Release: 2013 April 23

Please note that the API related to the search engine clients has changed in a non-backwards compatible manner. If you are calling the search engine clients directly in your code, then this release is not a drop-in replacement.

This release incorporates the following changes:

  1. License is changed to Apache License, Version 2.0.
  2. The API for search engine clients have changed to better support Twitter-style search engine APIs.
  3. Because all search engine APIs have changed since the previous release, all search engine clients have been reimplemented to use the newest available search engine APIs.
  4. Twitter Search client is no longer experimental, but support for Yahoo! Search is dropped (because the option for programmatically calling Yahoo! Search at zero monetary cost no longer exists).
  5. This release includes updated and replaced components.

Earlier releases

Click here to show earlier releases.

Development Snapshot

A development snapshot can be obtained from GitHub.

Development snapshots are unstable, may contain bugs, and might not even compile. It is highly recommended to download the latest release instead.