Map version

 

Global harvesters (other than Web search engines)

 

OAIster

http://www.oaister.org/index.html

Developed at UMichigan (originally in collaboration with U.Illinois). Harvests from over 1000 source collections and has almost 20 million records. Anticipates a number of enhancements including Google-like search, browse by subject, access to duplicates (provides access to all potential duplicates; attempt to discover and remove duplicates during ingest) and automated clustering of subject metadata in the test search interfaces. Discloses deep web resources.

 

BASE (Bielefeld Academic Search Engine)

(http://base.ub.uni-bielefeld.de/index_english.html)

Developed at the University of Bielefeld. Harvests from over 1000 source collections and has almost 16 million records. Anticipates a number of enhancements including browse function, provision of HTTP and SOAP interfaces and the inclusion of more source collections.

ScientificCommons

 

http://en.scientificcommons.org/about

Developed and hosted at the University of St.Gallen. Has around 13 million records and 6 million author names. Indexes full-text as well as metadata. Uses lexical and statistical tools to analyse metadata and develop keywords. Semantic processing of records to subject areas using Ontologys.

 

Regional harvesters

 

DRIVER Portal

http://search3.driver.research-infrastructures.eu/webInterface/simpleSearch.do;jsessionid=DDE0842B90CD8F2279448E0B129B75CB?

DRIVER is an EU-funded project aiming to facilitate the establishment of repositories in research-based institutions across Europe. It has established a set of guidelines (http://www.driver-repository.eu/DRIVER-Guidelines.html) offering a best practice tool and streamlining repository developments in Europe. Repositories register with DRIVER for harvesting: DRIVER currently provides a search service across around 170 repositories.

 

eIFL Portal

http://eifl.cq2.org/en/page/page.view/eifl.page

eIFL (Electronic Information for Libraries) supports and advocates for the wide availability of electronic resources for users of libraries in transitional and developing countries. It provides a portal through which the 100+ OA repositories in the member countries can be searched.  

 

DART Europe

/www.dart-europe.eu/basic-search.phpbtopic 

Partnership of research libraries providing European portal for electronic theses and dissertations. Is the European Working Group of NDLTD.

 

National harvesters

The number of national-level harvesting initiatives is growing. They are usually funded either by government or by the national library in each country. Some are still at project/pilot level and funding for a sustainable future is not secure in all cases. The main initiatives (there may well be more in development) are listed below and shown on the diagram.

 

ARROW (Australian Research Repositories Online to the World) Discovery Service

http://search.arrow.edu.au/

Developed by the National Library of Australia, ARROW harvests from 28 university repositories and a further 12 digital collections.

 

BUSCAR repositorios

http://www.accesoabierto.net/repositorios/default.php

Searches over Spanish institutional repositories. Operates via the Recolecta service (see below).

 

CARL/ARBC Metadata Harvester

http://carl-abrc-oai.lib.sfu.ca/

Canadian Association of Research Libraries. Harvests from 10 university repositories (2 of these are e-theses collections).

 

DRIVER Belgium

http://www.driver-repository.be/

The Belgian harvester built using DRIVER guidelines. Harvests Belgian university repositories.

 

DDF (Danish National Research Database)

http://www.forskningsdatabase.dk/About.html

Government-funded and part of the Danish Electronic Library (DEFF). Harvests OA repositories where present; focused so far mainly on institutional CRISes.

 

e-Ciencia

http://www.madrimasd.org/informacionIDI/e-ciencia/

Madrid region harvester for universities in the region and the CSIC (Spanish National Research Council) repository.

 

Intute Repository Search

http://www.intute.ac.uk/irs

JISC-funded project (until July 2009) developed by MIMAS and UKOLN. Harvests and searches across UK repositories. Developing additional semantic search capabilities including metadata clustering.

 

IRIScotland

http://cdlr.strath.ac.uk/iriscotland/

JISC-funded project (completed 2008) that produced a pilot search service working on 7 Scottish university repositories.

 

Irish National Research Platform

http://www.researchplatform.ie/

Launched mid-2008 as a feasibility project to harvest from Irish institutional repositories. Will also provide the base for 'research assessment, bibliometric analysis and benchmarking'.

 

JAIRO

http://jairo.nii.ac.jp/en/

Japanese national portal developed and maintained by the National Institute of Informatics(NII). Harvests from all Japanese institutional repositories and currently has more than 570,000 metadata records compliant with "junii2", which is the de facto standard used by Japanese repositories and was developed by the NII. Consistent with the Dublin Core Element Set. http://www.nii.ac.jp/irp/en/system/junii2_en_20090213.xls

In addition to JAIRO, some subject-based national harvesters are in service: DML-JP(http://dmljp.math.sci.hokudai.ac.jp/), 'Repository of Archaeological Reports', 'Education Subject Repository', 'Open access and bi-directional repository for medical science', etc.

 

NARCIS

http://www.narcis.info/background

Incorporates DAREnet (repositories of the Dutch universities and research organisations), Cream of Science and Promise of Science (theses) alongside the national research database NOD.

 

NORA

http://www.ub.uio.no/nora/noaister/topic.html?siteLanguage=eng

Government-funded but future unclear as the Government ministry involved has declined to continue funding to support NORA in 2009. Uses a national metadata standard (OAI-compliant) at present. Aiming to convert Norwegian national metadata to DRIVER standards. Intends NORA to be the single harvesting point for international services wishing to index Norwegian research.

 

OASIS.br

http://oasisbr.ibict.br/

Brazil's national service. Government-funded portal for Brazilian institutional repositories.

 

PLEIADI (Portale per la Letteratura scientifica Elettronica Italiana su Archivi aperti e Depositi Istituzionali

http://www.openarchives.it/pleiadi/index.php?sel_lang=english

The portal for Italian university repositories. Partner to PUMA in presenting a national search service for the OA literature.

 

PUMA

http://puma.isti.cnr.it/index.php?langver=en

Harvester for the 24 institutes of the Italian National Research Council (CNR). Partner to PLEIADI.

 

Recolecta

http://search.recolecta.driver.research-infrastructures.eu/

The Spanish service harvesting from institutional repositories. It currently harvests from 25 repositories, of which 14 are e-theses collections.

 

Repositório Científico de Acesso Aberto de Portugal

http://www.rcaap.pt/about_en.jsp

Portuguese national harvester. Harvests from 12 Portuguese university repositories.

 

SwePub

http://www.ub.gu.se/swepub.se/english/

Swedish national harvester developed by Universities of Uppsala and Gothenburg and the National Library of Sweden. Harvests from institutional publication databases (generally metadata-only records) and OA repositories where available. Provides metadata for harvesting by other services. Aim is to integrate it with the national bibliographic service LIBRIS (http://libris.kb.se). The requirements for this national service have been specified: the beta release is expected at the end of April 2009 and the final release in September 2009.

 

Notes

Google has recently changed the way it indexes repositories since it has dropped support for OAI when it indexes websites. It also appears to miss a considerable proportion of the hidden web. See:

Hagedorn K and Santelli J (2008) Google still not indexing hidden web URLs.  http://www.dlib.org/dlib/july08/hagedorn/07hagedorn.html [Google is 'missing' 55% of records in repositories]