en:user_advanced:oai
Differences
This shows you the differences between two versions of the page.
Next revision | Previous revisionNext revisionBoth sides next revision | ||
en:user_advanced:oai [2014/11/03 00:27] – external edit 127.0.0.1 | en:user_advanced:oai [2018/07/30 23:12] – [Downloading on the command line] kjdon | ||
---|---|---|---|
Line 46: | Line 46: | ||
then Greenstone downloads the page and scans though it looking for '' | then Greenstone downloads the page and scans though it looking for '' | ||
- | ===== Serving OAI Data using Greenstone===== | + | ==== Downloading |
- | + | ||
- | Greenstone comes with a built-in OAI data provider, called **oaiserver**. A configuration file provides options for the set up of the server. Collections can opt in or out of the server, and each collection will be advertised as an OAI set. Multiple collections can be grouped into a single OAI set using Greenstone' | + | |
- | + | ||
- | <TABAREA tabs=" | + | |
- | < | + | |
- | ====The Greenstone3 OAI server==== | + | |
- | The Greenstone3 OAI data provider facility comes enabled by default. It runs as a servlet called " | + | |
- | by replacing library with oaiserver. For example, ''< | + | |
- | + | ||
- | ==== Configuration ==== | + | |
- | Configuration is done via the files: **'' | + | |
- | + | ||
- | ===OAIConfig.xml=== | + | |
- | + | ||
- | This file specifies general information about the repository and can be found in '' | + | |
- | + | ||
- | Please modify this file and enter the correct values for repositoryName and repositoryIdentifier. | + | |
- | Other values may be modified as needed. The following table lists important configuration | + | |
- | options in '' | + | |
- | + | ||
- | ^ Configurations in OAIConfig.xml | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | |''< | + | |
- | + | ||
- | + | ||
- | ===collectionConfig.xml=== | + | |
- | Resides in the ''/ | + | You can also download OAI records on the command line, using the perl script that GLI uses in the background: **downloadfrom.pl**. There is lots of information about command line downloading on [[en: |
- | If the collection is part of a super set, then this information is added here too. | + | |
- | Here is a sample OAIPMH ServiceRack element. | + | * **Set up the Greenstone environment** in the terminal by running one of the following: |
< | < | ||
- | < | + | source gs3-setup.sh (linux/MacOS, gs3) |
- | < | + | gs3-setup (Windows, gs3) |
- | < | + | source setup.bash (Linux/MacOS, gs2) |
- | <!-- States that this collection is part of the humanity super set, which needs to be defined | + | setup (Windows, gs2) |
- | in the OAIConfig.xml file. --> | + | |
- | < | + | |
- | < | + | |
- | <!-- This collection supports the DC metadata set. --> | + | |
- | < | + | |
- | < | + | |
- | <!-- this will replace the dc: | + | |
- | <element name=" | + | |
- | <mapping elements=" | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
</ | </ | ||
- | ==== OAI super sets ==== | + | * **To see the options available**, run: |
- | + | ||
- | In the Greenstone OAI server, each collection is presented as an OAI set. You can use the oaiSuperSet to group several collections together to be presented as a single set. | + | |
- | + | ||
- | The format for a super set specification is like the following: | + | |
< | < | ||
- | < | + | perl -S downloadinfo.pl OAIDownload |
- | < | + | |
- | < | + | |
- | < | + | |
- | </ | + | |
</ | </ | ||
- | There can be more than one super set specified in OAIConfig.xml. Collections themselves state which super set they belong to. The format is the following, where xxx must match the setSpec of the set it belongs to. This line must be added into the OAIPMH serviceRack element | + | The options are the same as you can see in the GLI OAI download panel. They are: |
- | <code> | + | * **-url |
- | <oaiSuperSet name=" | + | * **-metadata_prefix |
- | </ | + | * **-set < |
- | ==== Metadata Formats ==== | + | * **-get_doc**: |
- | + | * **-get_doc_exts < | |
- | A repository must support the dublin core format, and may support others. Each format must be listed | + | * **-max_records < |
+ | ***An example usage would be:** | ||
< | < | ||
- | < | + | perl -S downloadfrom.pl -mode OAI -url http://www.nzdl.org/cgi-bin/oaiserver.cgi -set demo -max_records 5 |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
</ | </ | ||
- | The elementList lists all elements for the metadata format. Mapping rules dictate which elements in the greenstone collection get output for these OAI elements. | + | This will try to download 5 records from the set //demo// at the nzdl.org' |
- | The OAIConfig.xml file defines | + | The records (and optionally documents) will be downloaded into the folder |
+ | ===== Serving OAI Data using Greenstone===== | ||
- | < | + | Greenstone comes with a built-in OAI data provider, called |
- | < | + | |
- | < | + | |
- | < | + | |
- | <element name=" | + | |
- | <mapping elements=" | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | The collection can specify a custom mapping. | + | |
- | + | ||
- | Field mapping is done in two levels: globally for all collections in a repository, | + | |
- | and specifically for each collection. For a particular collection, the mapping specification | + | |
- | in the '' | + | |
- | the metadata mappings will be first looked for in each collection' | + | |
- | if not found, go to the '' | + | |
- | field names will be used to retrieve the metadata of the collection. | + | |
- | + | ||
- | Mappings take the following format: | + | |
- | < | + | |
- | <element name=" | + | |
- | <element name=" | + | |
- | elements=" | + | |
- | </ | + | |
- | + | ||
- | In the first case, the server will look for ' | + | |
- | In the second case, the server will look for any metadata in the elements list and map it to oai-name in the output. The select attribute determines how many values are output. The default setting is ' | + | |
- | | + | |
- | * **firstvalue**: | + | |
- | * **firstvalidmetadata**: | + | |
- | + | ||
- | Some examples: | + | |
- | + | ||
- | < | + | |
- | The server will look for dc:title metadata and output it if found. Note that standard Greenstone metadata uses ' | + | |
- | < | + | |
- | This will output all dc.Titles as dc:title metadata. If no dc.Title is found, then any Titles will be output. | + | |
- | < | + | |
- | This will output all dc.Dates and gs.Dates as dc:date. | + | |
- | + | ||
- | + | ||
- | + | ||
- | ====Disabling Collections==== | + | |
- | + | ||
- | By default, in Greenstone3, | + | |
- | A new collection in Greenstone contains the OAIPMH ServiceRack in its collectionConfig.xml | + | |
- | The collectionConfig.xml file lives in // | + | |
- | + | ||
- | ==== Disabling the OAI Server ==== | + | |
- | + | ||
- | If you do not want to provide an OAI server | + | |
- | Make sure Tomcat is not running (by closing the Greenstone3 server program, or running **ant stop** | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | </ | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | </ | + | |
- | + | ||
- | The second part to remove or comment out is the servlet mapping, which maps the url to the servlet. | + | |
- | It looks like this: | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | Once these two sections are gone or commented out, Tomcat will no longer provide the oaiserver. You can add them back in if you wish to reinstate it later. | + | |
- | + | ||
- | ==== OAI Datestamps ==== | + | |
- | + | ||
- | The " | + | |
- | + | ||
- | To manually | + | |
- | + | ||
- | ==== Resetting the server ==== | + | |
- | + | ||
- | If you have rebuilt | + | |
- | .</ | + | |
- | <!-- ####################################################################################### | + | |
- | ########################################################################################### | + | |
- | ########################################################################################## | + | |
- | < | + | |
- | ====The Greenstone2 OAI server==== | + | |
- | Greenstone comes with a built-in | + | |
- | '' | + | |
- | It can be accessed via the same URL as the Greenstone library (replacing | + | |
- | '' | + | |
- | + | ||
- | Configuration of the server is done via the '' | + | |
- | + | ||
- | Please edit '' | + | |
- | If you are not using the standard Apache setup that comes with Greenstone, | + | |
- | you may need to set oaiserverPath, | + | |
- | you can set baseServerURL to use a domain name instead of IP address in URLs. | + | |
- | + | ||
- | By default, collections are not accessible. To enable a collection, add its name to the '' | + | |
- | + | ||
- | Greenstone' | + | |
- | RFC1807 metadata. For collections that use other metadata sets, including extracted | + | |
- | metadata, metadata mapping rules should be provided to map the existing metadata | + | |
- | to Dublin Core. '' | + | |
- | + | ||
- | ==== To add a new metadata set for use with oaiserver ==== | + | |
- | You need to do the following: | + | |
- | | + | * [[oai_server_gs2|Greenstone 2 OAI server setup and configuration]] |
- | * Put the new schema somewhere web accessible | + | |
- | * Coding in GSDLHOME/ | + | |
- | * Create a new metaformat class for the metadata set. See dublincore.h/ | + | |
- | * edit Makefile.in, | + | |
- | * Edit recordaction.cpp to include the new header file and instantiate the new class (in recordaction()) | + | |
- | * Tell the server to use the new set: edit etc/oai.cfg and add the set name to the oaimetadata line. You may also need to add oaimapping information. | ||
- | * Recompile and test. | ||
- | </ | ||
====Testing==== | ====Testing==== |
en/user_advanced/oai.txt · Last modified: 2023/03/13 01:46 by 127.0.0.1