This version (2014/04/14 11:52) is a draft.
Approvals: 0/1

Z39.50

Z39.50 is an international client/server protocol for searching bibliographic data. It can use the Internet Protocol (TCP/IP), which makes the databases on a server available from almost anywhere around the globe. It is widely used, for example, in on-line library catalogues. It allows a user to search one or more databases and retrieve the results of the query.

Greenstone has support for Z39.50, both as a client and a server. GLI can download MARC records using Z39.50 and MARCXML records using SRU. These records can then be included in a collection. This support is enabled by default.

The Greenstone run-time can also act as a client to multiple Z39.50 servers, and a Z39.50 server program is also available. This support is not enabled by default, and recompilation is needed to enable it. To do this, you need the source code. If you don't already have the source, download the version of the source code component that matches the distribution you are using.

Z39.50 support in Greenstone is based around the YAZ toolkit, written by IndexData. We are currently using YAZ version 2.1.4. This is included in the Greenstone distribution without modification. Greenstone links against the libyaz.a library. We have also written yaz_zclient.h/cpp, which is based on the sample client.h/cpp. This is found in greenstone/runtime-src/src/z3950.

Download through Z39.50

See the download page for general information on downloading records through Greenstone.

Greenstone can download records from a specified Z39.50 server from the GLI (from the Download panel) or the command line (using the downloadfrom.pl script). Either way, you are presented with five options:

ArgumentDescription
Host (-host)(REQUIRED) name of the Z39.50 server, for example z3950.loc.gov
Port (-port)(REQUIRED) port number of the server, for example 7090
Database (-database)(REQUIRED) name of the database to which the query will be sent
Find (-find) (REQUIRED) query (see below)
Max records (-max_records)Select to limit the size of result set; Default: 500

If downloading via the GLI, you can view the downloaded MARC files on the Gather panel. On the left-hand side of the panel, double click the Downloaded Files folder to expand its content. The subfolders are named by the Z39.50 server url. The MARC files are named as the combination of database name, query, and max_records if max_records is specified. These MARC files are physically stored in a temporary cache directory. You can build a collection using these downloaded MARC files by dragging them across to the Collection section on the right-hand side of the Gather panel.

If downloading via the command line, a folder containing the files will be downloaded into the directory from which you run the script (unless the -cache_dir <full-path-to-folder> option is set).

As the files downloaded via Z39.50 are in MARC format, they can be processed by MARCPlugin, which must be included in the collection plugin list.

Querying in YAZ

The YAZ client supports the type-1 RPN query model. Both simple and advanced queries are supported, like single term query, boolean search and fielded search, etc. This page describes in detail the query syntax of YAZ. For example, the following query finds records that have the phrase car history in title field:

 @attr 1=4 "car history"

Download through SRU

SRU (Search/Retrieve via URL) is an alternative method that uses a web service to search and retrieve records from Z39.50 repositories. It replaces the Z39.50 communications protocol with HTTP, but still supports the Z39.50 query syntax. Search results from SRU are in XML format. (One method of SRU, SRU VIA HTTP SOAP, was previously referred to as SRW. This name is now deprecated, however some Greenstone scripts have not yet been updated and the name SRW is still used.)

Greenstone also supports downloading from a Z39.50 server through SRU. If using the GLI, select SRU on the Download panel. The options for SRU are the same as for Z39.50 download, but different host and port values should be used here. For example, to connect to the Library of Congress Z39.50 server through SRU, the following host and port should be specified:

 host: http://z3950.loc.gov
 port: 7090/voyager?

If downloading via the GLI, you can view the downloaded records on the Gather panel. They are in the same subfolder as downloading from Z39.50 above.

If downloading on the command line using downloadfrom.pl, the -download_mode is SRW, and a folder containing the downloaded files will be created in the current directory.

The downloaded records are in XML format. Here is a sample record.

To build a collection using the downloaded XML files, MARCXMLPlugin must be added to the collection plugin list.

Compiling Greenstone with Z39.50 support

  • Linux/MacOS: in the top level greenstone directory, run
 ./configure --enable-z3950 (--enable-apache-httpd)
 make
 make install
  • Use the enable-apache-httpd if you have been using or want to use the apache built in to Greenstone.
  • Windows:
    • needs Visual Studio 6
    • It only works with the web library, not the local library
    • In the runtime-src\packages\yaz directory, extract the files from yaz-2.1.4.tar.gz. You'll need to run the extraction twice, once for gzip, once for tar. You should end up with a yaz-2.1.4 directory here.
    • Edit the yaz-2.1.4/include/yaz/oid.h file and change "list" on line 256 to "greenstone_list".
    • Edit the runtime-src\src\z3950\z3950proto.cpp file and delete "extern" from lines 38 and 39 (yyin and yyout).
    • in the greenstone directory, run
 nmake /f win32.mak USE_Z3950=1
  • To enable iconv or xml2, edit greenstone\runtime-src\win32.mak, and remove HAVE_ICONV=0 and/or HAVE_LIBXML2=0 from the make command for Yaz. You'll need to install these libraries.

Using the Z39.50 client

Once Greenstone has Z39.50 support compiled in, it can act as a client to multiple z39.50 servers. The file

specifies a list of servers to connect to. By default, no servers are set up, although the config file comes with one (commented out) example Z39.50 server, for the United States' Library of Congress.

Each entry consists of:

  • A unique "short name" for internal use by Greenstone. (Note, this should not be the same as any local collection short name.)
  • The internet name or address of the server, and optionally the port that the server is running on if not the default port 210.
  • The name of the database to search on that server.
  • A string that provides a meaningful name for the "collection".
  • An optional "About" string, providing some information about the database and/or server.
  • Optional icon fields, which are displayed instead of the text on the front page.

The entries need only be separated by whitespace, but for the purposes of clarity the sample entry uses newlines and tabs.

There is a list at the Library Of Congress website containing some servers publicly available for testing.

Greenstone will display a new "collection" for each server listed.

Using the z3950 server

  • The z3950server program is installed into bin/linux (or bin/darwin, bin/windows, depending on the operating system). It can be run from there, or copied to somewhere else.
  • The gsdlsite.cfg file needs to be copied from the cgi-bin directory to the directory you are running z3950server from. Check that the 'gsdlhome' entry is valid. The other entries don't matter.
  • Before running the server, set up your Greenstone environment in the terminal by running the following in your top level Greenstone directory:
  • By default, the server listens on localhost:9999. To change the port or address, run
 z3950server tcp:server-name:port-num
  • For example, z3950server tcp:localhost:8080 or z3950server tcp:kanuka.cs.waikato.ac.nz:7070
  • On Windows, the yaz.dll file (runtime-src/packages/yaz/yaz-2.1.4/bin/yaz.dll) needs to be on your Path, or put in the same directory as the z3950server.exe.
  • You can run z3950server -h to see the list of options for the server.

Testing the server

You can test the z3950 server by connecting to it using the Greenstone z3950 client. In the greenstone/etc/packages/z3950/z3950.cfg file, add a server entry similar to the following:

 zdemo
        kanuka.cs.waikato.ac.nz:9999
        demo
        "The demo collection via z3950"
        About "This collection contains a few records from the Humanity Development Library"
  • The database name (line 3) specifies which Greenstone collection to search.
  • This works with MG, MGPP and Lucene collections.

Known Problems/Issues list

  • Because of the open nature of the standard, our client may not work with some servers.
  • Because of the large number of MARC fields, only the most frequently used fields have been given explicit names in the results. Furthermore, these are currently hard-coded to correspond to the USMARC field names.
  • The Z39.50 client will not work when fast-cgi is used. Currently, the use of the fast-cgi package (off by default) will disable the use of the Z39.50 client code for Greenstone.
  • When viewing a Z39.50 "collection" in Greenstone, sometimes the browser may insert a large amount of space between the collection title and the navigation bar. Reload or Shift Reload should fix this problem.