Lab 2: Adding metadata—and using it

In the Librarian Interface, open up the reports collection that you created in exercise 1.5. Remember that the extracted Title metadata for some documents was incorrect.

If you build and preview your collection at this point, you will see that the Titles list now shows your new Titles. However, the dc.Creator metadata is not displayed. You need to alter the collection design to use this metadata.

The next section is Partition Indexes. In this exercise, we will not make any changes to this.

2.2. A simple image collection

In the Librarian Interface, start a new collection (File → New...) called backdrop. Fill out the fields with appropriate information. For Base this collection on:, select the item Simple image collection (image-e) from the pull-down menu.

When you base a collection on an existing one, it inherits all the settings of the old one, including which metadata sets (if any) the collection uses.

Copy the images provided in sample_files → images into your newly-formed collection.

Change to the Create panel and build the collection.

Preview the result.

Click on Browse in the navigation bar to view a list of the photos ordered by filename and presented as a thumbnail accompanied by some basic data about the image. The structure of this collection is the same as Simple image collection (image-e), but the content is different.

Back in the Librarian Interface, change to the Enrich panel and view the extracted metadata for Bear.jpg.

Adding Title and Description metadata

We work with just the first three files (Bear.jpg, Cat.jpg and Cheetah.jpg) to get a flavour of what is possible. First, set each file's dc.Title field to be the same as its filename but without the filename extension:
Click on Bear.jpg so its metadata fields are available, then click on its dc.Title field on the right-hand side. Type in Bear.
Repeat the process for Cat.jpg and Cheetah.jpg.

Add a description for each image as dc.Description metadata.
What description should you enter? To remind yourself of a file's content, the Librarian Interface lets you open files by double-clicking them. It launches the appropriate application based on the filename extension, Word for .doc files, Acrobat for .pdf files and so on.
Double-click Bear.jpg: on Windows, the image will normally be displayed by Microsoft's Photo Editor (although this depends on how your computer has been set up).
Back in the Enrich pane, make sure that Bear.jpg is selected in the collection tree on the left hand side. Enter the text Bear in the Rocky Mountains as the value for the dc.Description field.
Repeat this process for Cat.jpg and Cheetah.jpg, adding a suitable description for each.

Go to the Create panel and click <Build Collection>. Once it has finished building, preview the collection. You will not notice anything new. That's because we haven't changed the design of the collection to take advantage of the new metadata.

Change Format Features to display new metadata

Now we customize the collection's appearance. Go to the Format panel and select Format Features from the left-hand list. Leave the feature selection controls at their default values, so that All Features is selected for Choose Feature, and VList is selected as the Affected Component. In the HTML Format String, edit the text as follows:
- Change _ImageName_: to Title:
- Change [Image] to [dc.Title]
- After [dc.Title]  add Description: [dc.Description] 
Metadata names are case-sensitive in Greenstone: it is important that you capitalize "Title" and "Description" (and don't capitalize "dc").

The new format statement is displayed in the list of assigned format statements. The first substitution alters the fragment of text that appears to the right of the thumbnail image, the second alters the item of metadata that follows it. The addition displays the description after the Title.

Preview the collection by clicking the <Preview Collection> button. When you click on Browse in the navigation bar the presentation has changed to "Title: Bear" and so on. Each image's description should appear beside the thumbnail, following the title.

After the first three items, the Title and Description become blank because we have only assigned Dublin Core metadata to these first three. To get a full listing, enter all the metadata.

Changes in the Format panel take place immediately and you can see the result straightaway by clicking reload (or refresh) in the web browser. If you modify anything in the Gather, Enrich or Design panels, you will need to rebuild the collection.

Changing the size of image thumbnails

Lets change the size of the thumbnail image and make it smaller. Thumbnail images are created by the ImagePlug plug-in, so we need to access its configuration settings. To do this, switch to the Design panel and select Document Plugins from the list on the left. Double-click ImagePlug to pop up a window that shows its settings. (Alternatively, select ImagePlug with a single click and then click <Configure Plugin...> further down the screen). Currently all options are off, so standard defaults are used. Select thumbnailsize, set it to 50, and click <OK>.

Build and preview the collection.

Once you have seen the result of the change, return to the Design panel, select the configuration options for ImagePlug, and switch the thumbnailsize option off so that the thumbnail reverts to its normal size when the collection is re-built.

Adding a browsing classifier based on Description metadata

Now we'll add a new browsing option based on the descriptions. In the Design panel, select Browsing Classifiers from the left-hand list. Set the menu item for Select classifier to add: to AZList; then click <Add Classifier...>.

A window pops up to control the classifier's options. Set the metadata option to dc.Description and click <OK>.

Build the collection, and preview it. Choose the new Descriptions link that appears in the navigation bar.

Only three items are shown, because only items with the relevant metadata (dc.Description in this case) appear in the list. The original browse list includes all photos in the collection because it is based on ex.Image, extracted metadata that reflects an image's filename, which is set for all images in the collection.

Creating a searchable index based on Description metadata

Now we'll add an index so that the collection can be searched by descriptions. Switch to the Design panel and select Search Indexes from the left-hand list. Click the <New Index> button. Select dc.Description from the list of metadata to include in the index, leave Indexing level: at its default, "document", and click <Add Index>.

Switch to the Create panel, build the collection, then preview it. There is now a Search button in the navigation bar. As an example, search for the term "bear" in the document:dc.Description index (which is the only index at this point).

To change the text that is displayed for the index (document:dc.Description), go to the Format panel back in the Librarian Interface. Select Search from the left-hand list. This panel allows you to change the text that is displayed on the search form. Change the Display text for the document:dc.Description index to "descriptions" (or other suitable text). Go back to the browser and reload the search page. Your new text will appear in the search form.

2.3. Enhanced collection of HTML files—Tudor

We return to the Tudor collection and add metadata that expresses a subject hierarchy. Then we build a classifier that exploits it by allowing readers to browse the documents about Monarchs, Relatives, Citizens, and Others separately.

Adding hierarchically-structured metadata and a Hierarchy classifier

Open up your tudor collection (the original version, not the webtudor version), switch to the Enrich panel and select the citizens folder (a subfolder of englishhistory.net → tudor). Set its dc.Subject and Keywords metadata to Tudor period|Citizens. The vertical bar ("|") is a hierarchy marker. Selecting a folder and adding metadata has the effect of setting this metadata value for all files contained in this folder, its subfolders, and so on. A popup alerts you to this fact. Click <OK> to close the popup.

Repeat for the monarchs and relative folders, setting their dc.Subject and Keywords metadata to Tudor period|Monarchs and Tudor period|Relatives respectively. Note that the hierarchy appears in the Existing values for dc.Subject and Keywords area.
If you don't want to see the popup each time you add folder level metadata, tick the Do not show this warning again checkbox; it won't be displayed again.

Finally, select all remaining files—the ones that are not in the citizens, monarchs, or relative folders—by selecting the first and shift-clicking the last. Set their dc.Subject and Keywords metadata to Tudor period|Others: this is done in a single operation (there is a short delay before it completes).
When multiple files are selected in the left hand collection tree, all metadata values for all files are shown on the right hand side. Items that are common to all files are displayed in black—e.g. dc.Subject and Keywords—while others that pertain to only one or some of the files are displayed in grey—e.g. any extracted metadata.
Metadata inherited from a parent folder is indicated by a folder icon to the left of the metadata name. Select one of the files in the relative folder to see this.

Switch to the Design panel and select Browsing Classifiers from the left-hand list. Set the menu item for Select classifier to add: to Hierarchy; then click <Add Classifier...>.

A window pops up to control the classifier's options. Change the metadata to dc.Subject and Keywords and then click <OK>.

For tidiness' sake, remove the classifier for Source metadata (included by default) from the list of currently assigned classifiers, because this adds little to the collection.

Now switch to the Create panel, build the collection, and preview it. Choose the new Subjects link that appears in the navigation bar, and click the bookshelves to navigate around the four-entry hierarchy that you have created.

Adding a hierarchical phrase browser (PHIND)

Next we'll add an interactive hierarchical phrase browsing classifier to this collection.

Switch to the Design panel and choose the Browsing Classifiers item from the left-hand list.

Choose Phind from the Select classifier to add: menu. Click <Add Classifier...>. A window pops asking for configuration options: leave the values at their preset defaults (this will base the phrase index on the full text) and click <OK>.

Build the collection again, preview it, and try out the new Phrases option in the navigation bar. An interesting PHIND search term for this collection is "king". Note that even though it is called a phrase browser, only single terms can be used as the starting point for browsing.

Partitioning the full-text index based on metadata values

Next we partition the full-text index into four separate pieces. To do this we first define four subcollections obtained by "filtering" the documents according to a criterion based on their dc.Subject and Keywords metadata. Then an index is assigned to each subcollection. This will enable users to restrict a search to a subset of the documents.

Switch to the Design panel, and click Partition Indexes. This feature is disabled because you are operating in Librarian mode (this is indicated in the title bar at the top of the window).

Switch to Library Systems Specialist mode by going to Preferences... (on the File menu) and clicking <Mode>. Read about the other modes too.

Return to the Partition Indexes section of the Design panel. Ensure that the Define Filters tab is selected (the default). Define a subcollection filter with name monarchs that matches against dc.Subject and Keywords, and type Monarchs as the regular expression to match with. Click <Add Filter>. This filter includes any file whose dc.Subject and Keywords metadata contains the word Monarchs.

Define another filter, relatives, which matches dc.Subject and Keywords against the word Relatives. Define a third and fourth, citizens and others, which matches it against the words Citizens and Others respectively.

Having defined the subcollection filters, we partition the index into corresponding parts. Click the Assign Partitions tab. Select the citizens subcollection and click <Add Partition>. Next select monarchs, and click <Add Partition>. Repeat for the other two subcollections, so that you end up with four partitions, one based on each subcollection filter.
The order they appear in the Assigned Subcollection Partitions list is the order they will appear in the drop down menu on the search page. You can change the order by using the <Move Up> and <Move Down> buttons.

Build and preview the collection.

The search page includes a pulldown menu that allows you to select one of these partitions for searching. For example, try searching the relatives partition for mary and then search the monarchs partition for the same thing.

To allow users to search the collection as a whole as well as each subcollection individually, return to the Partition Indexes section of the Design panel and select the Assign Partitions tab. Select all four subcollections by checking their boxes and click <Add Partition>.

To ensure that the combined index appears first in the list on the reader's web page, use the <Move Up> button to get it to the top of the list here in the Design panel. Then build and preview the collection.

Search for a common term (like the) in all five index partitions, and check that the numbers of words (not documents) add up.

The text in the drop down box on the search page is based on the filters each partition was built on. To change the text that is displayed, go to the Search section of the Format panel. The single filter partitions have sensible default text, but the combined partition does not. Set the Display text for the combined partition to "all". Preview the collection.

In the Librarian Interface, return to Librarian mode, using Preferences... (on the File menu).

Controlling the building process

Finally we look at how the building process can be controlled. Developing a new collection usually involves numerous cycles of building, previewing, adjusting some enrich and design features, and so on. While prototyping, it is best to temporarily reduce the number of documents in the collection. This can be accomplished through the maxdocs parameter to the building process.

Switch to the Create panel and view the options that are displayed in the top portion of the screen. Select maxdocs and set its numeric counter to 3. Now build.

Preview the newly rebuilt collection's Titles page. Previously this listed more than a dozen pages per letter of the alphabet, but now there are just three—the first three files encountered by the building process.

Go back to the Create panel and turn off the maxdocs option. Rebuild the collection so that all the documents are included.

2.4. Bibliographic collection—Part A

This exercise looks at using fielded searching in a collection. Fielded searching is best used for metadata rich collections. Here we use bibliographic data in MARC format. We also "explode" the database, enabling editing of the metadata with the Librarian Interface.

Start a new collection called Papers Bibliography which will contain a collection of example MARC records of the working papers published at the Computer Science Department, Waikato University. Enter the requested information and base it on -- New Collection --.

In the Gather panel, open the sample_files → marc folder, drag CMSwp-all.marc into the right-hand pane and drop it there. A popup window asks whether you want to add MARCPlug to the collection to process this file. Click <Add Plugin>, because this plugin will be needed to process the MARC records.

In the Document Plugins section of the Design panel, remove the plugins TextPlug to NULPlug by selecting each one in the Assigned Plugins list and clicking <Remove Plugin> (ZIPPlug, GAPlug and MARCPlug remain). It is not strictly necessary to remove these redundant plugins, but it is good practice to include only plugins that are needed, to avoid unwanted (and unexpected) side effects.

Now select Browsing Classifiers within the Design panel and remove the default classifier for Source metadata.

In the Search Indexes section, remove the ex.Source index. In this collection all records are from the same file, so ex.Source metadata, which is set to the filename, is not particularly interesting or useful.

Switch to the Create panel, build the collection, and preview it. Browse through the Titles and view a record or two. Try searching—for example, find items that include graphics.

Back in the Librarian Interface, go to the Browsing Classifiers section of the Design panel. Select AZCompactList from the Select classifier to add: drop down menu, and click <Add Classifier...>. In the popup window, select ex.Subject as the metadata item. Click <OK>.

AZCompactList is like AZList, except that terms that appear multiple times in the hierarchy are automatically grouped together and a new node, shown as a bookshelf icon, is formed.

Build the collection and preview the result.

Using fielded searching

Collections built with MGPP (the default indexer) provide the option of fielded searching. In the browser, go to the PREFERENCES page. You will notice that there is a Query style: option which enables you to switch between "normal" and "fielded" search. Change to fielded search now and click on the Search button. The search form has changed to a fielded form.

You can specify which search form types are available for a particular collection, and which one is the default, using the SearchTypes format statement. In the Format panel select Format Features from the left-hand list. Select the SearchTypes format statement from the list of assigned formats, and set the contents to form. This will make only fielded searching available for this collection.

Search type options include form and plain. You can specify one or both separated by a comma. If both are specified, the first one is used as the default: this is the one that the user will see when they first enter the collection.

Preview the collection again. Notice that the collection's home page no longer includes a query box. (This is because the search form is too big to fit here nicely.) To search, you have to click Search in the navigation bar. Note that the PREFERENCES page has changed so that the "normal" query style is no longer the default value.

Look at the search form in the collection. There are two fields that can be searched: text and Title. Add some more fields to search on by going back to the Librarian Interface.

In the Design panel, go to the Search Indexes section. Add a new index based on ex.Subject by clicking <New Index>, selecting ex.Subject in the list of metadata, and clicking <Add Index>.

Rebuild the collection and preview the results. Notice the extra field in the ... in field drop-down menus in the search form. You can do quite complicated queries by searching for words in different fields at the same time.

To change the text that is displayed in the drop-down menus of the search form, go to the Search section of the Format panel. Here you can change the display text for the indexes.

2.5. CDS/ISIS collection

This exercise is similar to the Bibliographic collection exercise, except that a CDS/ISIS database is used instead of a MARC database, and we do not explode the database.

Start a new collection called ISIS Collection.

Drag the files from sample_files → isis (excluding the format_tweaks folder) into the collection.

Build and preview the collection. The default indexes, classifiers and formats are not very useful for this data. There is no metadata searching, and the Titles classifier is completely empty. The filenames classifier is useless because all records come from the same file.

In the Search Indexes section of the Design panel, remove the useless Source and Title indexes, and add new indexes for Photographer^all, Country^all and Notes^all metadata.
CDS/ISIS metadata has subfields, and these are represented using ^.

In the Browsing Classifiers section, remove the existing (useless) classifiers for Title and Source, and add a new AZList for Photographer.

Rebuild and preview the collection.

In the Format Features section of the Format panel, change the VList format statement to display Photographer and Notes metadata. Change it to look like:

<td valign=top>[link][icon][/link]</td> <td valign=top>[ex.Photographer^all] [ex.Notes^all]</td>

Make fielded searching the default by changing the SearchTypes format statement to form,plain (instead of plain,form).

ISISPlug stores a nicely formatted version of the record as the document text, and this is what is displayed when we view a record. Lets tidy it up a little more.

Remove the DETACH and NO HIGHLIGHTING buttons by setting the DocumentButtons format statement to empty.

Remove the "Untitled" at the top of the document by setting the DocumentHeading format statement to empty.

Finally, lets link to the raw record, which is stored as ISISRawRecord metadata. Edit the DocumentText format statement to look like the following. (This format can be copied from sample_files → isis → format_tweaks → document_text.txt.)

[Text] {If}{_cgiargshowrecord_, CDS Record: <tt>[ISISRawRecord]</tt> <center><a href=\'_gwcgi_?e=_cgiarge_&a=d&d=_cgiargd_\'>Hide CDS Record</a></center>, <center><a href=\'_gwcgi_?e=_cgiarge_&a=d&d=_cgiargd_&showrecord=1\'>Show CDS Record</a></center> }

Preview the collection.

2.6. Editing metadata sets

GEMS (Greenstone Editor for Metadata Sets) can be used to modify existing metadata sets or create new ones. GEMS is launched from the Librarian Interface when you want to create a new metadata set, or edit an existing one. In this exercise, we run GEMS outside of the Librarian Interface.

Running GEMS

Start the Greenstone Editor for Metadata Sets (GEMS):
Start → All Programs → Greenstone Digital Library Software → Greenstone Editor for Metadata Sets

GEMS starts up with no metadata set loaded. You can start a new set, or open an existing one, from the File menu.

Creating a new metadata set

In this exercise, we will create a new metadata set. In order to save time, we will base it on an existing one: Development Library Subset. From the File menu, select New... (File → New...). A popup window appears: New Metadata Set. Fill in the fields. Use "My Metadata Set" for the Metadata set title:, "my" for the Metadata set namespace:, and select "Development Library Subset Example Metadata" from the Base this metadata set on: drop down list. Click <OK>.

The new metadata set will be displayed. The left hand side list the elements (and sub-elements, if any) for the set, and the right hand side displays the set or element attributes. Since the new set was based on the Development Library Subset metadata set, it already contains all the elements from that set.

Adding a new element to a metadata set

Right click on My Metadata Set in the left hand tree (or in the blank space in the left hand side) and choose Add Element from the menu that appears. In the popup window, type "Category" for the new element name, and click <OK>. The new element will appear in the list.

In the right hand side, the default attributes will appear for the new element. "Label" and "definition" are used in the Librarian Interface when displaying metadata elements and their descriptions. These attributes can be set in multiple languages.

Save the new metadata set by File → Save, then close the GEMS by File → Exit.