Table of Contents

Greenstone 3.10 Release Notes

This page may undergo further updates until the Greenstone 3.10 release and possibly for some time after. So continue to consult this page for the latest version of the 3.10 release notes.

Release Name: 3.10

Release Date: 28 February 2021

Released:

Release Candidate History

GS3.10rc1 issues and unknowns on Catalina Mac:

As ever, our thanks to our TSG (tech support) who made this and all Greenstone releases possible. Special thanks to our Tech Support staff member for Macs, Jacqui Elphick, who made Mac testing possible despite the difficult circumstances this year and thanks to whose efforts a Mac binary for the 3.10 series has become a reality.

Installation Instructions

Installing and running the binary release

  1. Ctrl+click (the Mac rightclick) and choose Open With or something like that to launch the installer .app inside the .dmg Mac binary file
  2. REPEAT: This second time you try to Ctrl+click and choose Open With to launch the installer .app inside the .dmg Mac binary file, Catalina will provide a buttonlink to allow you to "open anyway" and run the installer .app file in the .dmg. After a brief time the attempt to launch the installer will fail without admin rights, mentioning something about /tmp/jre failing. This is expected. Press Cancel.
  3. Open a Finder window and navigate to the /tmp folder. (One way to do this, is to launch a terminal through Go > Utilities > Terminal, then type cd /tmp and press Enter. Next type: open . and press Enter to get a Finder window opened at the /tmp folder)
  4. Now Ctrl+click on the jre inside /tmp. Press Open (or maybe the button link is labelled Run) to attempt to extract the jre. This will eventually fail as the .dmg launcher did before. That's fine too. Press Cancel.
  5. Now return to the installer .app file inside the .dmg. Ctrl+click on it and launch the installer once more. This time the launch process will complete and within a minute or so you should see the Greenstone Installer opening dialog appear.

Note that in some cases, the following doesn't work

 export TMPDIR=/something/else
 ./Greenstone-3.10-linux

Use the following instead

 TMPDIR=/something/else ./Greenstone-3.10-linux

* For Windows users: To install, download the file and double-click. On newer versions of Windows 10: If the green installer splash screen doesn't appear after a few seconds, then right click on the installer executable, choose Properties, go to the General tab and at the bottom (in the Security section) look for a tick box labelled Unblock and tick this.

When the installer is running

During the installation process you will be presented with several options. For many, the default settings will be sufficient. Some important options are

Once you have successfully installed Greenstone3, you can start up the server by choosing Grenstone3 Digital Library from the Start menu (Windows) or running gs3-server.sh/bat. This launches a small server program which starts up Tomcat and launches a browser. A small window pops up which allows you to change some settings for your library and restart the Tomcat server. Closing this program will stop Tomcat running. By default, your library will be available at localhost:8383/greenstone3/library. File→Settings in the Greenstone Server window gives you options to change the port number and which browser it uses by default. More notes about running Greenstone can be found in the README.txt file in the top level Greenstone folder.

To build collections, run GLI from the Start menu (Windows) or by running gli/gli.sh/bat in the top level Greenstone3 folder. Tutorial exercises about building collections in Greenstone 3 can be found here. Make sure you select the Greenstone3 tab at the top if it is not already selected.

Installing in text-only mode

* Refer to Running the installer in text-only mode

Adding source code to a binary release

Installing a source release

Further instructions

New (client-)GLI features

a. Metadata to CSV options:

- File > Export to metadata CSV: for a collection you have open, this option creates a metadata.csv file in a location of your choosing containing all the metadata you can see in GLI, including inherited metadata. If the metadata.csv file you selected already exists, then the metadata you see in GLI is amalgamated with the selected CSV file. This option allows you to backup your collection’s metadata to a spreadsheet file. There is NO RECONVERT feature, to convert back to metadata.xml files from metadata csv format. But you can build your collection with metadata from the CSV spreadsheet. See the following option below which explains how to redo your collection to work with metadata from a spreadsheet instead of using metadata in GLI/metadata.xml files.

- File > Convert to metadata CSV: for the collection you have open, this option creates a metadata.csv file in your collection’s “import” folder by default (which is the best location), by destructively removing all the metadata from the collection’s metadata.xml files (in other words, by removing the metadata you see in GLI) and shifting them out into the selected metadata.csv file. If you selected an existing metadata.csv file, then any metadata you currently see in GLI is amalgamated with the selected CSV file, before it gets removed from GLI/the metadata.xml files. Selecting this option prepares your collection so that you can switch over to using a MetadataCSVPlugin, configured with metadata_value_separator field set to “\|”, to then rebuild your collection producing the same results as before.

b. Collection security skeleton elements as discussed at http://wiki.greenstone.org/doku.php?id=en:user_advanced:security, can now be added through (client-)GLI’s Edit > Edit collectionConfig.xml menu option. At the bottom of the Config File Editor dialog that appears, you will find a small toolbar that allows you to choose which (skeleton) XML <security> element to add:

- to hide the current collection,

- to add the appropriate <security> element to make the entire collection private except for one or more groups you specify,

- to add the appropriate <security> element to make all the docs in the collection private except for the groups you specify (adds a <security> element),

- to add the appropriate <security> element to make select docs in the collection private except for the groups you specify (where you can then specify which docs as explained on the wiki link already provided),

- to add a further <documentSet> element to the existing <security> element

- to add a further <documentSet> element into the existing <security> element

One issue with this remains: if you want to undo the addition of a security element, you have to press the Undo button twice at present. I haven’t yet figured out why this is. (If you press Undo once, the entire XML content of your collectionConfig.xml becomes empty, so you’ll naturally press Undo again in alarm and then it will look right again.)

Setting up your Greenstone to run over https

Note: This section is untested for this release, as the prevalent method of supporting https for tomcat based servers appears to be to have a main apache httpd web server that works with https, then set up a reverse proxy to let apache httpd redirect requests for GS3's tomcat server internally to Greenstone.

The more secure https protocol is increasingly required by browsers and gradually superseding http. Given that you meet the following requirements and configure your GS3 as below, Greenstone 3 has now been automated to obtain an https certificate for you from the free Certification Authority "Let's Encrypt".

Requirements: because we need to temporarily run a server on port 80 to get a certificate issued and because port 80 has some access restrictions surrounding it on most machines,

Steps:

  1. Edit build.properties as follows:
    • set tomcat.server to the primary hostname/domain name that you want your Greenstone3 to run as and which is to be registered in your certificate. This would be the host name of your machine.
    • set a value for keystore.pass.
      This will be the password on your final certificate used by tomcat.
    • Ensure server.protocols contains https.
      The server.protocols property is a comma-separated list that indicates which protocols are to be supported by your Greenstone 3 server. This property can be set to one of
      • http
      • https
      • http,https
      • https,http
        The first in the list becomes the default protocol used for previewing with the GS3 server application, gs3-server.
    • By default tomcat.port.https is set to 8443. Ensure this port is not already in use, otherwise change it to a port value that's not in use.
  2. Make sure you have read and agree with the Let's Encrypt Subscriber Agreement
  3. Use a terminal to go into your GS3 installation folder, run gs3-setup.bat on windows and source ./gs3-setup.sh on linux and mac to set up the GS3 environment, then run the ant setup-https-cert target. For example on Linux,
    cd /path/to/GS3
    source ./gs3-setup.sh
    ant setup-https-cert


    You'll be asked for an email that Let's Encrypt can optionally communicate to you on, as well as any additional domain names you want in the same certificate (additional domains are untested), and whether you agree with the Let's Encrypt Subscriber Agreement.
    On linux or mac, you may be asked to enter your sudo password to briefly run a server on port 80. Note: On Mac and windows, GS3 uses the third party ZeroSSL to get Let's Encrypt to issue certificates, which will result in GS3's own tomcat server to be run on port 80 during certificate issuance. On Linux, we use Let's Encrypt's own certbot-auto script for the certification process, and have it set to run a standalone temporary server on port 80.

  4. Once the setup-https-cert ant target has finished, you can start your web GS3 server by either running the gs3-server application or by running "ant start" from the terminal.
  5. If you ran the gs3-server application, press the Enter Library button to open your DL home page. If you ran ant start from the command line, then open a browser manually. Point your browser to https://<tomcat.server>:<tomcat.port.https>/greenstone3/library, adjusting the tomcat.server and tomcat.port.https values as per what you set for these properties in your GS3 installation folder's toplevel build.properties file.
    BE AWARE:
    • when visiting your digital library over https, remember to prefix https to the address and use the https port defined with tomcat.port.https, and do not use localhost but the actual domain you registered which is defined in the tomcat.server property you set.
    • If you've also turned on http support alongside https, then to visit your DL over regular http, return the URL to having the http prefix and remember to change the port number to what's defined for http in property tomcat.port.http. For http URLs, you can use the domain name set in tomcat.server too, or try either of localhost or 127.0.0.1 denoting the local host machine.
  6. Once your https home page has loaded, confirm that your certificate is properly installed by looking for a green padlock next to the address bar. (Depending on your browser, you can click the padlock to get more information on the certificate issuer.)

There are 2 more https-related automated ant targets you can run from the command line:

Important: Beware that if you've configured your GS3 to support http and https, by setting the server.protocols property to include both http and https, then switching between the two protocols when you visit your GS3 pages in your browser could result in the http variants of GS3 web pages not remembering you when you log in to them. For the solution and for further details, consult the section Troubleshooting > Your browser doesn't remember you being logged into greenstone.

PDF plugin restructuring and the NEW PDFv2Plugin

From GS3.09 onward, GS3 binaries have been including additional tools for converting from PDF to various text/html/image/image+text formats. (For GS2, only the nightly binaries at http://www.greenstone.org/caveat-emptor/?latest=latest will contain these changes.)

The old "PDFPlugin" remains deprecated but continues to be included in GS3.10. In its place, there are 2 plugins to handle PDFs:

For the eventual 3.09 release, the old PDFPlugin that you're familiar with, the one which has the pdfbox_conversion flag but also makes use of the old pdftohtml tool behind the scenes, will hang around with a deprecated warning, to allow people to port over their collections and keep rebuilding with the old settings or to rebuild their collection with one of the 2 new PDF plugins. However, new collections will have the PDFv2Plugin in the Document Plugins pipeline by default for GS3, and PDFv1Plugin by default for GS2, since GS2 doesn't come with the PDFbox extension out of the box. So GS2 users will have to manually add in PDFv2Plugin in place of PDFv1Plugin for new collections, after setting up the pdfbox extension. But then it should work as usual.

The "convert_to" options/output formats of the new PDFv2Plugin are:

As always, text is only extracted from a PDF where extractable. This depends on user permissions for a PDF, whether the PDF contains actual extractable text and not just images of text, whether the PDF is undamaged, and any other such factors.

There may be further adjustments made, including to display strings, but so far, we've decided on the above output formats and they seem to work on my regular PDF test documents.

OpenOffice extension

If you have LibreOffice or OpenOffice installed, you can use the OpenOffice extension to process some newer versions of Microsoft Word, Excel and Powerpoint documents. Refer to the openoffice greenstone extension instructions.

Changing the admin password

Login to the administration page, 'edit' the admin account, and click 'change password'. Alternatively, you can login as admin via the login button at the top right of each page. Once you are logged in, this button will change to say 'admin'. Click this button and select 'account settings'. From there, you can select 'change password'.

Setting up your Greenstone OAI Server and using GLI to download over OAI from a Greenstone server

In Greenstone 3, collections should be available over OAI by default. Their collectionConfig.xml files already specify that each collection is OAI enabled, through use of an OAIPMH serviceRack element. If you want to disable a collection from being accessibile over OAI, comment out the OAIPMH serviceRack element in that collection's collectionConfig.xml. You would do so by embedding the entire element in comment markers:

  <!--
    <serviceRack name="OAIPMH">
        ...
    </serviceRack>
  -->

If you wish to validate the Greenstone 3 OAIServer, edit resources/oai/OAIConfig.xml to add in the adminEmail property to contain the email to where test results should be sent. Also set the repositoryId field to a ID name you want (e.g. to greenstone), beware that there are some naming conventions that govern valid values for repositoryID. If testing the behaviour of the resumptionToken, set the resumeAfter element to a low value like 5. Then restart the Greenstone server.

To validate your OAI server, visit http://www.openarchives.org/Register/ValidateSite. Your server must be available over the internet to do this. The machine on which you're running the Greenstone 3 server will have to have its firewall and virtual server (port-forwarding) settings set up such that the Greenstone server can be made accessible to the outside world.

Setting up your Greenstone 3 OAI Server is covered in further detail in the tutorial http://wiki.greenstone.org/gsdoc/tutorial/gs3-current/en/GS_OAI_server.htm

For further information on your Greenstone OAI Server, please read through OAI support.

Setting up a remote Greenstone 3 server

This will allow remote client-GLI applications to connect to your Greenstone server, to remotely create and upload new collections to be built and hosted by your server machine.

Remote Greenstone 3 Server

To install the server-side functionality:

1. If you're on Windows, you will need to teach Greenstone where the perl executable is.

You can do this either manually, by editing a couple of Greenstone config files as explained just below, or you could run the Greenstone server once and press Enter Library button to visit your library home page. Doing so will automatically set up the perl path in various Greenstone files.

To do this manually on Windows,

a. Open Tomcat's conf/web.xml file for editing (found in greenstone3/packages/tomcat/conf folder, if installed in the default location), as you may need to specify the full path of the Perl library for the parameter "executable" of CGIServlet. This takes the form:

<init-param>
	<param-name>executable</param-name>
        <param-value>C:\Program Files\greenstone3\gs2build\bin\windows\perl\bin\perl.exe</param-value>
</init-param>

b. Edit the first line of the greenstone3/web/WEB-INF/cgi/gliserver.pl file and specify the full path of the perl binary. On Windows this will be (if installed in the default location):

#!C:\Program Files\greenstone3\gs2build\bin\windows\perl\bin\perl -w

2. Make the Greenstone "collect" directory, located in web/sites/localsite, writeable by the webserver user.

On Unix, you would do

chmod -R a+w /path/to/your/GS3/web/sites/localsite/collect

On Windows, run in a DOS prompt:

cacls "C:\Program Files\Greenstone3\web\sites\localsite\collect" /P Everyone:F

3. Open up the file build.properties located in your greenstone installation folder. Edit the tomcat.server property's value to refer to your server machine's hostname instead of leaving it at the default value of "localhost":

# tomcat info
tomcat.server=your-server-computer-name

Once the server is started up, this will update the same property in greenstone's web/WEB-INF/classes/global.properties file. Then images viewed from a browser on the client side will refer to the correct location on the remote machine.

(If you don't know what your machine's host name is and you're on Windows, then open a DOS prompt and type:

ipconfig /all

Scroll to the top of the output that gets printed to the screen and note what it mentions for HOST NAME. Also note the DNS Suffix Search List.

Put these two together with a period mark to separate them and use this as the value for your tomcat.server property.)

4. Set up your Greenstone environment if you've not already done so by running gs3-setup.bat in your Greenstone 3 installation folder (source ./gs3-setup.bash on Linux and Mac machines). And then start up your Greenstone 3 server with:

ant start

(or ant restart)

5. Check that Tomcat and Greenstone3 are working correctly by visiting

http://<your-machine-name>:<port>/greenstone3/library

6. Add some user accounts by visiting the Greenstone 3 home page (http://YOURHOST:YOURPORT/greenstone3/library), clicking the admin link at the top right and logging in. The username is admin. By default, the password is admin too, unless you already set this during the installation process or if you changed this afterwards.

Once logged in, go to the Administration page. You can access this via the link on the home page. (Or you can click the admin link, choose Account Settings, and then click the Administration Page link on the top left. )

Add a new user by providing a new username, setting a password for the user that's a minimum of 3 characters long, and by using the drop-down provided for the Groups field to one or more of the available options, such as personal-collections-editor.

Even if only the admin user wishes to use the client-gli, they will still need to log in to the Administration Page once after installation in order for the user database to be set up.

7. Finally, visit the following page in the web browser to test that your remote Greenstone server is set up properly:

http://<your-machine-name>:<port>/greenstone3/cgi-bin/gliserver.pl?cmd=check-installation

You should get a message saying "Java found" and "Installation OK!" Important: You cannot continue until this is successful, as the Remote Greenstone 3 server will not work without it!

If you get a message saying "Java failed"

If you are using perl 5.22 or later, these no longer come with the perl CGI module. Please download https://trac.greenstone.org/export/36059/main/trunk/greenstone2/common-src/cgi-bin/CGIModule.tar.gz into web/WEB-INF/cgi, untar it, and copy the contents of the resulting CGIModule folder up one level (i.e. outside of that folder).

Client-GLI

Assuming that the remote Greenstone server is accessible to the outside world and you're not behind a firewall, you can access the remote Greenstone server from a client-gli application installed on any other machine. To do so,

1. Run client-gli quite as you would GLI. It's accessible from the Windows Start menu, otherwise you can run the client-gli script (located at the toplevel of a Greenstone installation) from a terminal.

2. You'll be asked for the gliserver.pl URL of the remote Greenstone 3 server that you wish to connect to. This is of the form

http://<remote-gs3-server-machine-name>:<remote-port>/greenstone3/cgi-bin/gliserver.pl

It's the same URL as in Step 10 of setting up the remote GS3 server above.

GLI Java Web Start application (replacement for GLI Applet) - Additional Steps

This feature untested in 3.10 and rc1, but tested for the 3.09 release. It is possible that Java support for Web Start applications is or will be deprecated too, as was the case with Java Applets.

You will need a Java Development Kit (JDK) v 7 or 8 installed for this.

Many browsers have stopped supporting Java applets including Microsoft Edge, though Microsoft's Internet Explorer still supported it. For this reason, 3.09's GLI is now no longer provided as an applet, but has been converted into a Java Web Start application.

Instructions for using the GLI Java Web Start, which works over the JNLP protocol, are below.

  1. First follow the instructions above for setting up the Remote GS Server
  2. Next, generate the SignedGatherer.jar as follows. Be aware that to run the jarsigner utilities, you will need a JDK installed.
    • Use a terminal to go into the Greenstone "gli" directory, then run
          keytool -genkey -alias privateKey -keystore appletstore -storepass greenstone

      Enter the appropriate details for your organization. When it asks to enter the key password for <privateKey>, choose your own password or hit Enter to use "greenstone".

    • Next, run
          /PATH/TO/YOUR/JDK_INSTALLED_FOLDER/bin/jarsigner -keystore appletstore -signedjar SignedGatherer.jar GLI.jar privateKey

When it prompts, enter the password you used above (greenstone).

  1. Move the created SignedGatherer.jar file from the gli directory into GS3's web/applet subdirectory.
  2. You need to associate JNLP files with Java Web Start (jre/bin/javaws).
    • On Windows, create the association in the usual way: when you first access the GLI Web Start application through Greenstone, a JNLP file called "GLIappWebStart.jnlp" will be offered for launching or download. If JavaWS is not already the default application to open JNLP files with, rightclick on the downloaded GLIappWebStart.jnlp file and choose Launch/Open with …. Browse to your Greenstone3's packages/jre/bin/javaws.exe or any installed Java's jre/bin/javaws.exe to use Java's Web Start application as the launcher.
    • On Mac, .jnlp files may already be associated with the Java Web Start application for launching JNLP files (Java web start applications). Otherwise, see page 3 of http://online.unm.edu/help/learn/faculty/tools/web-conferencing/common/web-conferencing-pdf-files/jnlp-file-association.pdf for instructions on creating the file association for this. Once the file association between the Java Web Start application and .jnlp files has been made, you can just double-click on the GLIappWebStart.jnlp file you downloaded.
    • For Linux, create a file with .desktop extension (e.g. "javawebstart.desktop") containing the following, edit the path to javaws, and save this file into ~/.local/share/applications:
      # This file makes Ubuntu associate .jnlp files with Java Web Start (javaws)
      # This file should be adjusted and then copied into ~/.local/share/applications
      # as a file with .desktop extension, e.g. javawebstart.desktop
      # https://askubuntu.com/questions/235861/how-to-associate-jnlp-file-with-javaws
      [Desktop Entry]
      Encoding=UTF-8
      Name=Java 7 Web Start
      Comment=Java 7 Web Start
      Exec=/EDIT-THIS-PATH-TO-YOUR-JAVA-JRE/bin/javaws %u
      Terminal=false
      Type=Application
      Icon=javaws
      Categories=Application;Network;
      MimeType=application/x-java-jnlp-file;
  3. Launch the Java Control Panel by running jre/bin/javacpl.exe on Windows, or jre/bin/jcontrol (formerly jre/bin/ControlPanel) on Linux or follow the instructions at https://www.java.com/en/download/help/mac_controlpanel.xml for mac. (GS3 binaries now include a JRE in the packages folder if you want to use the bundled JRE.) In the Java Control Panel, go to the Security tab, set Security level to High if not already set. Click Edit Site List, and then press Add to add the http://host:port (or https://host:HTTPS-port if you've set up https support) that the GS3 will run on. Remember, to be accessible to the outside world, the host can't be "localhost", but would be the hostname of your machine or public IP. Click OK to exit the Java Control Panel with your changes in effect.
  4. Make the GLI link on the home page active: Open web/interfaces/default/transform/pages/home.xsl for editing, find the line
    <!--<gslib:libraryInterfaceLink/><br/><br/>-->

    and remove the comments. i.e. change it to

    <gslib:libraryInterfaceLink/><br/><br/>

    Save the changes and close the file.

  5. (Re)Start the GS3 web server and visit your DL library home page, http://[hostname]:8383/greenstone3/library.
  6. Since you have set up the JNLP file association in a previous step, you can now click on the "The Librarian Interface" link and your browser should offer to save or launch a file called GLIappWebStart.jnlp. Click Open With and in the dropdown box beside it, one of the launcher applications, at least on Windows, should be the "Java(TM) Web Start Launcher" application (javaws.exe) that you associated with .jnlp file extensions. Choose that application as the launcher. If the browser is able to successfully launch GLIappWebStart.jnlp, then Java Web Start will be used to run the GLI application indicated by the JNLP file. If launching through the browser is not possible, for example, if the launcher application is not listed, as may happen on Linux machines, then choose to save the JNLP file. It will download the file to a temporary user area (like C:\Users\<user>\AppData\Local\Temp on windows). And then you can rightclick on the downloaded GLIappWebStart.jnlp file, to launch it with the Java Web Start program you already associated with this file type. This way should work on both Linux and Windows.
  7. You will need to authorise the GLI to run. After that, the JNLP version of the GLI Applet will eventually run and behave like the usual client-GLI (and like the old GLI applet) from this point onward: starting with providing your account username and password on Greenstone that you created when setting up the remote GS3 server. You may also be asked for your proxy authentication details if set up for behind a proxy.

Converting a GS2 collection to GS3 when working with a remote GS3 server

The new Format Conversion Wizard to convert GS2 format statements to GS3 format statements (see this page) only appears when you're working with GLI, not client-GLI. The client-GLI for GS3 will only perform the most basic initial step in the conversion process, which is to preserve the GS2 format statements in inactive XML tags in the new collection's collectionConfig.xml.

However, if you have a local Greenstone 3 installed, you can still manage to convert a remote collection's collect.cfg file to its GS3 equivalent. See here for details.

Important Changes and Bug Fixes

IMPORTANT information

For ease of access this section has been brought across from the 3.08 Release Notes and 3.09 Release Notes, but not all of it may be relevant to the 3.10 release.

Patches to 3.10

Troubleshooting

Some perl scripts don't run properly or to completion

If you're running Greenstone 3 on a non-English locale and certain perl scripts don't run properly or completely, such as when you run perl -S pluginfol.pl -describeall, then the problem may be locale/encoding related. To resolve this issue, refer to Supporting running Greenstone in other locales on Windows 10.

Content Encoding Error when visiting the local solr servlet page

If you see a Content Encoding Error when opening your GS3's solr servlet page at http://127.0.0.1:8383/solr or http://localhost:8383/solr in your browser, then this may have to do with the version of Java you have installed on your machine. From GS3.09 onward, if your machine has its own Java installed, then assuming that its version is sufficient and its bit-architecture (32 or 64 bit) matches, Greenstone will use your Java in preference to the bundled Java Runtime (JRE) that Greenstone ships with. We found that a recent version of Java (version 1.8.0_161 was problematic for us), caused the Content Encoding Error when visiting the solr servlet, whereas the bundled JRE and slightly earlier and much newer versions of Java such as 1.8.0_144 and 1.8.0_191 did not have these issues.

Solution: if you have a problematic version of Java installed, - either unset JAVA_HOME and remove this Java's bin folder from the PATH environment variable too, thus helping Greenstone 3 use its bundled JRE instead - install a newer version of Java on your system. We found that the current latest one, 1.8.0_191 worked successfully for this purpose.

SIGPIPE errors when building a collection

We've added a work around to one kind of SIGPIPE errors which could occur with large collections when using solr as indexer. However, a couple of people on the mailing list encountered SIGPIPE errors on occasions when solr was not the indexer. If your collection is using jdbm as the database type and the error messages surrounding the SIGPIPE mention issues with "transaction commit", then Mariana Pichinini on the mailing list found that the following helped:

Your browser doesn't remember you being logged into greenstone

The issue: The following scenario can occur if you set up GS3 with https, and your server.protocols property in build.properties contains both http and https (i.e. you have server.protocols=http,https or server.protocols=https,http).

Switching between visiting your Greenstone 3 digital library (DL) using http and https URLs can result in the http version of the pages not remembering your login details despite you logging in. This can happen if you ever started off with the https version of the URL to a Greenstone3 DL page and then moved to using the http version of your GS3 URL, or if you ever logged in to your GS3 over https and then attempt to log in later using http.

The solution: The solution is to either start a private window if you want to access your GS3 DL pages over http, or to first clear your browser cookies related to your GS3 DL before swapping from https to http.

The cause: Using https causes session cookies to have the secure flag set to true. When a session cookie has the secure flag thus set, non http URLs cannot return that cookie in their subsequent requests to the server. Only https URLs can. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies section "Secure and HttpOnly cookies" which states "A secure cookie is only sent to the server with an encrypted request over the HTTPS protocol" and https://stackoverflow.com/questions/2321224/cookie-across-http-and-https-in-php
It further seems that in http mode, the browser does not want to overwrite secure cookies created in https mode with new cookies sent by the server when using http mode. Thus after using https and acquiring secure session cookies, the server can no longer track a user's session when they switch to http until the cookies are cleared either explicitly or through opening a private window.

Mac Installer fails

If running the Mac installer results in a message about having failed to copy to /var/folders/zz/ permission denied), then restarting the Mac followed by running the installer once again worked for us, resulting in a successful installation.

Using the command line to reset the admin password when it isn't recognised

If at any stage, your admin password is no longer recognised from the Greenstone Reader interface (the Greenstone pages that you view through the web browser), then you can try to reset your admin password through the command line.

1. On Linux and Mac, open a terminal and use it to navigate to your Greenstone 3 installation folder.

On Windows, you can either open a DOS terminal the usual way and then likewise use it to navigate to your Greenstone 3 installation folder (using the cd command). Or, you could open a Windows File Explorer first and use it to navigate to your Greenstone 3 installation. Then you could easily get a DOS prompt at that File Explorer location, as explained at stackoverflow:

Hold Shift while Right-Clicking a blank space in the desired folder to bring up a more verbose context menu. One of the options is Open Command Window Here. This works in Windows Vista, 7, 8, and 10.

2. Now that your terminal is at the GS3 installation folder, you can type the following command in the terminal:

ant config-admin

4. It will ask for a new admin password. Type the new password and hit enter.

5. Once you restart the server, try out your new password.

Useful information

When you've built a collection of documents, you may discover that there appears to be a copy of all these documents in the collection's import, archives and index subfolders and wonder whether Greenstone could really be so inefficient with disk space as to keep 3 copies of everything. As it happens though, Greenstone uses hard-links both on Linux and Windows, in order to keep just one set of your documents. Then it simply hardlinks to these, instead of making copies. By default, Windows doesn't show you when files on your filesystem are hard-linked. If you choose to install the Windows extension application Link Shell Extension (LSE), it will put red arrows on files that are hard linked.

Known Issues

IsisGdl does not work yet on Mac Catalina (10.15) and onwards

With GS3.10rc1 we have not yet been able to produce an IsisGdl binary that works on Mac Catalina (10.15). Mac appears to have taken away backwards compatibility with 32 bit binaries, such as the IsisGdl 32-bit binary that still worked on Mojave. The 64 bit binary generated on Mojave does not work on either Mojave or Catalina. We are continuing to investigate this issue.

Changes to Tomcat port affects Solr collections

Currently when you change your tomcat port (either in build.properties, or using File→Settings in the Server program) the changes won't propagate to Solr which will still try to use the old port number. If you make changes to Tomcat port, please shutdown and restart the server from the Start Menu. If you are starting from within a terminal, you will need to shut down Greenstone and restart it from a fresh terminal.

Web document editor requires perl CGI module

Note: For GS2.87 and 3.10rc1 onwards, we hope to have included CGI.pm in a tarball at least (untested) for those whose perl installations do not have it and were it can't be easily installed.

Editing documents through the web interface requires your system perl to have CGI installed. Without CGI, you can edit metadata and save, but your changes won't have been applied. You can tell this is the case by looking at the build log files in the collection, e.g. greenstone3/web/sites/localsite/collect/<collname>/log/build_log.1480553047299.txt. The message might say something like

Document Editor Build 
Command = /bin/perl -S /greenstone/pei-jones/web/WEB-INF/cgi/metadata-server.pl 
Content-type:text/plain

ERROR: Can't locate CGI.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ./gsdlCGI.pm line 9.
BEGIN failed--compilation aborted at ./gsdlCGI.pm line 9.
Compilation failed in require at (eval 1) line 1.

CGI.pm is not included with versions of perl from 5.22 and onwards, which is when this issue can result.

If you get this error, then please install the CGI module for your system perl.

Work Arounds

Using filenames containing &, < and > in GS3.10 collections

The Enrich panel in GS3.10's GLI is better able to handle

  1. &, < and >
  2. and other non-basic ASCII characters.

When you create a collection and assign it metadata in GLI that contains such characters, GLI will now successfully preserve them between GLI sessions.

The second set of characters listed above are still not recommended for use in metadata when dealing with a remote GS3 server, as that has to do with successfully transferring characters from an operating system in one file system encoding on the client side to possibly another operating system using another file system encoding on the GS3 server side.

Beware: It is only GLI's Enrich pane that has been fixed up to support both sets of characters in metadata. No other part of Greenstone has been fixed to cope with these characters, should they likewise struggle with them.

The fix is often in the form of adding hex encoded entities into the metadata.xml file and decoding them back for display in GLI's Enrich pane.

Greenstone applets (Phind, Collage) crash Firefox

See bugzilla report.

If attempting to view a java applet (like Collage or Phind phrase classifiers) crashes Firefox, then make sure you have the Java Applet plugin installed. If it is installed and Firefox is still crashing, then open firefox and visit the page

about:config

Scroll down to the property:

dom.ipc.plugins.java.enabled

Set it to true (rightclick and choose toggle).

PDF to image conversion error on Linux

If you've configured a PDFPlugin to convert PDFs to images, increase the verbosity in Import Options and Build Options to 5 in GLI's Create panel.

When rebuilding the collection, check to see if you encounter the following error message mentioning that 'memory allocation failed', a 'corrupt image' at 'ReadPNGImage' and 'PostScript delegate failed':

import.pl> Converting pdf05-notext.pdf to pagedimg_jpg format
import.pl> calling cmd "/usr/bin/perl" -S gsConvert.pl -verbose 5 -pdf_zoom 2 -errlog "/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/err.log" -output pagedimg_jpg "/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf"
import.pl> Error executing pdfpstoimg.pl
import.pl> pdfpstoimg error log:
import.pl> convert: memory allocation failed `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadOnePNGImage/2160.
import.pl> convert: corrupt image `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadPNGImage/3794.
import.pl> convert: Postscript delegate failed `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/681.
import.pl> convert: no images defined `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext/pdf05-notext.jpg' @ error/convert.c/ConvertImageCommand/3068.
import.pl> Convert error for /research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf 
import.pl> Could not convert pdf05-notext.pdf to pagedimg_jpg format
import.pl> convert: memory allocation failed `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadOnePNGImage/2160.
import.pl> convert: corrupt image `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadPNGImage/3794.
import.pl> convert: Postscript delegate failed `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/681.
import.pl> convert: no images defined `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext/pdf05-notext.jpg' @ error/convert.c/ConvertImageCommand/3068.
import.pl> Convert error for /research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf 
import.pl> Converting pdf05-notext.pdf to html format

If you see the above error message, then:

1. Use a text editor to open your Greenstone 3's gs2build/ext/imagemagick/linux/etc/ImageMagick/delegates.xml

2. Find the line that would say:

<delegate decode="ps:alpha" stealth="True" command="&quot;gs&quot; -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 &quot;-sDEVICE=pngalpha&quot; -dTextAlphaBits=%u -dGraphicsAlphaBits=%u &quot;-r%s&quot; %s &quot;-sOutputFile=%s&quot; &quot;-f%s&quot; &quot;-f%s&quot;"/>

The above specifies the PostScript delegate for PNG images. It has the sDEVICE set to pngalpha.

3. Change the line to:

<delegate decode="ps:alpha" stealth="True" command="&quot;gs&quot; -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 &quot;-sDEVICE=**pnmraw**&quot; -dTextAlphaBits=%u -dGraphicsAlphaBits=%u &quot;-r%s&quot; %s &quot;-sOutputFile=%s&quot; &quot;-f%s&quot; &quot;-f%s&quot;"/>

The above changes the sDevice to pnmraw.

4. Save the file and re-run the build now.

Updated Translations

Thanks to the following people for new and updated translations since 3.09: