Table of Contents
Greenstone 3.11 Release notes
This release's key feature is WebSwing GLI, which is GLI running direct from your web browser. See the section Important changes.
This page may undergo further updates until the Greenstone 3.11 release and possibly for some time after. So continue to consult this page for the latest version of the 3.11 release notes.
Release Name: 3.11
Release Date: 14 August 2023
- Binaries for Windows, GNU/Linux 32 and 64 bit machines, a Mac binary for Mac Monterey (OS version 12.6.7)
All the binaries have only been spot-tested, since the 3.11 release candidate 1 binaries, which came out some months before, were tested more extensively on all the tutorials. Source distributions and source components have been tested (in week of 28 July 2023) to compile on Mac Monterey/12.6.7, Windows 10, Ubuntu 32 (12.04) and 64 bit (20.04.6 LTS).
svn tag page trac tag page. Code revision up to 37940. Tag revision: 37949.
- The Mac binary is generated on Monterey/12.6.7 (a VM). For rc1, most of the testing work was done on the same Monterey/12.6.7. The final release's spot-testing was also done on a Monterey/12.6.7, but slightly distinct: the physical machine that runs the release virtual machine/VM. (There was a fix made after the release candidate 1 was released, when we noticed the installed Mac Monterey's GS3-server and GLI had issues launching on a Monterey other than the VM it had been generated on.)
- The Windows release candidate 1 binary was tested on Windows 10, and the final release also spot tested on the same.
- The Linux release-candidate 1 binary was largely tested on Ubuntu 20.04.6 LTS and spot tested on Ubuntu 12.04. Spot testing of the final release was on both of these.
Release Candidate History
- Greenstone 3.11 rc1: Release Candidate 1. Released 10 July 2023.
Binaries for Windows, GNU/Linux 64 bit, Mac Monterey. Mac version generated on MacOS Monterey/12.6 and tested on the same. See notes directly below.
Download from the snapshots page
svn tag page trac tag page. Code revision up to 37807. Tag revision: 37810.
GS3.11rc1 unknowns on Mac Monterey:
- Due to the Monterey test machine being the same machine as where the Mac release was generated, testing may have been biased: things may have worked that may not have worked on a separate test machine where libraries set up for generating the release don't come pre-installed. We're hoping to spot test GS3.11 on a distinct Mac Monterey.
As ever, our thanks to our TSG (tech support) who made this and all Greenstone releases possible.
Installing and running the binary release
- Download the appropriate binary release for your operating system and run it.
Mac OS Notes
- The security settings in newer Mac OS versions have been altered to by default disallow users from casually opening and running .dmg executables that are not from Apple itself. When attempting to open the Greenstone binary .dmg file, if it pops up an error warning about security, you will need to set up the Security on your Mac to allow you to run .dmg files downloaded from the internet. Otherwise the Greenstone mac binary will not run. To do this, go to your Mac's System Preferences. Under "Personal", select Security & Privacy. In the General tab, tick "Allow apps downloaded from Anywhere", then confirm that you want to "Allow From Anywhere". You'll need to be admin to do this, otherwise click on the padlock at the bottom left of the Security & Privacy tab and log in as admin.
If you are on a Mac Catalina and do not have admin rights, you can still get around this (for now) and install GS3.11 on Mac Catalina as follows:
- Ctrl+click (the Mac rightclick) and choose
Open Withor something like that to launch the installer
.dmgMac binary file
- REPEAT: This second time you try to Ctrl+click and choose
Open Withto launch the installer
.dmgMac binary file, Catalina will provide a buttonlink to allow you to "open anyway" and run the installer .app file in the
.dmg. After a brief time the attempt to launch the installer will fail without admin rights, mentioning something about
/tmp/jrefailing. This is expected. Press Cancel.
- Open a Finder window and navigate to the
/tmpfolder. (One way to do this, is to launch a terminal through Go > Utilities > Terminal, then type
cd /tmpand press Enter. Next type:
open .and press Enter to get a Finder window opened at the
- Now Ctrl+click on the
/tmp. Press Open (or maybe the button link is labelled Run) to attempt to extract the jre. This will eventually fail as the
.dmglauncher did before. That's fine too. Press Cancel.
- Now return to the installer .app file inside the .dmg. Ctrl+click on it and launch the installer once more. This time the launch process will complete and within a minute or so you should see the Greenstone Installer opening dialog appear.
- You will need to set the file to be executable before running it. e.g.
In a terminal
chmod a+x Greenstone-3.11-linux
or on Ubuntu, you can right-click on the installer, choose Properties, Go to the Permissions tab and ensure the "Execute: Allow executing file as program" tickbox is ticked.
- The installer initially unpacks into a temporary directory (/tmp on linux). If you wish to change this, set the TMPDIR environment variable.
Note that in some cases, the following doesn't work
export TMPDIR=/something/else ./Greenstone-3.11-linux
Use the following instead
- To install, download the file and double-click. On newer versions of Windows 10: If the green installer splash screen doesn't appear after a few seconds, then right click on the installer executable, choose Properties, go to the General tab and at the bottom (in the Security section) look for a tick box labelled Unblock and tick this.
During the installation process you will be presented with several options. For many, the default settings will be sufficient. Some important options are
- Folder where you want greenstone3 to be installed.
- Choosing which packages to install.
- Greenstone3 will install the Apache Tomcat webserver by default. You can choose not to install it, but then you will need to set up your own version of Tomcat to serve Greenstone. We recommend using Greenstone's Tomcat, at least initially while you get everything set up.
- ImageMagick is bundled with Greenstone for binary web releases for all platforms, and includes JPEG2000 support. You can choose not to install it if you already have ImageMagick previously installed.
- Ghostscript is now bundled with Greenstone for binary web releases for Windows and Mac. You can choose not to install it if you already have Ghostscript previously installed.
- Choosing a password for the administration pages. These pages allow the admin user to inspect and manage the list of registered Greenstone users. You can add new users, and change group settings for existing users. Greenstone user registration is needed if you want to use remote GLI login to the Greenstone server, or if you want to make collections/documents only accessible by certain groups of users. (If the password is not set during installation, the default password for the 'admin' user is 'admin'. You can change the password any time after installation, by running the Greenstone 3 server and visiting the Administration pages. See below).
Once you have successfully installed Greenstone3, you can start up the server by choosing Grenstone3 Digital Library from the Start menu (Windows) or running gs3-server.sh/bat. This launches a small server program which starts up Tomcat and launches a browser. A small window pops up which allows you to change some settings for your library and restart the Tomcat server. Closing this program will stop Tomcat running. By default, your library will be available at localhost:8383/greenstone3/library. File→Settings in the Greenstone Server window gives you options to change the port number and which browser it uses by default. More notes about running Greenstone can be found in the README.txt file in the top level Greenstone folder.
To build collections, run GLI from the Start menu (Windows) or by running gli/gli.sh/bat in the top level Greenstone3 folder. Tutorial exercises about building collections in Greenstone 3 can be found here. Make sure you select the Greenstone3 tab at the top if it is not already selected.
Installing in text-only mode
* Refer to Running the installer in text-only mode
Adding source code to a binary release
Installing a source release
Setting up your Greenstone to run over https
The prevalent method of supporting https for tomcat based servers is to have a main apache httpd web server that works with https, then set up a
reverse proxy to let apache httpd redirect requests for GS3's tomcat server internally to Greenstone.
Release notes for 3.10 and 3.09 explains how to configure your GS3 to support https without an apache httpd web server and how to automate GS3 to obtain an https certificate for you from the free Certification Authority "Let's Encrypt".
PDF plugin restructuring and the NEW PDFv2Plugin
From GS3.09 onward, GS3 binaries have been including additional tools for converting from PDF to various text/html/image/image+text formats. (For GS2, only the nightly binaries at http://www.greenstone.org/caveat-emptor/?latest=latest will contain these changes.)
The old "PDFPlugin" remains deprecated but continues to be included in GS3.11. For details and instructions see the GS3.10 release notes.
If you have LibreOffice or OpenOffice installed, you can use the OpenOffice extension to process some newer versions of Microsoft Word, Excel and Powerpoint documents. Refer to the openoffice greenstone extension instructions.
Changing the admin password
Login to the administration page, 'edit' the admin account, and click 'change password'. Alternatively, you can login as admin via the login button at the top right of each page. Once you are logged in, this button will change to say 'admin'. Click this button and select 'account settings'. From there, you can select 'change password'.
Alternatively, you can set the admin password from the command line. See below.
Setting up your Greenstone OAI Server and using GLI to download over OAI from a Greenstone server
In Greenstone 3, collections should be available over OAI by default. Their collectionConfig.xml files already specify that each collection is OAI enabled, through use of an
OAIPMH serviceRack element. If you want to disable a collection from being accessibile over OAI, comment out the
OAIPMH serviceRack element in that collection's collectionConfig.xml. You would do so by embedding the entire element in
<!-- <serviceRack name="OAIPMH"> ... </serviceRack> -->
If you wish to validate the Greenstone 3 OAIServer, edit resources/oai/OAIConfig.xml to add in the adminEmail property to contain the email to where test results should be sent. Also set the repositoryId field to a ID name you want (e.g. to
greenstone), beware that there are some naming conventions that govern valid values for repositoryID. If testing the behaviour of the resumptionToken, set the resumeAfter element to a low value like 5. Then restart the Greenstone server.
To validate your OAI server, visit http://www.openarchives.org/Register/ValidateSite. Your server must be available over the internet to do this. The machine on which you're running the Greenstone 3 server will have to have its firewall and virtual server (port-forwarding) settings set up such that the Greenstone server can be made accessible to the outside world.
Setting up your Greenstone 3 OAI Server is covered in further detail in the tutorial http://wiki.greenstone.org/gsdoc/tutorial/gs3-current/en/GS_OAI_server.htm
For further information on your Greenstone OAI Server, please read through OAI support.
Webswing GLI replaces the GLI applet and the Remote GS3 server-client-GLI combination
With release 3.11, the Remote Greenstone 3 server, client-GLI and GLI Webstart Application (that replaced the GLI applet) have all been superseded by the new WebSwing GLI feature.
Important Changes and Bug Fixes
- Minimum Java version is now 1.8, and this is the version that is bundled in the releases. See also notes about Java 8 update 362.
- This release introduces the WebSwing GLI feature: GLI is now built into your browser. (But regular, desktop GLI is not going away, and still available as before.) WebSwing GLI replaces the client-GLI with Remote GS3 server combination and GLI applet/GLI Webstart application. Refer to Using WebSwing GLI (Web GLI) for instructions on how to start using WebSwing GLI. It's mostly the same as Desktop GLI, so the linked tutorial runs you through just the differences. See also Webswing GLI page for additional info on getting it working.
- GLI Changes: GUI support added for SOLR indexing, and for specifying sortfields (Lucece/SOLR) and facets (SOLR).
- CSVPlugin Changed - we have a new CSVPlugin which replaces both the old CSVPlugin and MetadataCSVPlugin plugins. This new plugin will assign the metadata to existing documents (specified by Filename field) or create dummy documents if no associated file is present. It handles input_encoding, and spaces in field names, There are many new options for finetuning its behaviour. If you would like to use the old versions of the plugins, these are now named CSVDeprecatedPlugin and MetadataCSVDeprecatedPlugin.
- The perl code has been updated to handle UTF16 surrogate pairs.
- The C++ code now compiles natively on Windows 64 bit.
- SOLR changes - support has been added for subcollections and language partitions, to match what you can do with other indexers. While faceted searching provides an equivalent functionality, there may be some cases where these index partitions suit the situation better.
- A cookie consent functionality has been implemented.
- Namespaced user groups, eg nz.ac.waikato - There is now support for "namespaced user groups" which are group names that use dots and look like a reversed domain name (but needn't be a real domain, just a sequence of characters separated by dots). Users who are members of namespaced user groups are automatically members of subsidiary groups (which is each prefix that can be formed upto any dot in the full namespaced user group name). For example, a user who is member of user group
nz.ac.waikato.cs.pinky, will be able to access not only documents and collections restricted to that very group,
nz.ac.waikato.cs.pinky, but will also have access to collections and documents restricted to
nz. Note however, that the group
nz.ac.waikato.cs.pinkydoes not give access to content restricted to a group whose name is not a prefix of it, like
- Groups now support users Users can now be made members of a group that is a domain reversed, e.g. user nz.ac.waikato.cs.pinky will hor any proper prefix thereof, and it will check users are members of any prefix of that group when choosing to give access to them.
- Developing a custom interface? You can now put the interface property files into web/interfaces/<my-interface>/resources instead of having to put them into the main web/WEB-INF/classes folder.
- Berry baskets have been replaced by Favourites - a more streamlined, accessible version. Customisation of favourites display is available via format statements. See the favourites wiki page for more details.
- From perl 5.22 and later, perl no longer comes with the perl CGI module, but Greenstone 3.11 bundles its own CGIModule and unpacks it for perl v5.22 onwards, so you should be good to go. If however, you find you don't have perl CGI module installed when you want to use remote gli or do online document editing, then manually unpack it as follows:
cd web/WEB-INF/cgi tar xzvf CGIModule.tar.gz mv CGIModule/* .
- Google SignIn - allows users to login to the library using their google credentials. Logging in will create a greenstone account for the user with no groups added. Requires a google client ID to be obtained, and set as a servlet parameter in resources/web/servlets.xml.in.
For ease of access this section has been brought across from the 3.10 Release Notes and earlier, but not all of it may be relevant to the 3.11 release.
Patches to 3.11
- To convert a collection's
collect.cfgfile to its GS3 equivalent, see here for details.
- When you've built a collection of documents, you may discover that there appears to be a copy of all these documents in the collection's import, archives and index subfolders and wonder whether Greenstone could really be so inefficient with disk space as to keep 3 copies of everything. As it happens though, Greenstone uses hard-links both on Linux and Windows, in order to keep just one set of your documents. Then it simply hardlinks to these, instead of making copies. By default, Windows doesn't show you when files on your filesystem are hard-linked. If you choose to install the Windows extension application Link Shell Extension (LSE), it will put red arrows on files that are hard linked.
Your browser doesn't remember you being logged into greenstone
The following scenario can occur if you set up GS3 with https, and your server.protocols property in build.properties contains both
https (i.e. you have
Switching between visiting your Greenstone 3 digital library (DL) using http and https URLs can result in the http version of the pages not remembering your login details despite you logging in. This can happen if you ever started off with the https version of the URL to a Greenstone3 DL page and then moved to using the http version of your GS3 URL, or if you ever logged in to your GS3 over https and then attempt to log in later using http.
The solution: The solution is to either start a private window if you want to access your GS3 DL pages over http, or to first clear your browser cookies related to your GS3 DL before swapping from https to http.
Using https causes session cookies to have the secure flag set to true. When a session cookie has the secure flag thus set, non http URLs cannot return that cookie in their subsequent requests to the server. Only https URLs can. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies section "Secure and HttpOnly cookies" which states "A secure cookie is only sent to the server with an encrypted request over the HTTPS protocol" and https://stackoverflow.com/questions/2321224/cookie-across-http-and-https-in-php
It further seems that in http mode, the browser does not want to overwrite secure cookies created in https mode with new cookies sent by the server when using http mode. Thus after using https and acquiring secure session cookies, the server can no longer track a user's session when they switch to http until the cookies are cleared either explicitly or through opening a private window.
Mac Installer fails
If running the Mac installer results in a message about having failed to copy to
permission denied), then restarting the Mac followed by running the installer once again worked for us, resulting in a successful installation.
Using the command line to reset the admin password when it isn't recognised
If at any stage, your admin password is no longer recognised from the Greenstone Reader interface (the Greenstone pages that you view through the web browser), then you can try to reset your admin password through the command line.
1. On Linux and Mac, open a terminal and use it to navigate to your Greenstone 3 installation folder.
On Windows, you can either open a DOS terminal the usual way and then likewise use it to navigate to your Greenstone 3 installation folder (using the
cd command). Or, you could open a Windows File Explorer first and use it to navigate to your Greenstone 3 installation. Then you could easily get a DOS prompt at that File Explorer location, as explained at stackoverflow:
Hold Shift while Right-Clicking a blank space in the desired folder to bring up a more verbose context menu. One of the options is Open Command Window Here. This works in Windows Vista, 7, 8, and 10.
2. Now that your terminal is at the GS3 installation folder, you can type the following command in the terminal:
4. It will ask for a new admin password. Type the new password and hit enter.
5. Once you restart the server, try out your new password.
Java 8.0.362 issues
We have found issues when running GLI using update 362 of Java 8. (These may occur with other updates). These are: errors when cancelling a build; GLI hanging when you add a new file type into a collection and add in the suggested new plugin. If you encounter this, please upgrade your Java 8 to the latest update (was 382 at the time of testing). Note, we bundle Java 8 update 301 with the release, which doesn't have these issues.
Changes to Tomcat port affects Solr collections
Currently when you change your tomcat port (either in build.properties, or using File→Settings in the Server program) the changes won't propagate to Solr which will still try to use the old port number. If you make changes to Tomcat port, please shutdown and restart the server from the Start Menu. If you are starting from within a terminal, you will need to shut down Greenstone and restart it from a fresh terminal.
Web document editor requires perl CGI module
Note: Greenstone 3.11rc1 and onwards should not exhibit CGIModule related perl issues, as 3.11rc1 and later will install the CGIModule if the perl on your system is version 5.22 or later. However, should you nevertheless encounter perl error messages about CGI.pm being missing, this section could be of help:
Editing documents through the web interface requires your system perl to have CGI installed. Without CGI, you can edit metadata and save, but your changes won't have been applied. You can tell this is the case by looking at the build log files in the collection, e.g. greenstone3/web/sites/localsite/collect/<collname>/log/build_log.1480553047299.txt. The message might say something like
We believe we've fixed the following issue in all instances where it may show up, but should you at any point encounter an error message as below
ERROR: Can't locate CGI.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ./gsdlCGI.pm line 9. BEGIN failed--compilation aborted at ./gsdlCGI.pm line 9. Compilation failed in require at (eval 1) line 1.
CGI.pm is not included with versions of perl from 5.22 and onwards, which is when this issue can result.
In such a case:
cd web/WEB-INF/cgi tar xzvf CGIModule.tar.gz mv CGIModule/* .
If you don't have web/WEB-INF/cgi/CGIModule.tar.gz in your installation, please install the CGI module for your system perl or try grabbing our prepared version from here and de-compressing it and moving its contents into web/WEB-INF/cgi.
Clicking a search result doesn't properly load the page nor take you to the search term when using old browsers
Problem: When you've found some document page(s) wherein a search term you searched for occurs, and clicking on such a document tries to load the document page but not sufficiently and doesn't take you to the section where the search term occurs.
Context: We've noticed this problem happens for us when using at least Firefox 45 and older.
Cause: The use of the
const keyword is not supported by the browsser. This happened for us even in Firefox version 45, despite
const support being reported as available in older Firefox versions per https://caniuse.com/const
- Put it into your GS3's web/interfaces/default/js folder
- Rename the existing
utility_scripts.jsfile already present there to
- Rename the newly downloaded
Using filenames containing &, < and > in GS3.11 collections
The Enrich panel in GS3.11's GLI is better able to handle
- &, < and >
- and other non-basic ASCII characters.
When you create a collection and assign it metadata in GLI that contains such characters, GLI will now successfully preserve them between GLI sessions.
The second set of characters listed above are still not recommended for use in metadata when dealing with a remote GS3 server, as that has to do with successfully transferring characters from an operating system in one file system encoding on the client side to possibly another operating system using another file system encoding on the GS3 server side.
Beware: It is only GLI's Enrich pane that has been fixed up to support both sets of characters in metadata. No other part of Greenstone has been fixed to cope with these characters, should they likewise struggle with them.
The fix is often in the form of adding hex encoded entities into the metadata.xml file and decoding them back for display in GLI's Enrich pane.
PDF to image conversion error on Linux
If you've configured a PDFPlugin to convert PDFs to images, increase the verbosity in Import Options and Build Options to 5 in GLI's Create panel.
When rebuilding the collection, check to see if you encounter the following error message mentioning that 'memory allocation failed', a 'corrupt image' at 'ReadPNGImage' and 'PostScript delegate failed':
import.pl> Converting pdf05-notext.pdf to pagedimg_jpg format import.pl> calling cmd "/usr/bin/perl" -S gsConvert.pl -verbose 5 -pdf_zoom 2 -errlog "/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/err.log" -output pagedimg_jpg "/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf" import.pl> Error executing pdfpstoimg.pl import.pl> pdfpstoimg error log: import.pl> convert: memory allocation failed `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadOnePNGImage/2160. import.pl> convert: corrupt image `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadPNGImage/3794. import.pl> convert: Postscript delegate failed `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/681. import.pl> convert: no images defined `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext/pdf05-notext.jpg' @ error/convert.c/ConvertImageCommand/3068. import.pl> Convert error for /research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf import.pl> Could not convert pdf05-notext.pdf to pagedimg_jpg format import.pl> convert: memory allocation failed `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadOnePNGImage/2160. import.pl> convert: corrupt image `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadPNGImage/3794. import.pl> convert: Postscript delegate failed `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/681. import.pl> convert: no images defined `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext/pdf05-notext.jpg' @ error/convert.c/ConvertImageCommand/3068. import.pl> Convert error for /research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf import.pl> Converting pdf05-notext.pdf to html format
If you see the above error message, then:
1. Use a text editor to open your Greenstone 3's gs2build/ext/imagemagick/linux/etc/ImageMagick/delegates.xml
2. Find the line that would say:
<delegate decode="ps:alpha" stealth="True" command=""gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pngalpha" -dTextAlphaBits=%u -dGraphicsAlphaBits=%u "-r%s" %s "-sOutputFile=%s" "-f%s" "-f%s""/>
The above specifies the PostScript delegate for PNG images. It has the sDEVICE set to pngalpha.
3. Change the line to:
<delegate decode="ps:alpha" stealth="True" command=""gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=**pnmraw**" -dTextAlphaBits=%u -dGraphicsAlphaBits=%u "-r%s" %s "-sOutputFile=%s" "-f%s" "-f%s""/>
The above changes the sDevice to pnmraw.
4. Save the file and re-run the build now.
Thanks to the following people for new and updated translations since 3.10:
- Lavji Zala for Gujarati translations
- Hilario Seo for Korean translations
- Yvan Arnaud and John Rose for French translations
- Tigran Zargaryan for Armenian translations