====== Greenstone 3.10 Release Notes ======
//This page may undergo further updates until the Greenstone 3.10 release and possibly for some time after. So continue to consult this page for the latest version of the 3.10 release notes.//
**Release Name:** 3.10
**Release Date:** 28 February 2021
** Released: **
* Binaries for Windows, GNU/Linux 32 and 64 bit machines, a Mac binary for Mac Mojave/10.14 and Catalina/10.15 and .\\ All the binaries have only been spot-tested, since the 3.10 release candidate 1 binaries, which came out some months before, were tested more extensively on all the tutorials. Source distributions and source components tested to compile on Mac Mojave/10.14, Windows 8.1, Ubuntu 32 (14.04) and 64 bit (16.04). \\ [[http://svn.greenstone.org/main/tags/3.10/|svn tag page]] [[https://trac.greenstone.org/browser/main/tags/3.10|trac tag page]]. Code revision up to 33119. Tag revision: 33122.
* The Mac binary is generated on Mojave/10.14. For rc1, most of the testing work was done on Catalina/10.15, with some features tested on Mojave/10.14, such as remote GS3. Final release's spot-testing done on Mojave/10.14.
* The Windows release candidate 1 binary was largely tested on Windows 10, but partially on Windows 8.1. Spot testing was on Windows 8.1.
* The Linux release-candidate 1 binary was largely tested on Ubuntu 18.04 LTS and partially on Ubuntu 14.04 and 16.04. Spot testing of the final release was on Ubuntu 16.04 LTS.
** Release Candidate History **
* Greenstone 3.10 rc1: Release Candidate 1. Released 6 Dec 2020.\\ Binaries for Windows, GNU/Linux 64 bit, Mac Mojave/Catalina. Mac version generated on MacOS Mojave/10.14 and tested predominantly on Catalina/10.15. See notes immediately below.\\ Download from the [[http://www.greenstone.org/snapshots|snapshots]] page\\ [[https://svn.greenstone.org/main/tags/3.10rc1/|svn tag page]] [[https://trac.greenstone.org/browser/main/tags/3.10rc1|trac tag page]]. Code revision up to 34587. Tag revision:34590.
**GS3.10rc1 issues and unknowns on Catalina Mac:**
* The IsisGdl binary, being 32-bit, does not work on Catalina, but does work on Mojave which was still backwards compatible with its bitness.
* Due to the Catalina test machine not allowing incoming connections, we weren't able to test its capabilities as a remote GS3 server or validate its OAI server online.
* Remote GS3 using GS3.10rc1 has been tested on Mac Catalina/Mojave as follows:
* client-GLI on Catalina, connecting to remote GS3 servers running on Windows and Linux machines
* GS3 remote server on Mojave, which client-GLI applications running on Windows, Linux and the same Mac machine itself connected to.
As ever, our thanks to our TSG (tech support) who made this and all Greenstone releases possible. Special thanks to our Tech Support staff member for Macs, Jacqui Elphick, who made Mac testing possible despite the difficult circumstances this year and thanks to whose efforts a Mac binary for the 3.10 series has become a reality.
===== Installation Instructions=====
==== Installing and running the binary release====
* Download the appropriate binary release for your operating system and run it.
* **Note to Mac users:** The security settings in newer Mac OS versions have been altered to by default disallow users from casually opening and running .dmg executables that are not from Apple itself. When attempting to open the Greenstone binary .dmg file, if it pops up an error warning about security, you will need to set up the Security on your Mac to allow you to run .dmg files downloaded from the internet. Otherwise the Greenstone mac binary will not run. To do this, go to your Mac's System Preferences. Under "Personal", select Security & Privacy. In the General tab, tick "Allow apps downloaded from Anywhere", then confirm that you want to "Allow From Anywhere". You'll need to be admin to do this, otherwise click on the padlock at the bottom left of the Security & Privacy tab and log in as admin.\\ **If you are on a Mac Catalina and do not have admin rights**, you can still get around this (for now) and install GS3.10 on Mac Catalina as follows:
- Ctrl+click (the Mac rightclick) and choose ''Open With'' //or something like that// to launch the installer ''.app'' inside the ''.dmg'' Mac binary file
- REPEAT: This //second// time you try to Ctrl+click and choose ''Open With'' to launch the installer ''.app'' inside the ''.dmg'' Mac binary file, Catalina will provide a buttonlink to allow you to "open anyway" and run the installer .app file in the ''.dmg''. After a brief time the attempt to launch the installer will fail without admin rights, mentioning something about ''/tmp/jre'' failing. //This is expected.// Press Cancel.
- Open a Finder window and navigate to the ''/tmp'' folder. (One way to do this, is to launch a terminal through Go > Utilities > Terminal, then type ''cd /tmp'' and press Enter. Next type: ''open .'' and press Enter to get a Finder window opened at the ''/tmp'' folder)
- Now Ctrl+click on the ''jre'' inside ''/tmp''. Press Open (or maybe the button link is labelled Run) to attempt to extract the jre. This will eventually fail as the ''.dmg'' launcher did before. That's fine too. Press Cancel.
- Now return to the installer .app file inside the .dmg. Ctrl+click on it and launch the installer once more. This time the launch process will complete and within a minute or so you should see the Greenstone Installer opening dialog appear.
* **For Linux**, you will need to set the file to be executable before running it. e.g. \\ In a terminal ''chmod a+x Greenstone-3.10-linux''\\ or on Ubuntu, you can right-click on the installer, choose Properties, Go to the Permissions tab and ensure the "Execute: Allow executing file as program" tickbox is ticked.
* The installer initially unpacks into a temporary directory (/tmp on linux). If you wish to change this, set the TMPDIR environment variable.
Note that in some cases, the following **doesn't** work
export TMPDIR=/something/else
./Greenstone-3.10-linux
Use the following instead
TMPDIR=/something/else ./Greenstone-3.10-linux
* **For Windows users:** To install, download the file and double-click. On newer versions of Windows 10: If the green installer splash screen doesn't appear after a few seconds, then right click on the installer executable, choose Properties, go to the General tab and at the bottom (in the Security section) look for a tick box labelled Unblock and tick this.
=== When the installer is running ===
During the installation process you will be presented with several options. For many, the default settings will be sufficient. Some important options are
* Folder where you want greenstone3 to be installed.
* Choosing which packages to install.
* Greenstone3 will install the Apache Tomcat webserver by default. You can choose not to install it, but then you will need to set up your own version of Tomcat to serve Greenstone. We recommend using Greenstone's Tomcat, at least initially while you get everything set up.
* ImageMagick is bundled with Greenstone for binary web releases for all platforms, and includes JPEG2000 support. You can choose not to install it if you already have ImageMagick previously installed.
* Ghostscript is now bundled with Greenstone for binary web releases for Windows and Mac. You can choose not to install it if you already have Ghostscript previously installed.
* Choosing a password for the administration pages. These pages allow the admin user to inspect and manage the list of registered Greenstone users. You can add new users, and change group settings for existing users. Greenstone user registration is needed if you want to use remote GLI login to the Greenstone server, or if you want to make collections/documents only accessible by certain groups of users. (//If the password is not set during installation, the default password for the 'admin' user is 'admin'. You can change the password any time after installation, by running the Greenstone 3 server and visiting the Administration pages. See [[#disabling_admin_access_in_installer| below]]//).
Once you have successfully installed Greenstone3, you can start up the server by choosing Grenstone3 Digital Library from the Start menu (Windows) or running gs3-server.sh/bat. This launches a small server program which starts up Tomcat and launches a browser. A small window pops up which allows you to change some settings for your library and restart the Tomcat server. Closing this program will stop Tomcat running. By default, your library will be available at localhost:8383/greenstone3/library. File->Settings in the Greenstone Server window gives you options to change the port number and which browser it uses by default.
More notes about running Greenstone can be found in the README.txt file in the top level Greenstone folder.
To build collections, run GLI from the Start menu (Windows) or by running gli/gli.sh/bat in the top level Greenstone3 folder. Tutorial exercises about building collections in Greenstone 3 can be found [[en:tutorials | here]]. Make sure you select the Greenstone3 tab at the top if it is not already selected.
==== Installing in text-only mode ====
* Refer to [[http://wiki.greenstone.org/doku.php?id=old:user_advanced:installation#running_the_installer_in_text-only_mode|Running the installer in text-only mode]]
==== Adding source code to a binary release ====
* [[http://wiki.greenstone.org/doku.php?id=old:user_advanced:installation#source_component|Windows: Source Component instructions]]
* [[http://wiki.greenstone.org/doku.php?id=old:user_advanced:installation#source_component1|Linux/Mac: Source Component instructions]]
==== Installing a source release ====
* [[http://wiki.greenstone.org/doku.php?id=old:user_advanced:installation#source_distribution|Windows: Source Distribution instructions]]
* [[http://wiki.greenstone.org/doku.php?id=old:user_advanced:installation#source_distribution1|Linux/Mac: Source Distribution instructions]]
===== Further instructions =====
==== New (client-)GLI features ====
**a. Metadata to CSV options:**
- File > Export to metadata CSV: for a collection you have open, this option creates a metadata.csv file in a location of your choosing containing all the metadata you can see in GLI, including inherited metadata. If the metadata.csv file you selected already exists, then the metadata you see in GLI is amalgamated with the selected CSV file. This option allows you to backup your collection’s metadata to a spreadsheet file. There is NO RECONVERT feature, to convert back to metadata.xml files from metadata csv format. But you can build your collection with metadata from the CSV spreadsheet. See the following option below which explains how to redo your collection to work with metadata from a spreadsheet instead of using metadata in GLI/metadata.xml files.
- File > Convert to metadata CSV: for the collection you have open, this option creates a metadata.csv file in your collection’s “import” folder by default (which is the best location), by destructively removing all the metadata from the collection’s metadata.xml files (in other words, by removing the metadata you see in GLI) and shifting them out into the selected metadata.csv file. If you selected an existing metadata.csv file, then any metadata you currently see in GLI is amalgamated with the selected CSV file, before it gets removed from GLI/the metadata.xml files. Selecting this option prepares your collection so that you can switch over to using a MetadataCSVPlugin, configured with metadata_value_separator field set to “\|”, to then rebuild your collection producing the same results as before.
**b. Collection security skeleton elements** as discussed at http://wiki.greenstone.org/doku.php?id=en:user_advanced:security, can now be added through (client-)GLI’s Edit > Edit collectionConfig.xml menu option. At the bottom of the Config File Editor dialog that appears, you will find a small toolbar that allows you to choose which (skeleton) XML element to add:
- to hide the current collection,
- to add the appropriate element to make the entire collection private except for one or more groups you specify,
- to add the appropriate element to make all the docs in the collection private except for the groups you specify (adds a element),
- to add the appropriate element to make select docs in the collection private except for the groups you specify (where you can then specify which docs as explained on the wiki link already provided),
- to add a further element to the existing element
- to add a further element into the existing element
One issue with this remains: if you want to undo the addition of a security element, you have to press the Undo button twice at present. I haven’t yet figured out why this is. (If you press Undo once, the entire XML content of your collectionConfig.xml becomes empty, so you’ll naturally press Undo again in alarm and then it will look right again.)
==== Setting up your Greenstone to run over https ====
//**Note:** This section is untested for this release, as the prevalent method of supporting https for tomcat based servers appears to be to have a main apache httpd web server that works with https, then set up a ''reverse proxy'' to let apache httpd redirect requests for GS3's tomcat server internally to Greenstone.//
The more secure https protocol is increasingly required by browsers and gradually superseding http. Given that you meet the following requirements and configure your GS3 as below, Greenstone 3 has now been automated to obtain an https certificate for you from the free Certification Authority "Let's Encrypt".
Requirements: because we need to temporarily run a server on port 80 to get a certificate issued and because port 80 has some access restrictions surrounding it on most machines,
* on unix (linux and mac) systems you need to have sudo permissions
* on windows, you probably need admin rights
* ensure nothing is running on port 80 when you're ready to set up https certification for your GS3
Steps:
- Edit build.properties as follows:
* set ''tomcat.server'' to the //primary// hostname/domain name that you want your Greenstone3 to run as and which is to be registered in your certificate. This would be the host name of your machine.
* set a value for ''keystore.pass''.\\ This will be the password on your final certificate used by tomcat.
* Ensure ''server.protocols'' contains ''https''.\\ The ''server.protocols'' property is a comma-separated list that indicates which protocols are to be supported by your Greenstone 3 server. This property can be set to one of
* ''http''
* ''https''
* ''http,https''
* ''https,http''\\ The first in the list becomes the default protocol used for previewing with the GS3 server application, ''gs3-server''.
* By default ''tomcat.port.https'' is set to 8443. Ensure this port is not already in use, otherwise change it to a port value that's not in use.
- Make sure you have read and agree with the [[https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf|Let's Encrypt Subscriber Agreement]]
- Use a terminal to go into your GS3 installation folder, run ''gs3-setup.bat'' on windows and ''source ./gs3-setup.sh'' on linux and mac to set up the GS3 environment, then run the ''ant setup-https-cert'' target. For example on Linux,\\ cd /path/to/GS3
source ./gs3-setup.sh
ant setup-https-cert
\\ You'll be asked for an **email** that Let's Encrypt can optionally communicate to you on, as well as **any additional domain names** you want in the //same// certificate (additional domains are **untested**), and whether you **agree** with the Let's Encrypt Subscriber Agreement.\\ **On linux or mac, you may be asked to enter your sudo password** to briefly run a server on port 80. **Note:** On Mac and windows, GS3 uses the **//third party//** [[https://zerossl.com/usage.html|ZeroSSL]] to get Let's Encrypt to issue certificates, which will result in GS3's own tomcat server to be run on port 80 during certificate issuance. On Linux, we use //Let's Encrypt//'s own certbot-auto script for the certification process, and have it set to run a standalone temporary server on port 80.
- Once the setup-https-cert ant target has finished, you can start your web GS3 server by either running the gs3-server application or by running "ant start" from the terminal.
- If you ran the gs3-server application, press the Enter Library button to open your DL home page. If you ran ''ant start'' from the command line, then open a browser manually. Point your browser to ''https:%%//%%:/greenstone3/library'', adjusting the tomcat.server and tomcat.port.https values as per what you set for these properties in your GS3 installation folder's toplevel ''build.properties'' file.\\ //**BE AWARE:**//
* when visiting your digital library over http**s**, remember to prefix htt**ps** to the address and use the htt**ps** port defined with ''tomcat.port.https'', and do not use ''localhost'' but the actual domain you registered which is defined in the ''tomcat.server'' property you set.
* If you've also turned on http support alongside https, then to visit your DL over regular http, return the URL to having the htt**p** prefix and remember to change the port number to what's defined for htt**p** in property ''tomcat.port.http''. For http URLs, you can use the domain name set in ''tomcat.server'' too, or try either of ''localhost'' or ''127.0.0.1'' denoting the local host machine.
- Once your https home page has loaded, confirm that your certificate is properly installed by looking for a green padlock next to the address bar. (Depending on your browser, you can click the padlock to get more information on the certificate issuer.)
There are 2 more https-related automated ant targets you can run from the command line:
* ''ant remove-https-cert'': to revoke your https certificate
* ''ant renew-existing-https-cert'': This is to renew a certificate that you'd already earlier obtained with ''ant setup-https-cert'' explained above. A Let's Encrypt certificate needs renewing every 90 days, at which point your certificate will need to be reinstalled. For renewal, you will once more need to ensure all the same conditions as for issuance (the same conditions as when you ran ''ant setup-https-cert''), such as nothing running on port 80. Since renewal reinstall your certificate, you will need to stop your GS3 server first before running the ''ant renew-existing-https-cert'' target, then after the target has finished, run your GS3 server once more. Renewal will not take place despite running the ''ant renew-existing-https-cert'' target unless the approximate time for expiry has been reached (+/- 10 days on Windows/Mac).
**Important:** Beware that if you've configured your GS3 to support http and https, by setting the ''server.protocols'' property to include both http and https, then switching between the two protocols when you visit your GS3 pages in your browser could result in the //http// variants of GS3 web pages not remembering you when you log in to them. For the solution and for further details, consult the section [[en:release:3.09_release_notes#troubleshooting|Troubleshooting > Your browser doesn't remember you being logged into greenstone]].
==== PDF plugin restructuring and the NEW PDFv2Plugin ====
From GS3.09 onward, GS3 binaries have been including additional tools for converting from PDF to various text/html/image/image+text formats. (For GS2, only the nightly binaries at http://www.greenstone.org/caveat-emptor/?latest=latest will contain these changes.)
The old "PDFPlugin" remains deprecated but continues to be included in GS3.10. In its place, there are 2 plugins to handle PDFs:
* "//PDFv1Plugin//" which is the same as the old PDFPlugin but minus the PDFBox_conversion option. It returns to using the old ''pdftohtml'' tool to do the conversions, and is limited to older versions of PDFs.
* **the recommended "//PDFv2Plugin//"**, which will contain the new functionality and should handle a greater range of PDF versions, including the newer ones that the old ''pdftohtml'' (now used by PDFv1Plugin) can't handle. The "PDFBox conversion" facility has been moved to the new PDFv2Plugin, but is now invisible: it will be triggered automatically depending on the "convert_to" format that you select when you Configure the PDFv2Plugin. PDFv2Plugin also uses additional conversion tools in the background to support the additional output formats.
For the eventual 3.09 release, the old PDFPlugin that you're familiar with, the one which has the ''pdfbox_conversion'' flag but also makes use of the old ''pdftohtml'' tool behind the scenes, will hang around with a deprecated warning, to allow people to port over their collections and keep rebuilding with the old settings or to rebuild their collection with one of the 2 new PDF plugins. However, **new collections will have the //PDFv2Plugin// in the Document Plugins pipeline by default for GS3, and PDFv1Plugin by default for GS2, since GS2 doesn't come with the PDFbox extension out of the box.** So GS2 users will have to manually add in PDFv2Plugin in place of PDFv1Plugin for new collections, after setting up the pdfbox extension. But then it should work as usual.
The "convert_to" options/output formats of the new PDFv2Plugin are:
* ''text'': a single stream of text;
* ''html'': a single stream of basic html from just the extracted text, no images;
* ''pretty_html'': each page is now an HTML page consisting of extracted text overlaid on top of a screenshot of the rest of the PDF page;
* ''paged_pretty_html'' (also the default when convert_to is set to auto): ''pretty_html'', but each page is a section;
* ''pagedimg_'': every PDF page as an image, sectionalised by page. Not searchable, since there's only images;
* ''pagedimgtxt_'': every PDF page as an image plus that page's extracted text, sectionalised by page.
As always, text is only extracted from a PDF where extractable. This depends on user permissions for a PDF, whether the PDF contains actual extractable text and not just images of text, whether the PDF is undamaged, and any other such factors.
There may be further adjustments made, including to display strings, but so far, we've decided on the above output formats and they seem to work on my regular PDF test documents.
==== OpenOffice extension ====
If you have LibreOffice or OpenOffice installed, you can use the OpenOffice extension to process some newer versions of Microsoft Word, Excel and Powerpoint documents. Refer to the [[http://wiki.greenstone.org/doku.php?id=en:user_advanced:greenstone_extensions|openoffice greenstone extension]] instructions.
====Changing the admin password====
Login to the administration page, 'edit' the admin account, and click 'change password'. Alternatively, you can login as admin via the login button at the top right of each page. Once you are logged in, this button will change to say 'admin'. Click this button and select 'account settings'. From there, you can select 'change password'.
==== Setting up your Greenstone OAI Server and using GLI to download over OAI from a Greenstone server ====
In Greenstone 3, collections should be available over OAI by default. Their collectionConfig.xml files already specify that each collection is OAI enabled, through use of an ''OAIPMH serviceRack'' element. If you want to disable a collection from being accessibile over OAI, comment out the ''OAIPMH serviceRack'' element in that collection's collectionConfig.xml. You would do so by embedding the entire element in '''' comment markers:
If you wish to validate the Greenstone 3 OAIServer, edit **resources/oai/OAIConfig.xml** to add in the **adminEmail** property to contain the email to where test results should be sent. Also set the **repositoryId** field to a ID name you want (e.g. to ''greenstone''), beware that there are some naming conventions that govern valid values for repositoryID. If testing the behaviour of the resumptionToken, set the **resumeAfter** element to a low value like 5. Then restart the Greenstone server.
To validate your OAI server, visit http://www.openarchives.org/Register/ValidateSite. Your server must be available over the internet to do this. The machine on which you're running the Greenstone 3 server will have to have its firewall and virtual server (port-forwarding) settings set up such that the Greenstone server can be made accessible to the outside world.
Setting up your Greenstone 3 OAI Server is covered in further detail in the tutorial http://wiki.greenstone.org/gsdoc/tutorial/gs3-current/en/GS_OAI_server.htm
For further information on your Greenstone OAI Server, please read through [[en:user_advanced:oai|OAI support]].
==== Setting up a remote Greenstone 3 server ====
This will allow remote client-GLI applications to connect to your Greenstone server, to remotely create and upload new collections to be built and hosted by your server machine.
**Remote Greenstone 3 Server**
To install the server-side functionality:
1. If you're on Windows, you will need to teach Greenstone where the perl executable is.
You can do this either manually, by editing a couple of Greenstone config files as explained just below, or you could run the Greenstone server once and press ''Enter Library'' button to visit your library home page. Doing so will automatically set up the perl path in various Greenstone files.
To do this manually on Windows,
a. Open Tomcat's conf/web.xml file for editing (found in greenstone3/packages/tomcat/conf folder, if installed in the default location), as you may need to specify the full path of the Perl library for the parameter "executable" of **CGIServlet**. This takes the form:
executable
C:\Program Files\greenstone3\gs2build\bin\windows\perl\bin\perl.exe
b. Edit the first line of the greenstone3/web/WEB-INF/cgi/gliserver.pl file and specify the full path of the perl binary.
On Windows this will be (if installed in the default location):
#!C:\Program Files\greenstone3\gs2build\bin\windows\perl\bin\perl -w
2. Make the Greenstone "collect" directory, located in web/sites/localsite, writeable by the webserver user.
On Unix, you would do
chmod -R a+w /path/to/your/GS3/web/sites/localsite/collect
On Windows, run in a DOS prompt:
cacls "C:\Program Files\Greenstone3\web\sites\localsite\collect" /P Everyone:F
3. Open up the file build.properties located in your greenstone installation folder. Edit the //tomcat.server// property's value to refer to your server machine's hostname instead of leaving it at the default value of "localhost":
# tomcat info
tomcat.server=your-server-computer-name
Once the server is started up, this will update the same property in greenstone's web/WEB-INF/classes/global.properties file. Then images viewed from a browser on the client side will refer to the correct location on the remote machine.
(If you don't know what your machine's host name is and you're on Windows, then open a DOS prompt and type:
ipconfig /all
Scroll to the top of the output that gets printed to the screen and note what it mentions for //HOST NAME//. Also note the //DNS Suffix Search List//.
Put these two together with a period mark to separate them and use this as the value for your tomcat.server property.)
4. Set up your Greenstone environment if you've not already done so by running ''gs3-setup.bat'' in your Greenstone 3 installation folder (''source ./gs3-setup.bash'' on Linux and Mac machines). And then start up your Greenstone 3 server with:
ant start
(or ''ant restart'')
5. Check that Tomcat and Greenstone3 are working correctly by visiting
http://:/greenstone3/library
6. Add some user accounts by visiting the Greenstone 3 home page (http://YOURHOST:YOURPORT/greenstone3/library), clicking the ''admin'' link at the top right and logging in. The username is ''admin''. By default, the password is ''admin'' too, unless you already set this during the installation process or if you changed this afterwards.
Once logged in, go to the Administration page. You can access this via the link on the home page. (Or you can click the admin link, choose ''Account Settings'', and then click the ''Administration Page'' link on the top left. )
Add a new user by providing a new username, setting a password for the user that's a minimum of 3 characters long, and by using the drop-down provided for the ''Groups'' field to one or more of the available options, such as ''personal-collections-editor''.
Even if only the ''admin'' user wishes to use the client-gli, they will still need to log in to the Administration Page once after installation in order for the user database to be set up.
7. Finally, visit the following page in the web browser to test that your remote Greenstone server is set up properly:
http://:/greenstone3/cgi-bin/gliserver.pl?cmd=check-installation
You should get a message saying "Java found" and "Installation OK!" Important: You cannot continue until this is successful, as the Remote Greenstone 3 server will not work without it!
If you get a message saying "Java failed"
* check that the Java run-time is installed and on the webserver's path. If you get a "500 Internal Server Error", check the error log of your webserver for the cause (greenstone3/web/logs/greenstone.log).
* consult the more detailed instructions on the [[en:user_advanced:remote_greenstone|Remote Greenstone page]] for further steps that may be necessary to carry out.
If you are using perl 5.22 or later, these no longer come with the perl CGI module. Please download https://trac.greenstone.org/export/36059/main/trunk/greenstone2/common-src/cgi-bin/CGIModule.tar.gz into web/WEB-INF/cgi, untar it, and copy the contents of the resulting CGIModule folder up one level (i.e. outside of that folder).
**Client-GLI**
Assuming that the remote Greenstone server is accessible to the outside world and you're not behind a firewall, you can access the remote Greenstone server from a client-gli application installed on any other machine. To do so,
1. Run client-gli quite as you would GLI. It's accessible from the Windows Start menu, otherwise you can run the client-gli script (located at the toplevel of a Greenstone installation) from a terminal.
2. You'll be asked for the gliserver.pl URL of the remote Greenstone 3 server that you wish to connect to. This is of the form
http://:/greenstone3/cgi-bin/gliserver.pl
It's the same URL as in Step 10 of setting up the remote GS3 server above.
===GLI Java Web Start application (replacement for GLI Applet) - Additional Steps===
//This feature untested in 3.10 and rc1, but tested for the 3.09 release. It is possible that Java support for Web Start applications is or will be deprecated too, as was the case with Java Applets.//
**You will need a Java Development Kit (JDK) v 7 or 8 installed for this.**
Many browsers have stopped supporting Java applets including Microsoft Edge, though Microsoft's Internet Explorer still supported it. For this reason, 3.09's GLI is now no longer provided as an applet, but has been converted into a Java Web Start application.
Instructions for using the GLI Java Web Start, which works over the JNLP protocol, are below.
- First follow the instructions above for [[http://wiki.greenstone.org/doku.php?id=en:release:3.09_release_notes#setting_up_a_remote_greenstone_3_server | setting up the Remote GS Server]]
- Next, generate the SignedGatherer.jar as follows. Be aware that to run the ''jarsigner'' utilities, you will need a JDK installed.
* Use a terminal to go into the Greenstone "gli" directory, then run keytool -genkey -alias privateKey -keystore appletstore -storepass greenstone Enter the appropriate details for your organization. When it asks to enter the key password for , choose your own password or hit Enter to use "greenstone".
* Next, run /PATH/TO/YOUR/JDK_INSTALLED_FOLDER/bin/jarsigner -keystore appletstore -signedjar SignedGatherer.jar GLI.jar privateKey
When it prompts, enter the password you used above (''greenstone'').
- Move the created ''SignedGatherer.jar'' file from the ''gli'' directory into GS3's ''web/applet'' subdirectory.
- You need to associate JNLP files with Java Web Start (''jre/bin/javaws'').
* On Windows, create the association in the usual way: when you first access the GLI Web Start application through Greenstone, a JNLP file called "GLIappWebStart.jnlp" will be offered for launching or download. If JavaWS is not already the default application to open JNLP files with, rightclick on the downloaded GLIappWebStart.jnlp file and choose ''Launch/Open with ...''. Browse to your Greenstone3's ''packages/jre/bin/javaws.exe'' or any installed Java's ''jre/bin/javaws.exe'' to use Java's Web Start application as the launcher.
* On Mac, .jnlp files may already be associated with the Java Web Start application for launching JNLP files (Java web start applications). Otherwise, see page 3 of http://online.unm.edu/help/learn/faculty/tools/web-conferencing/common/web-conferencing-pdf-files/jnlp-file-association.pdf for instructions on creating the file association for this. Once the file association between the Java Web Start application and .jnlp files has been made, you can just double-click on the ''GLIappWebStart.jnlp'' file you downloaded.
* For Linux, create a file with ''.desktop'' extension (e.g. "javawebstart.desktop") containing the following, **edit the path to javaws**, and save this file into ''~/.local/share/applications'':\\ # This file makes Ubuntu associate .jnlp files with Java Web Start (javaws)
# This file should be adjusted and then copied into ~/.local/share/applications
# as a file with .desktop extension, e.g. javawebstart.desktop
# https://askubuntu.com/questions/235861/how-to-associate-jnlp-file-with-javaws
[Desktop Entry]
Encoding=UTF-8
Name=Java 7 Web Start
Comment=Java 7 Web Start
Exec=/EDIT-THIS-PATH-TO-YOUR-JAVA-JRE/bin/javaws %u
Terminal=false
Type=Application
Icon=javaws
Categories=Application;Network;
MimeType=application/x-java-jnlp-file;
- Launch the Java Control Panel by running ''jre/bin/javacpl.exe'' on Windows, or ''jre/bin/jcontrol'' (formerly jre/bin/ControlPanel) on Linux or follow the instructions at https://www.java.com/en/download/help/mac_controlpanel.xml for mac. (GS3 binaries now include a JRE in the ''packages'' folder if you want to use the bundled JRE.) In the Java Control Panel, go to the ''Security'' tab, set Security level to ''High'' if not already set. Click ''Edit Site List'', and then press ''Add'' to add the //http:%%//%%host:port// (or //https:%%//%%host:HTTPS-port// if you've [[http://wiki.greenstone.org/doku.php?id=en:release:3.09_release_notes#setting_up_your_greenstone_to_run_over_https|set up https support]]) that the GS3 will run on. Remember, to be accessible to the outside world, the host can't be "localhost", but would be the hostname of your machine or public IP. Click OK to exit the Java Control Panel with your changes in effect.
- Make the GLI link on the home page active: Open web/interfaces/default/transform/pages/home.xsl for editing, find the line
and remove the comments. i.e. change it to
Save the changes and close the file.
- (Re)Start the GS3 web server and visit your DL library home page, ''http://[hostname]:8383/greenstone3/library''.
- Since you have set up the JNLP file association in a previous step, you can now click on the "The Librarian Interface" link and your browser should offer to save or launch a file called ''GLIappWebStart.jnlp''. Click ''Open With'' and in the dropdown box beside it, one of the launcher applications, at least on Windows, should be the "Java(TM) Web Start Launcher" application (javaws.exe) that you associated with .jnlp file extensions. Choose that application as the launcher. If the browser is able to successfully launch ''GLIappWebStart.jnlp'', then Java Web Start will be used to run the GLI application indicated by the JNLP file. If launching through the browser is not possible, for example, if the launcher application is not listed, as may happen on Linux machines, then choose to save the JNLP file. It will download the file to a temporary user area (like C:\Users\\AppData\Local\Temp on windows). And then you can rightclick on the downloaded GLIappWebStart.jnlp file, to launch it with the Java Web Start program you already associated with this file type. This way should work on both Linux and Windows.
- You will need to **authorise** the GLI to run. After that, the JNLP version of the GLI Applet will eventually run and behave like the usual client-GLI (and like the old GLI applet) from this point onward: starting with providing your account username and password on Greenstone that you created when setting up the remote GS3 server. You may also be asked for your proxy authentication details if set up for behind a proxy.
=== Converting a GS2 collection to GS3 when working with a remote GS3 server ===
The new Format Conversion Wizard to convert GS2 format statements to GS3 format statements (see [[en:user:gs2_to_gs3 |this page]]) only appears when you're working with GLI, not client-GLI. The client-GLI for GS3 will only perform the most basic initial step in the conversion process, which is to preserve the GS2 format statements in inactive XML tags in the new collection's collectionConfig.xml.
However, if you have a local Greenstone 3 installed, you can still manage to convert a remote collection's ''collect.cfg'' file to its GS3 equivalent. See [[en:user:gs2_to_gs3#using_a_remote_greenstone_server |here]] for details.
===== Important Changes and Bug Fixes =====
* Bundling of Apache Tika for [[http://wiki.greenstone.org/doku.php?id=en:plugin:unknownconverterplugin|better support for docx files]]. An UnknownConverterPlugin is largely set up to use Tika for docx processing. But you will need JRE8 installed locally into your GS3 and Windows users will further need to tweak the UnknownConvertPlugin configuration: [[http://wiki.greenstone.org/doku.php?id=en:plugin:unknownconverterplugin#steps_for_310_users|instructions for both here]]. Tika can also be used as a fallback for extracting text from a wide range of documents, by configuring an UnknownConverterPlugin in similar fashion for other document types as determined by file extensions.
* There has been a change to the naming of custom properties files for a collection. Instead of having resources/collname.properties (eg resources/niupepa.properties), now the file is called resources/interface_custom.properties. If you already have a custom properties file, then please rename it.
* 2 new features in GLI: exporting metadata fron GLI to a CSV file, and some graphical help setting up document or collection level security when editing the collection configuration through GLI. Described [[http://wiki.greenstone.org/doku.php?id=en:release:3.10_release_notes#new_client-_gli_features|in this section]].
* Better filename support in GLI:
* GLI now supports the & character in filenames.
* Better GLI support for user-assigned metadata in UTF-8 and possibly other character sets.
* GLI will now also load Greenstone extracted (ex.*) metadata for documents of a built collection whose filenames are in UTF-8 or possibly other character sets. Any nested subfolders within which such documents are contained can have folder names that are in UTF-8 or possibly other character sets also.\\ This will now also work if a document type's plugin has the ''file_rename_method'' (under BaseImporter section of the plugin configuration) set to Base64.
* More robust client-gli and remote server interaction, with [[http://www.greenstone.org/blog/2020-07-24/improvements-to-remote-gs3-and-client-gli/|a number of bugfixes]] enabling better real-word usage. No long delay typing up format statements in client-GLI.
* Better support for running Greenstone 3 on a non-English locale, tested with Mandarin. (For the previous 3.09 release, the [[http://wiki.greenstone.org/doku.php?id=en:supporting_other_locales|workaround is described here]].)
* New property in build.properties on Diego Spano's request: [[https://trac.greenstone.org/changeset/33461|tomcat.user.allowLinking]]
* Some additional work done on mapGPS.
* Expanding on GS3 customisation tutorials: [[http://wiki.greenstone.org/doku.php?id=en:user:expanding_on_gs3_customisation_tutorials|incorporating Depositor and Collection Groups]].
===== IMPORTANT information =====
For ease of access this section has been brought across from the [[en:release:3.08_release_notes|3.08 Release Notes]] and [[en:release:3.09_release_notes|3.09 Release Notes]], but not all of it may be relevant to the 3.10 release.
==== Patches to 3.10 ====
==== Troubleshooting ====
=== Some perl scripts don't run properly or to completion ===
//**If you're running Greenstone 3 on a non-English locale**// and certain perl scripts don't run properly or completely, such as when you run ''perl -S pluginfol.pl -describeall'', then the problem may be locale/encoding related. To resolve this issue, refer to [[en:supporting_other_locales|Supporting running Greenstone in other locales on Windows 10]].
=== Content Encoding Error when visiting the local solr servlet page ===
If you see a Content Encoding Error when opening your GS3's solr servlet page at ''http://127.0.0.1:8383/solr'' or ''http://localhost:8383/solr'' in your browser, then this may have to do with the version of Java you have installed on your machine. From GS3.09 onward, if your machine has its own Java installed, then assuming that its version is sufficient and its bit-architecture (32 or 64 bit) matches, Greenstone will use your Java in preference to the bundled Java Runtime (JRE) that Greenstone ships with. We found that a recent version of Java (version 1.8.0_161 was problematic for us), caused the Content Encoding Error when visiting the solr servlet, whereas the bundled JRE and slightly earlier and much newer versions of Java such as 1.8.0_144 and 1.8.0_191 did not have these issues.
**Solution:** if you have a problematic version of Java installed,
- either unset JAVA_HOME and remove this Java's ''bin'' folder from the PATH environment variable too, thus helping Greenstone 3 use its bundled JRE instead
- install a newer version of Java on your system. We found that the current latest one, 1.8.0_191 worked successfully for this purpose.
=== SIGPIPE errors when building a collection ===
We've added a work around to one kind of SIGPIPE errors which could occur with large collections when using ''solr'' as indexer. However, a couple of people on the mailing list encountered SIGPIPE errors on occasions when solr was not the indexer. **If your collection is using ''jdbm'' as the database type** and the error messages surrounding the SIGPIPE mention issues with "transaction commit", then Mariana Pichinini on the mailing list found that the following helped:
* change the database type from ''jdbm'' to ''gdbm''
* or leave the database type at ''jdbm'' and move your GS3's bundled JRE (the GS3's ''packages/jre'' subfolder) outside your GS3 installation. Next install a newer Java on your system so that GS3.09 can find that. If on Linux, ensure you open a new terminal before running GLI or command line building your collection.
=== Your browser doesn't remember you being logged into greenstone ===
**The issue:**
The following scenario can occur if you set up GS3 with https, and your server.protocols property in build.properties contains both ''http'' and ''https'' (i.e. you have ''server.protocols=http,https'' or ''server.protocols=https,http'').
Switching between visiting your Greenstone 3 digital library (DL) using http and https URLs can result in the http version of the pages not remembering your login details despite you logging in. This can happen if you ever started off with the https version of the URL to a Greenstone3 DL page and then moved to using the http version of your GS3 URL, or if you ever logged in to your GS3 over https and then attempt to log in later using http.
**The solution:**
The solution is to either start a private window if you want to access your GS3 DL pages over http, or to first clear your browser cookies related to your GS3 DL before swapping from https to http.
**The cause:**
Using https causes session cookies to have the secure flag set to true. When a session cookie has the secure flag thus set, non http URLs cannot return that cookie in their subsequent requests to the server. Only https URLs can. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies section "Secure and HttpOnly cookies" which states "A secure cookie is only sent to the server with an encrypted request over the HTTPS protocol" and https://stackoverflow.com/questions/2321224/cookie-across-http-and-https-in-php\\ It further seems that in http mode, the browser does not want to overwrite secure cookies created in https mode with new cookies sent by the server when using http mode. Thus after using https and acquiring secure session cookies, the server can no longer track a user's session when they switch to http until the cookies are cleared either explicitly or through opening a private window.
=== Mac Installer fails ===
If running the Mac installer results in a message about having failed to copy to ''/var/folders/zz/'' ''permission denied''), then restarting the Mac followed by running the installer once again worked for us, resulting in a successful installation.
=== Using the command line to reset the admin password when it isn't recognised ===
If at any stage, your admin password is no longer recognised from the Greenstone Reader interface (the Greenstone pages that you view through the web browser), then you can try to reset your admin password through the command line.
1. On Linux and Mac, open a terminal and use it to navigate to your Greenstone 3 installation folder.
On Windows, you can either [[http://wiki.greenstone.org/doku.php?id=old:miscellaneous_questions&s[]=dos#how_do_i_open_a_terminal_called_dos_console_in_windows | open a DOS terminal the usual way]] and then likewise use it to navigate to your Greenstone 3 installation folder (using the ''cd'' command). Or, you could open a Windows File Explorer first and use it to navigate to your Greenstone 3 installation. Then you could easily get a DOS prompt at that File Explorer location, as explained at [[https://stackoverflow.com/questions/378319/windows-explorer-command-prompt-here/379804 | stackoverflow]]:
> Hold Shift while Right-Clicking a blank space in the desired folder to bring up a more verbose context menu. One of the options is //Open Command Window Here//. This works in Windows Vista, 7, 8, and 10.
2. Now that your terminal is at the GS3 installation folder, you can type the following command in the terminal:
ant config-admin
4. It will ask for a new admin password. Type the new password and hit enter.
5. Once you restart the server, try out your new password.
==== Useful information ====
When you've built a collection of documents, you may discover that there appears to be a copy of all these documents in the collection's //import//, //archives// and //index// subfolders and wonder whether Greenstone could really be so inefficient with disk space as to keep 3 copies of everything. As it happens though, Greenstone uses **hard-links** both on Linux and **Windows**, in order to keep just one set of your documents. Then it simply hardlinks to these, instead of making copies.
By default, Windows doesn't show you when files on your filesystem are hard-linked. If you choose to install the Windows extension application [[http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html|Link Shell Extension (LSE)]], it will put red arrows on files that are hard linked.
====Known Issues====
=== IsisGdl does not work yet on Mac Catalina (10.15) and onwards ===
With GS3.10rc1 we have not yet been able to produce an IsisGdl binary that works on Mac Catalina (10.15). Mac appears to have taken away backwards compatibility with 32 bit binaries, such as the IsisGdl 32-bit binary that still worked on Mojave. The 64 bit binary generated on Mojave does not work on either Mojave or Catalina. We are continuing to investigate this issue.
=== Changes to Tomcat port affects Solr collections ===
Currently when you change your tomcat port (either in build.properties, or using File->Settings in the Server program) the changes won't propagate to Solr which will still try to use the old port number.
If you make changes to Tomcat port, please shutdown and restart the server from the Start Menu.
If you are starting from within a terminal, you will need to shut down Greenstone and restart it from a fresh terminal.
=== Web document editor requires perl CGI module ===
//**Note:** For GS2.87 and 3.10rc1 onwards, we hope to have included [[https://trac.greenstone.org/changeset/34151|CGI.pm in a tarball]] at least (untested) for those whose perl installations do not have it and were it can't be easily installed.//
Editing documents through the web interface requires your system perl to have CGI installed. Without CGI, you can edit metadata and save, but your changes won't have been applied. You can tell this is the case by looking at the build log files in the collection, e.g. //greenstone3/web/sites/localsite/collect//log/build_log.1480553047299.txt//. The message might say something like
Document Editor Build
Command = /bin/perl -S /greenstone/pei-jones/web/WEB-INF/cgi/metadata-server.pl
Content-type:text/plain
ERROR: Can't locate CGI.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ./gsdlCGI.pm line 9.
BEGIN failed--compilation aborted at ./gsdlCGI.pm line 9.
Compilation failed in require at (eval 1) line 1.
CGI.pm is not included with versions of [[https://stackoverflow.com/questions/45434990/why-does-this-code-have-issues-with-the-cgi-module|perl from 5.22 and onwards]], which is when this issue can result.
If you get this error, then please install the CGI module for your system perl.
====Work Arounds====
===Using filenames containing &, < and > in GS3.10 collections===
The Enrich panel in GS3.10's GLI is better able to handle
- &, < and >
- and other non-basic ASCII characters.
When you create a collection and assign it metadata in GLI that contains such characters, GLI will now successfully preserve them between GLI sessions.
The second set of characters listed above are still not recommended for use in metadata when dealing with a remote GS3 server, as that has to do with successfully transferring characters from an operating system in one file system encoding on the client side to possibly another operating system using another file system encoding on the GS3 server side.
**Beware:** It is only GLI's Enrich pane that has been fixed up to support both sets of characters in metadata. No other part of Greenstone has been fixed to cope with these characters, should they likewise struggle with them.
The fix is often in the form of adding hex encoded entities into the metadata.xml file and decoding them back for display in GLI's Enrich pane.
===Greenstone applets (Phind, Collage) crash Firefox===
See [[https://bugzilla.redhat.com/show_bug.cgi?id=789959|bugzilla report]].
If attempting to view a java applet (like Collage or Phind phrase classifiers) crashes Firefox, then make sure you have the Java Applet plugin installed. If it is installed and Firefox is still crashing, then open firefox and visit the page
about:config
Scroll down to the property:
dom.ipc.plugins.java.enabled
Set it to true (rightclick and choose //toggle//).
===PDF to image conversion error on Linux===
If you've configured a PDFPlugin to convert PDFs to images, increase the verbosity in Import Options and Build Options to 5 in GLI's Create panel.
When rebuilding the collection, check to see if you encounter the following error message mentioning that 'memory allocation failed', a 'corrupt image' at 'ReadPNGImage' and 'PostScript delegate failed':
import.pl> Converting pdf05-notext.pdf to pagedimg_jpg format
import.pl> calling cmd "/usr/bin/perl" -S gsConvert.pl -verbose 5 -pdf_zoom 2 -errlog "/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/err.log" -output pagedimg_jpg "/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf"
import.pl> Error executing pdfpstoimg.pl
import.pl> pdfpstoimg error log:
import.pl> convert: memory allocation failed `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadOnePNGImage/2160.
import.pl> convert: corrupt image `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadPNGImage/3794.
import.pl> convert: Postscript delegate failed `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/681.
import.pl> convert: no images defined `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext/pdf05-notext.jpg' @ error/convert.c/ConvertImageCommand/3068.
import.pl> Convert error for /research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf
import.pl> Could not convert pdf05-notext.pdf to pagedimg_jpg format
import.pl> convert: memory allocation failed `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadOnePNGImage/2160.
import.pl> convert: corrupt image `/tmp/magick-31829ofGIFuaNZ1xy1' @ error/png.c/ReadPNGImage/3794.
import.pl> convert: Postscript delegate failed `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf': No such file or directory @ error/pdf.c/ReadPDFImage/681.
import.pl> convert: no images defined `/research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext/pdf05-notext.jpg' @ error/convert.c/ConvertImageCommand/3068.
import.pl> Convert error for /research/myfolder/gs3-svn-12Sep2013/web/sites/localsite/collect/Enhanced-PDF/tmp/1378957949/pdf05-notext.pdf
import.pl> Converting pdf05-notext.pdf to html format
If you see the above error message, then:
1. Use a text editor to open your Greenstone 3's gs2build/ext/imagemagick/linux/etc/ImageMagick/delegates.xml
2. Find the line that would say: