Release Name: 3.04
Release Date: 18 Aug 2009
Released: Windows, GNU/Linux, Mac OS X and Source releases of Greenstone 3.04
A long-overdue release of Greenstone3. (The last release was in October 2007). For this release we have been concentrating on the installation process. Uses the same open source installer and release generation system as Greenstone 2. This is the first release using this system and installer. Now that we have one release out using this system, hopefully future releases will happen with more regularity.
Collection building uses Greenstone 2 code, so this release brings that part of the system almost up to date with Greenstone 2.
Main improvements in this release are detailed below after the installation instructions.
Binary release
Note that in some cases, the following doesn't work export TMPDIR=/something/else ./Greenstone-3.04-linux.bin
Use the following instead TMPDIR=/something/else ./Greenstone-3.04-linux.bin
To add source code to a binary release
Installing from a source release
The Greenstone installer downloads are now created using only open source software, including Ant Installer (http://antinstaller.sourceforge.net/), Apache Ant (http://ant.apache.org/), our own search4j (based on launch4j http://launch4j.sourceforge.net/), and our own Greenstone release kits.
We felt it was essential to move away from our existing, closed-source installer suite as it was not in the spirit of Greenstone to rely on proprietary software, and it invited the unnecessary cost of keeping the suite up-to-date.
Using only open source software has also given us the freedom to customise the user experience of the installers. Where a feature was lacking in one of the open source packages, we have enjoyed the ability to "open the hood" and implement it.
With the development of the Greenstone release kits we are now able to generate releases of Greenstone automatically and unattended. This process takes just 20 minutes, and it has helped us a lot in the development and testing of new features of Greenstone. We have been able to set up a feedback loop, where code committed one day is included in an automatic nightly snapshot release made available for download on our website the next day. Our users have also benefited from being able to obtain up-to-date snapshot releases of Greenstone in between our official releases.
These daily snapshots are available through:
http://www.greenstone.org/snapshots
On Windows, the version number is included in the Start Menu which means that multiple installations of Greenstone will have their own entries in the Start Menu. The installer no longer writes to the registry, so installation no longer requires administrator privileges.
ImageMagick is now bundled with Greenstone for binary web releases for all platforms (previously it was only provided on a CD-ROM release) and includes JPEG2000 support. The installer offers the option to install this or not. You can skip installing it if you already have ImageMagick previously installed.
Ghostscript is now bundled with Greenstone for binary web releases for Windows and Mac. The installer offers the option to install this or not. You can skip installing it if you already have Ghostscript previously installed.
The releases are compiled with Java 1.5, and Tomcat 6 is used (previously Tomcat 5). Tomcat 6 is incompatible with Java 1.4, so you will need Java 1.5 or higher to work with this release. However, if you are working from svn, you can still use Java 1.4, and the ant prepare process will install Tomcat 5.
Please see this blog entry. Note this was written by a Greenstone user.
It contains the following instructions:
Requirements
(Note: Although my machine and OS are 64-bit versions, the software above is all 32-bit).)
The installation procedure is divided into the following main steps.
1. Install Sun JDK 2. Install Apache Tomcat 3. Install Greenstone3 4. Configure Greenstone 5. Configure Tomcat 6. Run Greenstone3
1. Sun JDK Installation
* Double click on the jdk-6u17-windows-i586.exe file to install it into the default directory, which is C:\Program Files (x86)\Java.
* Set JAVA_HOME.
o Control Panel > System and Security > System > Advanced System Settings > Environment Variables.
o Under the System Variables, click the “New…” button.
o Enter values for the following two variables: Variable name: JAVA_HOME, and Variable value: C:\Program Files (x86)\Java\Jdk1.6.0_17.
o Click the OK button to complete the setup.
* Add Java to the PATH environment.
o Under the System Variables, click to highlight the “Path” variable.
o Click the “Edit…” button.
o Add %JAVA_HOME%\bin to the end of Path’s variable value.
o Click the OK, OK and OK buttons to complete the setup.
</code
2. Apache Tomcat Installation
<code>
* Unzip the “apache-tomcat-6.0.26-windows-x86.zip” file into any directory
* Copy the unzipped directory and paste it into C:\, resulting in C:\apache-tomcat-6.0.26 directory.
* Set up Apache Tomcat home path
o Go to the Environment Variables, using the same approach as used in setting JAVA_HOME above.
o Click the “New…” button.
o Enter values for the following two variables: Variable name: CATALINA_HOME, and Variable value: C:\apache-tomcat-6.0.26.
o Click the OK, OK and OK buttons to complete the setup.
* Test the Apache Tomcat installation
o Open a web browser
o Type: http://localhost:8080/ in the address bar.
o The Apache Tomcat startup page should be displayed.
3. Greenstone3 Installation
* Double click on the greenstone-3.04-win32.exe file.
* Install it into C:\Users\YourUserName directory where YourUserName is the user name used in your machine such as John.
* During the installation, install everything but exclude Apache-Tomcat (by unchecking the textbox) that comes with Greenstone3.
* The installed directory would be C:\Users\YourUserName\Greenstone3.
<code>
4. Greenstone3 Configuration
<code>
* Edit the greenstone3.xml in the C:\Users\YourUserName\Greenstone3\resources\web\tomcat by replacing @gsdl3webhome@ with C:\Users\YourUserName\Greenstone3\web for the docBase= variable.
* Copy and paste the edited greenstone3.xml file into C:\apache-tomecat-6.0.26\conf\Catalina\localhost.
* Edit build.properties file in the C:\Users\YourUserName\Greenstone3 directory by adding "C:\apache-tomcat-6.0.26" to the tomcat.installed.path variable in the file
(This step is required for running Greenstone Librarian Interface (GLI)).
5. Apache Tomcat Configuration
* Copy and paste all the .dll files in the C:\Users\YourUserName\Greenstone3\lib\jni directory into C:\apache-tomcat-6.0.26\bin directory.
* Copy and paste all the .jar files in the C:\Users\YourUserName\Greenstone3\lib\jni directory into C:\apache-tomcat-6.0.26\lib directory.
(Note: I know that there is a better way to set up these .dll and .jar files for Tomcat server, but I still do not know how to do it yet.)
6. Running Greenstone3
* Start the Apache Tomcat server
o Open a command prompt.
o Issue a command to change directory to the C:\apache-tomcat-6.0.26\bin
o Issue the following command: startup.bat
* Open the Greenstone3 web server
o Open a web browser
o Type: http://localhost:8080/greenstone3/ in the address bar.
o The Greenstone 3 welcome page should shown up. On this page, there are four hypertext links.
+ Run the test servlet.
+ Run the default library servlet.
+ Run the ‘standard’ servlet.
+ Run the gateway servlet.
o The top three links should be working well. The bottom link requires Apache Axis to be installed. (Note: I have not tried it.)
* Run the Greenstone Librarian Interface (GLI)
o Open a command prompt.
o Issue a command to change directory to C:\Users\YourUserName\gli directory.
o Issue the following command: gli.bat. The GLI should be activated successfully.
Improved interface customisation (in development). Customising greenstone3 interfaces is set to become a lot easier, with simpler and easier stylesheets, and support for <gslib> elements in stylesheets. Use the 'dev' servlet to test this new functionality in greenstone3.04. (http://localhost:8080/greenstone3/dev or equivalent URL)
Notes about old style interfaces:
Classic interface was renamed to gs2. library servlet now uses gs2 interface, while gs3library servlet uses default interface.
On the collection building side, Greenstone3 incorporates GLI and collection building code from Greenstone 2. Almost all the changes incorporated into the latest greenstone releases will be available in this release.
Here is a summary of changes since the 3.03 release (which was October 2007, and roughly coincided with the 2.80 release). These notes have been copied from the relevant Greenstone 2 release notes: 2.82 (June 2009) 2.82rc (April 2009) 2.81 (Dec 2008)
Note that only collection building and GLI changes in the Greenstone 2 release notes will be applicable to the Greenstone 3 release.
Collection building now supports incremental addition, deletion and modification of documents and metadata. Incremental import is available for all indexers, while incremental build is only available using Lucene.
The new functionality is controlled by the new incremental-import.pl/incremental-buildcol.pl scripts, using the -incremental flag to import/buildcol.pl, or the "Minimal Rebuild" checkbox in GLI's Create Panel.
During import, file timestamps are used to detect modified files, and a global file scan is used to detect new or deleted documents. The easiest way to use this feature is to use the new incremental-import.pl script. The first time the collection is built, incremental-import.pl notices this and runs a full import.pl, otherwise it will run it with the -incremental flag.
Incremental indexing is only available with the Lucene indexer. MG/MGPP collections will exit with an error if you try to use the -incremental flag when running buildcol.pl. If you want to incrementally update your index directory, you need to use the -builddir option.
The new incremental-buildcol.pl script will call buildcol.pl with the appropriate options set.
The two stages can be run with incremental-rebuild.pl (which just calls incremental-import.pl followed by incremental-buildcol.pl).
MGPP and Lucene indexing tidied up. 'allfields' now means combined searching over all specified indexes (not all document metadata) for both mgpp and lucene. 'metadata' will index all metadata, but no longer reindexes metadata that has already been specified. MGPP indexing over combined fields now works properly.
Greenstone now comes with it's own modified version of gdbm which can auto-detect the format of database files (ldb or bdb format) and load either format on any of Greenstone's supported platforms. This achieves a better level of portability for Greenstone collections, and allows collections built on one operating system to be served from other operating systems supported by Greenstone.
The plugins have been restructured. This has been done mostly for coding efficiency. The most noticeable change is that plugins have been renamed. xxxPlug now becomes xxxPlugin, and some have also had their names modified or expanded to be clearer. For example, DBPlug is now DatabasePlugin, PPTPlug is PowerPointPlugin. The complete list of new plugins can be seen at the plugins page. Old collections should still build okay in GLI, which will map the old plugin names to the new ones. Command line building should also work, but the configuration file won't be altered to use the new names.
A lot of work has been done trying to get Greenstone to work properly with non-ascii filenames. Two versions of the filename are stored, [Source] for displaying the filename, and [SourceFile] for linking to the file from a web page.
The -smart_block option (most commonly seen with HTMLPlugin) has been deprecated . Instead we have a first pass through the documents to find out which ones should be blocked.
MARCPlugin and MARCXMLPlugin now import as Dublin Core or Qualified Dublin Core, rather than extracted metadata. The marc2(q)dc.txt mapping files have been changed to handle this. Rebuilds of old MARC collections will need index/classifier/format statement upgrades to deal with the new metadata.
The marc2dc.txt mapping file has been updated to more closely match the LOC crosswalk.
OAIPlugin now has a -metadata_set option. If set to auto, the metadata will keep whatever namepsace it had in the file (eg dc). If set to something else, all elements will be mapped to that namespace. A special case exists for "dc": qualified dc elements are represented slightly differently in GLI: spatial becomes dc.Coverage^spatial. A mapping is provided to convert to the GLI version when dc is used. Since metadata is no longer saved in the extracted metadata set, it won't show up in GLI unless the files are exploded. Exploding is now available for OAI records.
MetadataEXIFPlugin (experimental): A new plugin is available to extract EXIF metadata from image, audio and video files. It uses the EXIF perl module from CPAN.This plugin works alongside rather that instead of the standard image and video plugins, so these still need to be included in the collection. MARCXMLPlugin now assigns metadata from metadata.xml files.
EMAILPlugin now saves binary attachments as binary files on Windows
ImagePlugin and PagedImagePlugin now offer the -cache_generated_images option to prevent thumbnails and screenview images being recreated each import.
ProCitePlugin no longer uses the pc namespace. Metadata is stored using the field names set in the workform definition, rather than using Field1Name and Field1Value etc..
-reversesort option to import.pl to be used with -sortmeta option to sort in reverse order.
Tidied up OID generation. OIDtype and OIDmetadata options are now available for each plugin as well as globally from import.pl. Hashing usually occurs on the original file, but some plugins specify that hashing should be done on the Greenstone XML document.
The List classifier has been renamed to SimpleList.
The GenericList classifier has been renamed to List, and is now the default classifier in GLI. We encourage use of List rather than AZList/AZCompactList as it handles non-ascii text.
A new right-click file option for binary files that go through a conversion process to be imported, such as PDF, Word and PowerPoint files. "Replace source document with HTML" will convert the original file to HTML and replace it in the collection. This means that the converted HTML (which may be unattractive) can now be modified in the collection. This is available for Remote GLI too.
A new right-click file option on the collection tree background to refresh the file view. This is useful if you have manually added files to the collection outside of GLI.
MetadataXMLPlugin has been moved 'below the line' in the Plugins panel so cannot be removed in GLI. If you are using GLI to add metadata, then you need this plugin.
Lots of work done on making the Download panel work better, including getting download processes to terminate when they are cancelled or when GLI is closed while they are still running.
CJK segmentation option has been added to GLI. This doesn't do proper word segmentation, but adds a space in between each CJK character. The character ranges have been expanded to work with Japanese and Korean. It is applied to metadata as well as document text. NOTE: Greenstone 3 currently doesn't do anything at runtime to support this.
Collection Grouping - supported in GLI, but not yet by Greenstone 3 runtime.
Exploding a database file: the options you have set for the exploding plugin in the Design pane will now be used when exploding.
Library Systems Specialist mode has been removed. GLI now has three modes:
GLI now displays Right-to-Left for those languages that read right-to-left. Thanks to Amin Hedjazi for the code.
Rename and Replace options (for documents in the collection tree) now available for the GLI client building collections on a remote Greenstone server.
Downloading is possible now from within the GLI client, if you have a local Greenstone installation and have set up the greenstone environment by running the setup script (setup.bat if on Windows and source setup.bash if on Linux).
if (($collection = &colcfg::use_collection("", $collection, "")) eq "") {
Change this to
if (($collection = &colcfg::use_collection("localsite", $collection, "")) eq "") {
(If you are not using localsite, then please add the site name you are using.)
Please see the following pages for what is still to be worked on: