Source Installation on Linux - Greenstone3

This walkthrough is for Linux. But it may have the side-effect of helping Windows users too.

• Install SVN
• SVN
• Compiling Greenstone3
• Greenstone3-Home

• The correct folder for installing programs on Linux

The first step for University of Waikato students installing Greenstone 3 from source:

You need to install it on the research partition, ELSE IT DOESN'T WORK! Go to /Scratch and create folder your-username, if this folder isn't already there. Then work from there.

$ /Scratch/your-username

• Download apr, apr-util, and subversion as specified on the following page. Also check that the version of neon is as given on the page install SVN

• Install apr and apr-util as outlined at install SVN.

For students or developers at the University of Waikato: make sure these are installed in the RESEARCH partition, else it doesn't work! I'm assuming below that these will be installed in /Scratch/your-username/sources (sources is just a folder where I put the unzipped source code for Greenstone 3 related programs). For everyone else, replace this with your FULL PATH to where the apr, apr-util and subversion will be installed.

• Installing Subversion (SVN)

To install the versioning control system "subversion" (SVN)–the example of SVN version 1.4.4 is covered in the steps below and –type in the Xterm:

$ cd /path/to/folder/containing/subversion-1.4.4.tar.gz
$ tar -xzf /path/to/subversion-1.4.4.tar.gz
$ cd subversion-1.4.4

The following is separated into multi-lines, but type them all on the same line in the x-term separated by spaces. Make sure you use the paths to subversion, apr and apr-util that are appropriate for your situation:

$ ./configure    --prefix=/Scratch/your-username/subversion
--without-apxs
--with-apr=/Scratch/your-username/apr/bin/apr-1-config
--with-apr-util=/Scratch/your-username/apu/bin/apu-1-config
--with-neon=/usr/

Then run the following two commands in the x-term:

make
make install

Now it's created a folder called 'subversion' in your-username, because that's what –prefix told it to do above.

Next, add the path to the bin directory to the paths already exported:

$ export PATH=/Scratch/your-username/subversion/bin:$PATH

For more details, see Install SVN

• Apache Ant

The latest version of Greenstone 3 checked out from SVN requires Ant 1.7.1. Therefore, get and install Apache Ant, at least version 1.7.1.

• Create the ANT_HOME environment variable, and set its value to be the full path to the root directory of your ant installation.

• Then adjust the existing PATH variable to include the bin directory of ANT_HOME.

E.g. if, after unzipping the ant binary (which you can download from the Apache Ant site), you ended up with a folder called apache-ant-1.7.1, then you would add the following into your .profile or .bash_profile file to make it permanent, or just type it into the terminal to make these changes active for the current terminal's session:

export ANT_HOME=/full/path/to/apache-ant-1.7.1
export PATH=$ANT_HOME/bin:$PATH

* Make sure $JAVA_HOME and $ANT_HOME are set, and that $JAVA_HOME/bin, $ANT_HOME/bin and your svn/bin are on the PATH. * If you're on Linux 64 bit,

export CFLAGS=-fPIC $CFLAGS

• Checking out Greenstone 3 source code from subversion (SVN)

This is a slight variation on what's given in SVN

Go to /Scratch/your-username (in the following, Greenstone 3 will be checked out into a folder called greenstone 3).

Type:

$ svn co http://svn.greenstone.org/main/trunk/greenstone3 greenstone3

[co stands for checkout, which will checkout code from SVN]

Whenever you need to update with the latest source code from the SVN repository, cd to the $GSDL3HOME directory (your Greenstone installation directory, for example /Scratch/your-username/greenstone3) and type:

$ svn update

• Checking out, compiling and running the Greenstone Librarian Interface (GLI) for Greenstone 3

Go to $GSDL3HOME, your greenstone3 home folder:

$ cd greenstone3 
$ svn co http://svn.greenstone.org/main/trunk/gli gli

Then go to gli folder:

$ cd gli
$ svn update

$ ./makegli.sh 
$ ./makejar.sh 

To run GLI for Greenstone 3:

$ ./gli.sh

• Editing your .profile: JAVA_HOME and the PATH variable

Update your .profile file, containing your settings, to export JAVA_HOME and update the PATH to point to your Java installation. Go to /Scratch/your-username, then either use emacs (or some other text editor) to make the changes to the .profile file by launching it as:

$ emacs ~/.profile

Find out where your Java installation is:

$ whereis java

Export the JAVA_HOME variable such that it points to the installation of the JDK you are using. And append to the PATH the location of the JDK's bin directory and the compiler, javac. To do so, in the .profile file, export (or try setting and then exporting) the JAVA_HOME variable, and append the location of your JDK to the PATH variable.

For example, if you're working with Greenstone 3 at the University of Waikato, your .profile file should now additionally have the following:

	# commenting out JAVA_HOME, as we'll be exporting it below
	#JAVA_HOME=

	PATH=/opt/j2sdk1.4.2_13/bin:/opt/j2sdk1.4.2_13/bin/javac:$PATH
	export JAVA_HOME=/opt/j2sdk1.4.2_13/

It is important that it is the direct path–and not an symbolic link or shortcut–to the Java installation.

Save the file. Note that if you are using Java 5 (JDK 1.5) or higher, see Using Java 5 (JDK 1.5) or higher with Greenstone 3.

Now source it to make the changes active (alternatively, you can log out and log back in):

$ source ~/.profile

• Compiling Greenstone 3's source code and installing it

NOTES:

• Before moving on, you should make sure that your SVN client's version is compatible with the SvnAnt extension (which works with Ant) that comes with Greenstone 3. For instance, if you run "svn –version" in the terminal and your version is SVN 1.6.x, then at present this is still incompatible. To still be able to get your Greenstone 3 to work, you would need to grab a version of SvnAnt appropriate to your SVN client. In the case of SVN 1.6.x, for instance, this would be SvnAnt 1.3.x. Please refer to Subcli===== Greenstone2 ===== pse's page on SvnAnt which details the versions of SvnAnt suitable for the various versions of SVN client. If you have had to download another version of SvnAnt, then you would unzip your SvnAnt download and copy all the jar files in it (some 5 jar files) into your Greenstone 3 installations's lib/java folder, replacing the older versions of the same file.

• You also need to make sure that if you are behind a proxy, you set the proxy in the build.properties file in your Greenstone installation folder. At the start, there will be no such file. To generate it, you need to type the following in your terminal (fro===== Greenstone2 =====

m the Greenstone installation folder):

ant

Now open the build.properties file and adjust the proxy settings (proxy host, port, username and password). You may want to take the opportunity to change the tomcat server details for your Greenstone server as well (host, port, shutdown port).

Now you are ready to compile up Greenstone:

As advised by Quan, qq6 (another explanation is given at Compiling Greenstone3).

Go to $GSDLHOME, Greenstone3 home directory (which in the example above is /Scratch/your-username/greenstone3):

$ ant prepare

Press 'y' to accept port 8080 if nothing has been deployed there already, which should be the case. Else see further below on changing port number Changing the port number where tomcat is installed.

$ ant install

Page Greenstone3-Home gives more information, stating:

Notes

• Read the README for installation notes.
• Installing from SVN:
  • You will need to have Apache Ant and a Subversion client installed to compile and install Greenstone3. Please read the README.txt file in the greenstone3 directory for more information.
  • Run:

svn co http://svn.greenstone.org/greenstone3/trunk greenstone3 
cd greenstone3 
(edit build.properties as appropriate) 
ant prepare install

 
   * Debian Sarge system: You need to have libgdbm-dev installed, and create a symlink from libgdbm-compat.so.3 → libgdbm.so.2.

• Setting up the Greenstone3 environment

$ source gs3-setup.sh

During Greenstone 3's installation process, it asks whether you accept the default port 8080. If you don't (or decide afterwards to make your web server work on a different port), then you can do this as follows.

• Change the port-number in the build.properties file:

e.g. set tomcat port number to 8081 and its stop port to 8006. Or you can set tomcat port number to 9090 and its stop port to 9005.

• After that, in an x-term type:

$ ant -projecthelp

This will tell you that when you change the port number, you need to run ant configure. So:

$ ant configure

Sometimes this step does not change the port after all! In such a case, the actual place to change it would be the following file. First stop tomcat (as explained somewhere above). Open the Greenstone 3 server.xml file:

$ GSDL_HOME/packages/tomcat/conf/server.xml

And make the necessary changes. For example:

• Change tomcat startup port num from 8080 to 9090, and
• Change the shutdown port num from 8005 to 9005

Then restart tomcat.

Make sure tomcat has stopped.

In your greenstone3 home folder, $GSDLHOME:

$ ant stop

Then, deploy (as written in the Greenstone 3 manual, p. 5):

$ ant deploy-localsite

Check that it works, by starting tomcat again and going to:

http://localhost:8080/greenstone3/services/

Make sure that the page appears. You can also look at the wsdl file for the gs3 web service that's been exposed:

http://localhost:8080/greenstone3/services/localsite?wsdl

You can also deploy other sites, see the Greenstone 3 Developer's Manual.

You would want to do this if you ever choose to execute Greenstone's Perl scripts (like import.pl and export.pl) from the command line instead of via GLI.

Go to the gs2build folder of your Greenstone 3 home folder and source setup.bash:

$ cd $GSDLHOME/gs2build
$ source setup.bash

This will set up the capabilities for export.pl and import.pl (it will have set the necessary environment variables for these two perl programs to work).

The following steps apply if

• your Greenstone 3 installation is compiled with Java 5;
• you are using Java 5 to write a Java program which uses (calls) Greenstone 3's code;
• in your browser, the servlet throws some root exception about 'xalan'.

In order for GS3 to work with Java 5 (Java 1.5) # go to $GSDLHOME/web/WEB-INF/lib/ and # rename the file xalan.jar.tmp to xalan.jar # don't need to recompile after renaming xalan.jar.tmp to xalan.jar, but just do an

$ ant restart

(or: an $ ant stop followed by $ant start)

Of course, your .profile file should have JAVA_HOME pointing to JDK1.5, while its bin folder and javac should be on your PATH too, as explained earlier in this document.

If your Greenstone 3 source code was compiled with Java 5 and you didn't make these small changes, then running the Greenstone test servlet will still work, but running any of the standard library servlet, the classic servlet or the gateway servlet won't work. In those last three cases, it will throw an exception where the Root exception will mention something to do with xalan.

If you're on Linux and in the BUILD FAILED message, you see it mention an error related to GDBM, then go to your greenstone 3 installation folder's src/packages/gdbm<something> folder and compile up the GDBM manually.

• ./configure
• make

• make install

Now try installing Greenstone 3 again.

You could also try to modify your .profile file as described in step 3 of Checking out and installing Greenstone 3 and GLI: Editing your .profile: JAVA_HOME and the PATH variable. (Make sure it has the actual path to the JDK instead of giving it the path to a shortcut/symbolic link to the JDK.) This has sometimes helped solve build failure problems as well.

If BUILD FAILED errors still occur during Greenstone3 installation with ant, then you could try checking out greenstone3 from SVN again and compiling it once more. This solution has often worked for me.

• Make sure tomcat is not running (from the $GSDLHOME folder):

$ ant stop

This will throw an exception if tomcat wasn't running, but that's ok. You may also want to make sure *all* Greenstone3 tomcat instances–in case multiple instances had been launched–are stopped as explained in the section Problems using ant start to run Tomcat.

• Delete the existing greenstone3 folder, by going to one directory higher than $GSDLHOME and typing:

$ rm -rf greenstone3

• And then check out greenstone3 again from the SVN code repository:

$ svn checkout http://svn.greenstone.org/greenstone3/trunk greenstone3

• Go into folder greenstone3 ($GSDLHOME) and compile it:

$ cd greenstone3
$ ant prepare (press 'y' to accept port 8080); 
$ ant install

• Once it has all been installed successfully, run Tomcat with

$ ant start

• Check everything works by pointing your browser to:

http://localhost:<your-port>/greenstone3

If executing $ant start doesn't complete with a BUILD SUCCESSFUL message, then it might have to do with there being multiple instances of GS3's tomcat running. This might happen if you forgot to stop the running GS3's tomcat and then started another one. Often this causes problems. Already having an instance of GS3 tomcat server running could cause the $ant start command to hang or the build to Fail.

Solution: kill all other instances of GS3 tomcats that are running. (If you have tomcat instances running for something other than Greenstone3, make sure you don't kill those unless you're alright with that.)

How:

• In an xterm, go into the $GSDLHOME folder and find all the running instances of Tomcat by typing:

ps aux|grep tomcat

This gives you stats on how many instances of tomcat are running.

For example, you may see something like

<username>     12556  0.0  0.0   3896   668 pts/1    R+   15:26   0:00 grep tomcat
<username>     28212  0.0  6.2 593840 64128 ?        Sl   Oct04   2:08 /opt/j2sdk1.4.2_13//bin/java -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager

• We want to terminate the process-id(s) marked with "Sl". Use the command "kill -9 <process-id>". In the example above, we would type the following in the x-term:

$ kill -9 28212

• Again, do:

$ ps aux|grep tomcat 

to check that there are no more running instances of tomcat ("Sl"). If there are any still left, use kill -9 <process-id> to kill that tomcat process.

• If there are no previous instances of tomcat to kill anymore, then try an ant start now:

$ ant start

Hopefully it will work and come back with BUILD SUCCESSFUL.

See Using Java 5 (JDK 1.5) or higher with Greenstone 3