Greenstone Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Source Installation on Linux ======


<TABAREA tabs="Greenstone3,Greenstone2">
<TAB>
===== Greenstone3 =====
This walkthrough is for **Linux**. But it may have the side-effect of helping Windows users too.

=====Useful links and further information=====
  * [[en:developer:install_svn|Install SVN]]
  * [[en:developer:svn|SVN]]
  * [[en:developer:compiling_greenstone|Compiling Greenstone]]
  * [[http://www.greenstone.org/greenstone3-home|Greenstone3-Home]]


===== Preliminaries: Downloading the necessary utilities (apr, apr-util, SVN, Ant) =====

  * **The correct folder for installing programs on Linux**


The first step for University of Waikato students installing Greenstone 3 from source:


You need to install it on the research partition, ELSE IT DOESN'T WORK! Go to /Scratch and create folder your-username, if this folder isn't already there. Then work from there.
<code>$ /Scratch/your-username</code>


  * **Download apr, apr-util, and subversion** as specified on the following page. Also check that the version of neon is as given on the page [[en:developer:install_svn|install SVN]]

  * **Install apr and apr-util** as outlined at [[en:developer:install_svn|install SVN]].


For students or developers at the University of Waikato: make sure these are installed in the RESEARCH partition, else it doesn't work! I'm assuming below that these will be installed in /Scratch/your-username/sources
(**sources** is just a folder where I put the unzipped source code for Greenstone 3 related programs).
For everyone else, replace this with your FULL PATH to where the apr, apr-util and subversion will be installed.


  * **Installing Subversion (SVN)**


To install the versioning control system "subversion" (SVN)--the example of SVN version 1.4.4 is covered in the steps below and --type in the Xterm:
<code>
$ cd /path/to/folder/containing/subversion-1.4.4.tar.gz
$ tar -xzf /path/to/subversion-1.4.4.tar.gz
$ cd subversion-1.4.4
</code>
The following is separated into multi-lines, but type them all on the same line in the x-term separated by spaces. Make sure you use the paths to subversion, apr and apr-util that are appropriate for your situation:
<code>
$ ./configure    --prefix=/Scratch/your-username/subversion
--without-apxs
--with-apr=/Scratch/your-username/apr/bin/apr-1-config
--with-apr-util=/Scratch/your-username/apu/bin/apu-1-config
--with-neon=/usr/
</code>
Then run the following two commands in the x-term:
<code>make
make install</code>
Now it's created a folder called 'subversion' in your-username, because that's what --prefix told it to do above. 


Next, add the path to the bin directory to the paths already exported:
<code>$ export PATH=/Scratch/your-username/subversion/bin:$PATH</code>
For more details, see [[en:developer:install_svn|Install SVN]]


  * **Apache Ant**


The latest version of Greenstone 3 checked out from SVN requires Ant 1.7.1. Therefore, get and install [[http://ant.apache.org/|Apache Ant]], at least version 1.7.1.

  * Create the ANT_HOME environment variable, and set its value to be the full path to the root directory of your ant installation.

  * Then adjust the existing PATH variable to include the bin directory of ANT_HOME.




E.g. if, after unzipping the ant binary (which you can download from the [[http://ant.apache.org/|Apache Ant]] site), you ended up with a folder called apache-ant-1.7.1, then you would add the following into your .profile or .bash_profile file to make it permanent, or just type it into the terminal to make these changes active for the current terminal's session:
<code>export ANT_HOME=/full/path/to/apache-ant-1.7.1
export PATH=$ANT_HOME/bin:$PATH</code>

===== Checking out and installing Greenstone 3 and GLI =====

* Make sure $JAVA_HOME and $ANT_HOME are set, and that $JAVA_HOME/bin, $ANT_HOME/bin and your svn/bin are on the PATH.
* **If you're on Linux 64 bit,**
<code>export CFLAGS=-fPIC $CFLAGS</code>

  * **Checking out Greenstone 3 source code from subversion (SVN)**


This is a slight variation on what's given in [[en:developer:svn|SVN]]

 
Go to /Scratch/your-username (in the following, Greenstone 3 will be checked out into a folder called greenstone 3).


Type:
<code>
$ svn co http://svn.greenstone.org/main/trunk/greenstone3 greenstone3
</code>
[co stands for checkout, which will checkout code from SVN]

Whenever you need to update with the latest source code from the SVN repository, cd to the $GSDL3HOME directory (your Greenstone installation directory, for example /Scratch/your-username/greenstone3) and type:
<code>$ svn update</code>


  * **Checking out, compiling and running the Greenstone Librarian Interface (GLI) for Greenstone 3**


Go to $GSDL3HOME, your greenstone3 home folder:
<code>
$ cd greenstone3 
$ svn co http://svn.greenstone.org/main/trunk/gli gli
</code>
Then go to gli folder:
<code>
$ cd gli
$ svn update
</code>
<code>
$ ./makegli.sh 
$ ./makejar.sh 
</code>

To run GLI for Greenstone 3:
<code>$ ./gli.sh</code>


  * **Editing your .profile: JAVA_HOME and the PATH variable**


Update your .profile file, containing your settings, to export JAVA_HOME and update the PATH to point to your Java installation. Go to /Scratch/your-username, then either use emacs (or some other text editor) to make the changes to the .profile file by launching it as:
<code>$ emacs ~/.profile</code>
Find out where your Java installation is:
<code>$ whereis java</code>
Export the JAVA_HOME variable such that it points to the installation of the JDK you are using. And //append// to the PATH the location of the JDK's bin directory and the compiler, javac. To do so, in the .profile file, export (or try setting and then exporting) the JAVA_HOME variable, and append the location of your JDK to the PATH variable.

For example, if you're working with Greenstone 3 at the University of Waikato, your .profile file should now additionally have the following:
<code>	# commenting out JAVA_HOME, as we'll be exporting it below
	#JAVA_HOME=

	PATH=/opt/j2sdk1.4.2_13/bin:/opt/j2sdk1.4.2_13/bin/javac:$PATH
	export JAVA_HOME=/opt/j2sdk1.4.2_13/
</code>
**It is important that it is the direct path--and not an symbolic link or shortcut--to the Java installation.**


Save the file. Note that if you are using Java 5 (JDK 1.5) or higher, see [[#Using Java 5 (JDK 1.5) or higher with Greenstone 3]]. 


Now source it to make the changes active (alternatively, you can log out and log back in):
<code>
$ source ~/.profile
</code>


  * **Compiling Greenstone 3's source code and installing it**


//NOTES://
  * Before moving on, you should make sure that your SVN client's version is compatible with the SvnAnt extension (which works with Ant) that comes with Greenstone 3. For instance, if you run "svn --version" in the terminal and your version is SVN 1.6.x, then at present this is still incompatible. To still be able to get your Greenstone 3 to work, you would need to grab a version of SvnAnt appropriate to your SVN client. In the case of SVN 1.6.x, for instance, this would be SvnAnt 1.3.x. Please refer to [[http://subclipse.tigris.org/svnant.html|Subclipse's page on SvnAnt]] which details the versions of SvnAnt suitable for the various versions of SVN client. If you have had to download another version of SvnAnt, then you would unzip your SvnAnt download and copy all the jar files in it (some 5 jar files) into your Greenstone 3 installations's lib/java folder, replacing the older versions of the same file.

  * You also need to make sure that if you are behind a proxy, you set the proxy in the build.properties file in your Greenstone installation folder. At the start, there will be no such file. To generate it, you need to type the following in your terminal (from the Greenstone installation folder):
<code>ant</code>
Now open the build.properties file and adjust the proxy settings (proxy host, port, username and password). You may want to take the opportunity to change the tomcat server details for your Greenstone server as well (host, port, shutdown port).

Now you are ready to compile up Greenstone:
 
As advised by Quan, qq6 (another explanation is given at [[en:developer:compiling_greenstone|Compiling Greenstone]]).


Go to $GSDLHOME, Greenstone3 home directory (which in the example above is /Scratch/your-username/greenstone3):
<code>$ ant prepare</code>
Press 'y' to accept port 8080 if nothing has been deployed there already, which should be the case. Else see further below on changing port number [[#Changing the port number where tomcat is installed]].
<code>$ ant install</code>

Page [[http://www.greenstone.org/greenstone3-home|Greenstone3-Home]] gives more information, stating:

**Notes**

     * Read the README for installation notes.  
     * Installing from SVN:
          * You will need to have Apache Ant and a Subversion client installed to compile and install Greenstone3. Please read the README.txt file in the greenstone3 directory for more information.
          * Run:
<code>svn co http://svn.greenstone.org/greenstone3/trunk greenstone3 
cd greenstone3 
(edit build.properties as appropriate) 
ant prepare install
</code>
   
     * Debian Sarge system: You need to have libgdbm-dev installed, and create a symlink from libgdbm-compat.so.3 → libgdbm.so.2.


  * **Setting up the Greenstone3 environment**
<code>$ source gs3-setup.sh</code>


===== Advanced =====
==== Changing the port number where tomcat is installed ==== 
During Greenstone 3's installation process, it asks whether you accept the default port 8080. If you don't (or decide afterwards to make your web server work on a different port), then you can do this as follows.

  * Change the port-number in the build.properties file: 


e.g. set tomcat port number to 8081 and its stop port to 8006.
Or you can set tomcat port number to 9090 and its stop port to 9005.

  * After that, in an x-term type:
<code>
$ ant -projecthelp
</code>
This will tell you that when you change the port number, you need to run ant configure. So:
<code>
$ ant configure
</code>


**Sometimes this step does not change the port after all!** In such a case, the actual place to change it would be the following file. First **stop tomcat** (as explained somewhere above). Open the Greenstone 3 server.xml file:
<code>$ GSDL_HOME/packages/tomcat/conf/server.xml</code>
And make the necessary changes. For example:
  * Change tomcat startup port num from 8080 to 9090, and
  * Change the shutdown port num from 8005 to 9005
Then **restart tomcat**.

=== Deploying localsite === 
Make sure tomcat has **stopped**. 


In your greenstone3 home folder, $GSDLHOME:
<code>$ ant stop</code>
Then, deploy (as written in the [[http://files.greenstone.org/technical/gs3manual.pdf|Greenstone 3 manual]], p. 5):
<code>$ ant deploy-localsite</code>
Check that it works, by **starting** tomcat again and going to:
<code>http://localhost:8080/greenstone3/services/</code>
Make sure that the page appears. You can also look at the wsdl file for the gs3 web service that's been exposed: 
<code>http://localhost:8080/greenstone3/services/localsite?wsdl</code>
You can also deploy other sites, see the [[http://files.greenstone.org/technical/gs3manual.pdf|Greenstone 3 Developer's Manual]].



==== Getting Greenstone Perl scripts working from the command line==== 
You would want to do this if you ever choose to execute Greenstone's Perl scripts (like import.pl and export.pl) from the command line instead of via GLI.


Go to the gs2build folder of your  Greenstone 3 home folder and source setup.bash:
<code>$ cd $GSDLHOME/gs2build
$ source setup.bash
</code>
This will set up the capabilities for export.pl and import.pl (it will have set the necessary environment variables for these two perl programs to work).


==== Using Java 5 (JDK 1.5) or higher with Greenstone 3 ==== 

The following steps apply if 
  * your Greenstone 3 installation is compiled with Java 5;
  * you are using Java 5 to write a Java program which uses (calls) Greenstone 3's code;
  * in your browser, the servlet throws some root exception about 'xalan'.

In order for GS3 to work with Java 5 (Java 1.5) 
# go to $GSDLHOME/web/WEB-INF/lib/ and
# rename the file xalan.jar.tmp to xalan.jar
# don't need to recompile after renaming xalan.jar.tmp to xalan.jar, but just do an 
<code>$ ant restart</code>
(or: an $ ant stop followed by $ant start)


Of course, your .profile file should have JAVA_HOME pointing to JDK1.5, while its bin folder and javac should be on your PATH too, as explained earlier in this document.

If your Greenstone 3 source code was compiled with Java 5 and you didn't make these small changes, then running the Greenstone //test servlet// will still work, but running any of the //standard library servlet//, the //classic servlet// or the //gateway servlet// won't work. In those last three cases, it will throw an exception where the Root exception will mention something to do with xalan.


==Troubleshooting== 
====Greenstone 3 installation problems==== 
If you're on Linux and in the BUILD FAILED message, you see it mention an error related to GDBM, then go to your greenstone 3 installation folder's src/packages/gdbm<something> folder and compile up the GDBM manually.

  * ./configure
  * make

  * make install


Now try installing Greenstone 3 again.


You could also try to modify your .profile file as described in step 3 of [[#Checking out and installing Greenstone 3 and GLI]]: //Editing your .profile: JAVA_HOME and the PATH variable//. (Make sure it has the actual path to the JDK instead of giving it the path to a shortcut/symbolic link to the JDK.) This has sometimes helped solve build failure problems as well.


If BUILD FAILED errors still occur during Greenstone3 installation with ant, then you could try checking out greenstone3 from SVN again and compiling it once more. This solution has often worked for me.

  * Make sure tomcat is not running (from the $GSDLHOME folder):
<code>$ ant stop</code>
This will throw an exception if tomcat wasn't running, but that's ok. You may also want to make sure *all* Greenstone3 tomcat instances--in case multiple instances had been launched--are stopped as explained in the section [[#Problems using ant start to run Tomcat]].

  * Delete the existing greenstone3 folder, by going to one directory higher than $GSDLHOME and typing:
<code>$ rm -rf greenstone3</code>

  * And then check out greenstone3 again from the SVN code repository:
<code>$ svn checkout http://svn.greenstone.org/greenstone3/trunk greenstone3</code>

  * Go into folder greenstone3 ($GSDLHOME) and compile it:
<code>$ cd greenstone3
$ ant prepare (press 'y' to accept port 8080); 
$ ant install
</code>

  * Once it has all been installed successfully, run Tomcat with
<code>$ ant start</code>

  * Check everything works by pointing your browser to:
<code>http://localhost:<your-port>/greenstone3</code>



====Problems using ant start to run Tomcat==== 
If executing **$ant start** doesn't complete with a BUILD SUCCESSFUL message, then it might have to do with there being multiple instances of GS3's tomcat running. This might happen if you forgot to stop the running GS3's tomcat and then started another one. Often this causes problems. Already having an instance of GS3 tomcat server running could cause the **$ant start** command to hang or the build to Fail.

**Solution:** kill all other instances of GS3 tomcats that are running. (If you have tomcat instances running for something other than Greenstone3, make sure you don't kill those unless you're alright with that.)


**How:**

  * In an xterm, go into the $GSDLHOME folder and find all the running instances of Tomcat by typing:
<code>ps aux|grep tomcat</code>
This gives you stats on how many instances of tomcat are running.


For example, you may see something like
<code>
<username>     12556  0.0  0.0   3896   668 pts/1    R+   15:26   0:00 grep tomcat
<username>     28212  0.0  6.2 593840 64128 ?        Sl   Oct04   2:08 /opt/j2sdk1.4.2_13//bin/java -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
</code>

  * We want to terminate the process-id(s) marked with "Sl". Use the command "kill -9 <process-id>". In the example above, we would type the following in the x-term:
<code>$ kill -9 28212</code>

  * Again, do: 
<code>$ ps aux|grep tomcat </code>
to check that there are no more running instances of tomcat ("Sl"). If there are any still left, use kill -9 <process-id> to kill that tomcat process.

  * If there are no previous instances of tomcat to kill anymore, then try an ant start now:
<code>$ ant start</code>
Hopefully it will work and come back with BUILD SUCCESSFUL.




====Servlet throws root exception about 'xalan'==== 
See [[#Using Java 5 (JDK 1.5) or higher with Greenstone 3]]


</TAB>
<!--#############################################################################
#############################################################################
#############################################################################
#############################################################################-->
<TAB>
===== Greenstone2 =====
NB: These instructions will apply from 7th June 2007

 * [[http://mikesopensourcestuff.blogspot.co.nz/2009/06/step-by-step-installing-greenstone-282.html|Step By Step Installing Greenstone 2.82 on Ubuntu 8.10 (Quickstart)]]


===== Getting Greenstone2 running under linux, using SVN and Apache =====

My sample user is ''xxx'' - substitute your username for ''xxx'' in the following instructions.

==== 1. Get Greenstone from SVN ==== 
We generally advise people to install greenstone on the research partition of your computer - some people have had problems installing in their home directory. But you can try in your home directory if you like. Assuming you will use the research partition:

<code>
mkdir /Scratch/xxx (create a place for your research files if you haven't already)
cd /Scratch/xxx 
svn co http://svn.greenstone.org/gsdl/trunk gsdl
</code>

Change to the ''gsdl'' directory and check out the indexers:
<code>
cd gsdl
svn co http://svn.greenstone.org/indexers/trunk indexers
</code>

Still in the ''gsdl'' directory, compile the code:
<code>
./configure
make
make install
</code>

Now greenstone is installed in ''/Scratch/xxx/gsdl''

==== 2. If you want to use the GLI ==== 
In the ''gsdl'' directory, run:
<code>
svn co http://svn.greenstone.org/gli/trunk gli
cd gli
./makegli.sh
</code>

To start the GLI, in the ''gli'' directory run:
<code>
./gli.sh
</code>

==== 3. Install Apache ==== 
You will need to install Apache to be able to run the cgi executable.

Download a recent Apache2 binary from [[http://httpd.apache.org/download.cgi|http://httpd.apache.org/download.cgi]]
(Apache1.3 will also work, but Apache2 is much better.)

Unpack the tar file:
<code>
cd /Scratch/xxx
tar -xzf apache_2.x.x.tar.gz
</code>

Compile the source:
<code>
cd apache_2.x.x
./configure --prefix=/Scratch/xxx/httpd
make
make install
</code>
Now apache is installed in ''/Scratch/xxx/httpd''

To start and stop apache, type
<code>
 /Scratch/xxx/httpd/bin/apachectl start
 /Scratch/xxx/httpd/bin/apachectl stop
</code>
To change the port that apache runs on, edit the ''/Scratch/xxx/conf/httpd.conf'' file and 
change the Port or Listen directive to the correct port - ''8080'' is a good choice. 
After restarting apache you will then be able to access your apache installation at 
[[http://localhost:8080/|http://localhost:8080/]].

==== 4. Tell apache about greenstone ==== 
The basic method for making a greenstone installation available through a web server is to
  * Make the contents of the gsdl home folder visible on the web by placing them (or a symbolic link to them) somewhere within the htdocs folder of your apache installation.
  * Make some or all of the contents of the gsdl/cgi-bin folder available for execution on your apache server by placing a them (or a symbolic link to them) into the cgi-bin folder of your apache installation. (Make sure you include at least the 'library' or 'library.exe' executable.)
  * Create/edit the ''gsdlsite.cfg'' file inside the cgi-bin folder of apache, as a sibling of the 'library' or 'library.exe' executable.

This is the way I used:
<code>
cd /Scratch/xxx/httpd/cgi-bin
ln -s /Scratch/xxx/httpd/gsdl
ln -s /Scratch/xxx/gsdl/cgi-bin/library
cp /Scratch/xxx/gsdl/cgi-bin/gsdlsite.cfg .
cd /Scratch/xxx/htdocs
ln -s /Scratch/xxx/gsdl
</code>

You may need to edit the ''gsdlsite.cfg'' file.
Make sure the following are uncommented and set like this:
<code>
gsdlhome /Scratch/xxx/gsdl
httpprefix /gsdl
httpimg /gsdl/images
</code>
Now you need to edit apaches config file ''/Scratch/xxx/httpd/conf/httpd.conf''
 
In the ''<Directory "/Scratch/xxx/httpd/cgi-bin">'' bit, change 'Options None' to 'Options 
FollowSymlinks'

After editing the ''httpd.conf'' file, you will need to restart apache for the changes 
to take effect.

You should now be able to access your greenstone installation at 
''<nowiki>http://localhost:8080/cgi-bin/library</nowiki>''

===== Updating your installation =====
//Contributed by Julian Fox to the Greenstone mailing list.//


There is nothing particularly difficult about an upgrade on Linux, except that you may need to keep a close eye on several of the factors indicated below.  Here is one person's experience of upgrading (successfully!) from 2.73 to 2.80, not with a view to running two or more GS's on the server however - just the most recent version.

1. Initially, 2.73 had been installed at usr/local/gsdl.  In retrospect it would have been better to have created a usr/local/greenstone/2.73, then let the installation create its gsdl folder there. It's just neater, that's all, and ensures the preservation of whatever has been customised.  So this time I created usr/local/greenstone/2.80 (you could also just create usr/local/2.80, up to you) and having downloaded and unpacked the gsdl-2.80-unix file, then began the installation process. For anyone relatively new to working with Linux, it's worth noting that you find the Install script (Install.sh) in the Unix folder, and that you need to change its permission to make it executable, usually simply done by right clicking it and then checking the 'execute' spot under properties-permissions. Then you simply run ./Install.sh from that Unix folder and it gets on with the job!

2. The installation process asks questions throughout. Most of them have obvious answers. It's a good idea to have checked one's Apache setup earlier in order to have the right responses to questions relevant to that. Clearly, one obvious change given the creation of the usr/local/greenstone folder above was that Apache would now have to point to a different location; the /gsdl alias needs altering. I will not deal with the Apache changes here - that can be followed separately from existing documentation, just by making
relevant changes. I'll just note that I let the installation use the cgi-bin folder in the new gsdl installation rather than the Apache one. It's one of the questions you are asked. I had a systems administrator nearby, so any dealings with Apache had his advice behind it!

3. There was sufficient space on the disk to actually copy all collections (a whopping 30Gbs worth) across to the new 'collect' folder created in the installation process under the new gsdl folder in 2.80. I moved each one individually and checked it but my observation was that this part could hardly go wrong - just a simple copying of the collections from 2.73 to 2.80 involved no dramatic changes.

4. The really important part of the upgrade however is to ensure that you move across all other relevant files.  To be sure, I changed certain existing new files concerned (main.cfg, users.db, key.db, history.db, all to be found in /etc, and any *dm files I had altered, including the style.dm, along with new images I had added in the /images folder, and style.css there too, since I had altered that ) all to *.old or something like that, and simply transferred all my existing files from the older installation to
their appropriate folders.  I wasn't sure if there had been any change to the way macro files worked, so that's why I played safe, so I could return to the 'proper' 2.80 macros or *.cfg files if necessary and then make relevant changes afterwards.  But in fact, simply replacing the older files, at least for this upgrade, would have made no difference. Everything worked.



To update your gsdl installation, or gli, do an ''svn update'' in the appropriate 
directory and then recompile the source code. For example, to update the main gsdl 
installation, run:
<code>
cd /Scratch/xxx/gsdl 
svn update
make clean
./configure
make
make install
</code>

===== Less Complex Explanation =====
Compile up gnome-lib yourself and compile up GS:

1. svn co http://svn.greenstone.org/main/trunk/greenstone2

2. cd greenstone2
svn co http://svn.greenstone.org/main/trunk/gli
cd /back/to/greenstone2

3. ./makegs2.sh gnome-lib
This will check out the gnome-lib src code and then

---
Get a gnome-lib binary for your OS and compile up GS:

1. svn co http://svn.greenstone.org/main/trunk/greenstone2

2. cd greenstone2
svn co http://svn.greenstone.org/main/trunk/gli
cd /back/to/greenstone2

3. check out gnome-lib minimal for your OS from 
http://trac.greenstone.org/browser/gs2-extensions/gnome-lib/trunk
into your greenstone2-home/ext

4. cd ext/gnome-lib-src
   ./CASCADE-MAKE.sh
   source devel.bash
   cd /back/to/greenstone2-home
	
5. ./configure --enable-apache-httpd

6. make

5. make install
</TAB>
</TABAREA>