ReleaseKits
From GreenstoneWiki
NOTE: Some of this is still in development - there may be a few mismatches between documentation and implementation
Contents |
Introduction
The Greenstone Release Kits are software programs which assist in creating Greenstone2 and Greenstone3 releases in the three operating systems we support. They download source code from the Greenstone SVN repository, compile it, and package it up in a friendly installer program which can be distributed on the web or on CD. They are named by the operating system and the version of Greenstone they are for. For example, the Linux Release Kit for Greenstone3 is called LiRK3
There are (will be) six in total:
- LiRK2 - Linux Release Kit for Greenstone2
- WiRK2 - Windows Release Kit for Greenstone2
- MaRK2 - Mac Release Kit for Greenstone2
- LiRK3 - Linux Release Kit for Greenstone3
- WiRK3 - Windows Release Kit for Greenstone3
- MaRK3 - Mac Release Kit for Greenstone3
Note: To say that a release kit is for a particular operating system means that it runs under that operating system, and it produces releases which run under that operating system. For example, LiRK3 runs on linux, and produces linux releases.
Note: In my examples, I use LiRK3 to demonstrate use of a release kit. Please adjust the examples for the release kit you are using. I have also omitted blank lines from command output in some cases.
Installing a Release Kit
Note: For more specific installation instructions, see the wiki page for the particular release kit you want to install.
To use a particular a release kit, you must check it out from svn and install it to your system. The Release Kits are available in svn at:
http://svn.greenstone.org/release-kits/kit-name
where kit-name is one of: lirk3, wirk3, mark3, lirk2, wirk2, mark2
To get LiRK3, you would run this command:
svn co http://svn.greenstone.org/release-kits/lirk3
Installing it simply involves setting an environment variable to point to the directory where the program is stored, and adding the bin subdirectory to your PATH. E.g., you might add something like this to your ~/.profile
LIRK3_HOME=/research/oranfry/programs/lirk3 PATH=$PATH:$LIRK3_HOME/bin
To check if the release kit is properly installed, try to bring up the help page. E.g.,
$ lirk3 -help
Using a Release Kit to Create a Release
Start out by creating an empty directory to store the release files. Then create and edit the file build.properties inside this directory. In this file, we define some properties for the release. Define properties with name:value, one per line. branch.path and version are required.
Set build properties
branch.path
E.g. branch.path:trunk
E.g. branch.path:branches/my-branch
Greenstone2 releases use code from four top-level directories in our repository (which we call projects): gsdl, indexers, gli, and documentation. Greenstone3 releases use code from the same four projects plus the greenstone3 project. The branch.path property specifies whether we want code from the trunk, or a particular branch or tag, of each of these projects.
Usually, the code for the release will come from the main development trunk of each project. In this case, you should set branch.path:trunk.
Sometimes it is appropriate to use the code stored in branches or tags of each project. For example, when creating an official release of Greenstone(2 or 3) we like to keep the code for that release in branches so that we can exclude potentially buggy or untested code, in order to get a stable release. A branch is created in each project, and named after the release. For example, for release 2.75, we created gsdl/branches/2.75, indexers/branches/2.75, gli/branches/2.75 and documentation/branches/2.75. In this case we used branch.path:branches/2.75.
branch.revision
E.g. branch.revision:HEAD
E.g. branch.revision:15051
Defaults to HEAD. Specifies the repository revision to get code from. For Greenstone3 releases, don't use a revision earlier that 15050, because the Greenstone3 build.xml was not set up to honour this property before that revision.
version
E.g. version:2.80
E.g. version:3s2008.02.07
Give your release a version string to identify it. As you know, version strings for official releases are constructed from their major and minor version numbers, e.g., 2.75 or 3.04. A version string can be constructed for snapshot releases (releases from the trunk) in a similar way, such as 3s2008.02.07, which would indicate that this is a snapshot release of Greenstone3 using the code as at the 7th of February 2008. Special releases might call for a special version string, such as 3-flax-edition-1.0 for Greenstone3 bundled with flax 1.0.
The version string shows up in various places in a Greenstone installation, such as in the GLI title bar and in the server app window, and also in the filename of the installer, so choose appropriately.
The version property can either be set in the build.properties file, or it can be set in a version.properties file inside the repository itself. For Greenstone3, use /greenstone3/<branch.path>/version.properties. For Greenstone2, use /gsdl/<branch.path>/version.properties. If the version string is set in both version.properties and build.properties, the one from build.propeties will be used.
Run the Release Kit
Change to the release directory and run the release kit by typing its name on the command line (all lowecase). E.g.,
$ cd /research/oranfry/releases/3.04 $ lirk3
If all goes well, after number of minutes, the release kit will exit successfully and you will end up with four folders in the release directory. One of these is called wrapped-installer. Inside this folder is a single executable file, which is the binary installer for the release. Enjoy!
Of course, things don't always go so well. When the release process fails partway through, it pays to know a bit about how the release kit works, so that you can get the process started again from where it failed. See the next section for details.
Release Kit Usage Reference
The target tree
Each release kit is actually a wrapper to an ant script which performs all the steps of the release process. This ant script, like all ant scripts, it is made up of a target tree. Unlike all ant scripts, each target in the tree has not only a name but also an address which can be used to reference it. (Technically, these addresses are assigned by the hacked version of ant which the release kits use rather than being intrinsic to the script, but you probably don't need to worry about that.) You can view this target tree at any time by running (in the release directory):
$ lirk3 -sim
As at 2008-02-08, the target tree for LiRK3 looked like this:
lirk3:
1 compile:
1.1 checkout-greenstone3:
1.2 set-version-number-property:
1.3 dist-prepare-unix:
1.4 tweak-configure-scripts:
1.4.1 mgpp-add-static:
1.4.2 mg-add-static:
1.4.3 gs2buildextra-add-static:
1.5 dist-update-unix:
1.6 dist-configure-unix:
1.7 tweak-makefiles:
1.7.1 wv-add-static:
1.7.2 xlhtml-add-static:
1.7.3 ppthtml-add-static:
1.7.4 rtftohtml-add-static:
1.7.5 gdbm-add-static:
1.8 dist-build-unix:
1.9 fix-wget:
1.10 linux-strip-execs:
1.11 prepare-documentation:
2 create-distribution:
2.1 export-greenstone3:
2.2 dist-set-version-number-property:
2.3 create-distributions-1:
2.4 create-distributions-2:
2.5 create-distributions-3:
2.6 copy-over-build-xml:
2.7 insert-user-manual:
2.8 insert-compiled-binaries:
3 create-installer:
4 wrap:
4.1 setup-for-wrapping:
4.2 compile-wrapper:
4.3 clean-up-after-wrapping:
For an explanation of what the targets do, see the wiki page for the particular release kit you are interested in.
the -sim flag
The sim flag, which we saw in the previous section, actually stands for 'simulation'. It tells the release kit to simulate the release process. A simulation is performed by traversing the target tree as usual, but not executing any of the actual code in each target. As the tree is traversed, the name and address of each target is printed out, which results in a visual tree of targets. If you are ever in doubt, you can always pass the -sim flag to the release kit to see what it will do, before you actually ask it to go ahead and do it.
the -from argument
Sometimes you may need to restart the release process after an error, but won't want to go through the whole process from the start. In these cases, use the -from argument to specify a target address to start from. For example, let's simulate starting the process from target 2.7
$ lirk3 -sim -from 2.7
... init output omitted ...
lirk3:
2 create-distribution:
2.7 insert-user-manual:
2.8 insert-compiled-binaries:
3 create-installer:
4 wrap:
4.1 setup-for-wrapping:
4.2 compile-wrapper:
4.3 clean-up-after-wrapping:
This shows the list of targets which would be executed.
Note that target target 2 create-distribution is executed. This may seem odd since we asked the release kit to start from target 2.7. But it is necessary for target 2 to be executed, and luckily it does not cause any problems. If we want to start execution from a target deep in the tree, the release kit must execute all of its ancestor targets too. In this particular case, this means the release kit has to execute target 2 - the parent (i.e., immediate ancestor) of target 2.7. Only leaf targets have code in them, so we can safely execute these ancestor targets without fear of executing code we don't mean to execute.
Hint: in linux, the first-level targets can always be specified by name instead of address. For example, this command is valid: lirk3 -from create-installer
the -to argument
This is the opposite of the -from argument. It specifies a target which execution will proceed to, and then stop. For example, let's simulate stopping the process at target 1.5
$ lirk3 -sim -to 1.5
... init output omitted ...
lirk3:
1 compile:
1.1 checkout-greenstone3:
1.2 set-version-number-property:
1.3 dist-prepare-unix:
1.4 tweak-configure-scripts:
1.4.1 mgpp-add-static:
1.4.2 mg-add-static:
1.4.3 gs2buildextra-add-static:
1.5 dist-update-unix:
The -to argument can be combined with the -from argument. Also, as you may have noticed, the -from argument is inclusive in nature, but the -to argument is not. That means that if you specified -from 1.8, one of the targets to be run will be target 1.8 itself. On the other hand, if you specified -to 2.3, execution would proceed up to target 2.3 but target 2.3 itself would not be run.
the -descend flag
The -descend flag, like the -from and -to arguments, is a way to select a subset of targets to execute. It only makes sense when used in combination with the -from argument. It tells the release kit to only execute targets which are descendants of the target specified in -from. For example, let's simulate descending from target 1.7
$ lirk3 -from 1.7 -descend
... init output omitted ...
lirk3:
1 compile:
1.7 tweak-makefiles:
1.7.1 wv-add-static:
1.7.2 xlhtml-add-static:
1.7.3 ppthtml-add-static:
1.7.4 rtftohtml-add-static:
1.7.5 gdbm-add-static:
This flag is helpful because often we want to execute a chunk of code which corresponds exactly to a particular high-level target. For example, if you wanted to execute the create-distribution (target 1) target without carrying on to the create-installer target (target 2), you could use lirk3 -from 1 -descend.
(Actually, you could achieve the same result by using the command lirk3 -from 1 -to 2. Arguably, -to is more powerful as executing multiple -descend commands like lirk3 -from 1 -descend then lirk3 -from 2 -descend can be done in one go with a -to command like lirk3 -from 1 -to 3)
init
As you will have noticed by now, each time you invoke the release kit, it starts by executing an target called init (which has a null address). This target checks that certain conditions are being satisfied, like that the version string and branch path are set. If a condition is not satisfied, an explanation will be printed out and execution will stop. Read it and fix the problem, then try again.
notes
- The release process takes time. LiRK3 takes about an hour from start to finish on my machine. Don't Panic.
- The ant script which underpins each release kit is actually divided up into many files for modularity, and these files are brought together by include statements in a unifying build.xml file.
- In the release process, the release kit invokes other ant scripts which aren't addressed. That is why you see some targets being executed without any address.
About the Installer
The installer jar
The release kit is bundled with the software ant-installer. When the target create-installer is executed, it uses ant-installer to create an executable jar installer (a executable jar program which installs Greenstone onto the machine it's running on). The customisation of the installer is done in three xml files:
$LIRK3_HOME/installer/antinstaller-config.xml - This file describes the GUI interface for the installer. It describes the sequence of pages to click through during installation.
$LIRK3_HOME/installer/build.xml - This is an ant script which is executed on the user's machine after they have clicked gone through the GUI mentioned above and selected the installation options. It does the actual copying of files onto the user's machine, changes permissions etc. The main attraction of the ant-installer software, and the reason we use it for releases, is that it allows us to execute an ant script on the user's machine during install, which means we can capitalise on ant's massive library.
$LIRK3_HOME/ant-scripts/create-installer.xml - This ant script contains the create-installer target, and is the bootstrap to creating the installer jar. It runs the installer task which comes with ant-installer. This task creates the jar file by packaging up the Greenstone binaries, slotting in the installation build.xml script, and adding the GUI based on antinstall-config.xml as the Main-class.
The language strings for the different languages of the installer are installer found in $LIRK3_HOME/installer/cp/resources/LanguagePack*.properties
The installer wrapper
After the executable jar file has been created, the release kit will wrap it in a native binary executable for the current platform (an exe file on windows, and an executable .bin file on unix). This executable will contain: the executable jar, a tool for searching for java, some logic for extracting and running these. A second executable is also created which is the same as the first except it also contains a bundled jre. We create this executable for user's who don't know how to use a executable jar, but can handle running a native executable. The second binary executable is for user's who don't have java or don't know if they have java.
Search4j
This is a small utility for
