PHAROS Image/Video indexing

Instructions for installing PHAROS Image/Video indexing with Greenstone3. Based on GCC 4.4.3 and Ubuntu Linux 10.4 (Lucid Lynx).

Initial instructions by Katherine Don, with modifications thanks to Evan Brown and Suzanne Little, Knowledge Media Institute

We use svn 1.5 and ant 1.7.1 and java 1.6.

svn --version
ant -version
java --version

ANT_HOME and JAVA_HOME should be set, and the correct versions on your path.

Especially with a fresh install check that the following packages are installed: gettext, automake, autoconf, libtool, libmp4v2, libglib-2.0-dev(+dependencies), libgsf-1-dev(+dependencies), libpng12-dev. The build-essentials package might help also. Make sure it's sun java and not openJDK or icedtea!

svn co http://svn.greenstone.org/main/trunk/greenstone3 pharos-greenstone3
cd pharos-greenstone3

Generate build.properties from build.properties.in

ant  

At this stage, you may like to change tomcat's port number (default is 8080) in build.properties (and shutdown port too if you have or are likely to have more than one tomcat running). This is also the place to add proxy settings if required.

Check out building and other supplementary code

ant prepare 

cd ext
svn co http://svn.greenstone.org/gs3-extensions/pharos-videois/trunk pharos-videois
svn co http://svn.greenstone.org/gs3-extensions/pharos-imageis/trunk pharos-imageis
cd ../gs2build/ext
svn co http://svn.greenstone.org/gs2-extensions/video-and-audio-src/trunk video 
cd ../../

Prior to running the Ant install build script, if you are running with GCC 4.4.x and/or the latest release of Ubuntu Linux (10.4 Lucid Lynx), the following changes and steps are required:

1. pharos-greenstone3/gs2build/common-src/indexers/mgpp/text/mg_files.cpp: Line 45: Change to const char *s (to avoid invalid conversion error).

2. Make sure that the GNU Gettext package is installed - to avoid a missing msgfmt error during compile. On Ubuntu with standard repositories enabled, this is accomplished most easily by: sudo apt-get install gettext

3. Make sure that automake and autoconf packages are installed and available.

4. The faac audio encoder module: changes are required to pharos-greenstone3/gs2build/ext/video/build-srcpack/packages/faac-1.26/configure.in so that an appropriate Makefile is generated in the mp4v2 folder.

Remove line: AC_DEFUN(MY_DEFINE, [ AC_DEFINE($1, 1, [define if needed]) ]) Replace with: AC_DEFUN([MY_DEFINE], [ AC_DEFINE($1, 1, [define if needed]) ])

Remove: AC_OUTPUT(common/Makefile common/mp4v2/Makefile \ libfaac/Makefile frontend/Makefile \ include/Makefile Makefile)

Replace with: AC_CONFIG_FILES(common/Makefile) AC_CONFIG_FILES(common/mp4v2/Makefile) AC_CONFIG_FILES(libfaac/Makefile) AC_CONFIG_FILES(frontend/Makefile) AC_CONFIG_FILES(include/Makefile) AC_CONFIG_FILES(Makefile) AC_OUTPUT

Then do autoreconf –vif to rebuild the makefiles

5. Ruby 1.86 package: a change is required to pharos-greenstone3/gs2build/ext/video/build-srcpack/packages/ruby-1.8.6/math.c Line 37: change to start #elif defined( rather than #elif define( This is fixed in Ruby 1.87 which is the currently used version.

Now proceed with the installation:

  ant install
  source gs3-setup.sh
compile video extension
  cd  gs2build/ext/video
  ./CASCADE-MAKE.sh

You can ignore the instruction about AddHandler and httpd.conf - that is for Apache in GS2.

compile gs3 extensions

  cd ../../../ext/pharos-imageis

Copy the axis jar files into tomcat:

  mkdir ../../packages/tomcat/sharedlibs
  cp lib/*.jar ../../packages/tomcat/sharedlibs

Edit tomcat's properties file:

  emacs ../../packages/tomcat/conf/catalina.properties

Find the shared.loader property and set it to the following:

  shared.loader=${catalina.home}/sharedlibs/*.jar

To compile and install pharos:

  ant configure   #generates config files
  ant compile  #compiles code
  ant install  #copies files into greenstone3 area
  cd ../..

(Note, there is nothing to do to set up pharos-videois)

 source gs3-setup.sh
 pharos-start-derby.pl
 ant start

Then visit in a browser the URL printed on the screen Choose 'standard servlet' ⇒ should be …/gs3library in URL, not ../library.

Note, pharos derby database and tomcat need to running before building the collections. If you use a new terminal, 'source gs3-setup.sh' must be run in top level pharos-greenstone3 folder before you can run the perl scripts.

We have prepared a sample collection - contains some images and an appropriate config file. The images don't matter, but the config file can be tricky to set up.

Download the collection from http://www.cs.waikato.ac.nz/~kjdon/pharos/image-demo.tgz, unpack into pharos-greenstone3/web/sites/localsite/collect

Build the collection:

  import.pl -site localsite image-demo
  buildcol.pl -site localsite image-demo

inside web/sites/localsite/collect/image-demo, rename building to index (delete old index if rebuilding)

Retstart tomcat: 'ant restart' in top level pharos-greenstone3

Tomcat must be restarted to (re)load the collection (can also be done with a=s&sa=c arguments to gs3library in browser), and to reload the derby database for pharos - it seems to cache the contents?

View the collection - go back to browser and reload your greenstone3 home page.

From browse→titles, can see a list of documents, then click on one and get to the document page. From here can find similar images *from the current collection*. It shouldn't be too hard to modify this to find images across all collections. The search is currently returning all results and we are filtering out the ones from this collection. So could just leave them all in. But then need to link to those images. And would there be images in the database that are not in a greenstone collection for this servlet?

We have prepared a sample collection - contains some videos and an appropriate config file. The config file can be tricky to set up. And the sample videos are ones that we know can be converted to the correct form. Also contains flash stuff for streaming video player.

Download the collection from http://www.cs.waikato.ac.nz/~kjdon/pharos/video-demo.tgz, unpack into pharos-greenstone3/web/sites/localsite/collect

Need to change the default collectionConfig.xml on line 60 should be:

  <option name="-enable_streaming" value="flv"/>

Build the collection:

  import.pl -site localsite video-demo
  buildcol.pl -site localsite video-demo

inside web/sites/localsite/collect/video-demo, rename building to index (delete old index if rebuilding)

Retstart tomcat: 'ant restart' in top level pharos-greenstone3

Go to home page and click on Pharos Video Demo.

  mkcol.pl -site localsite <collname>

Will make a new collection in pharos-greenstone3/web/sites/localsite/collect/collname Put images and/or videos into the import directory.

Modify the etc/collectionConfig.xml file. Would be best to copy the image/video demo one probably, and make any appropriate modifications.

Need to copy the flash directory to the new collection so videos will display embedded in the page:

  cp -r $GSDL3SRCHOME/gs2build/ext/video/web/flash $GSDL3SRCHOME/web/sites/localsite/collect/video-demo/

Then run the import and build scripts as above, rename building to index, and restart tomcat.

• Greenstone log file is in web/logs/greenstone.log
• Tomcat log file is in packages/tomcat/logs/catalina.out

The above directions may be enough to get a working version. However if you get a java.lang.NoClassDefFoundError: org/apache/axis2/client/Stub or similar when trying to view the collections page after setting up image or video collections then the following may fix it. The underlying cause seems to be some issue with the tomcat shared.loader not finding the extra jars for pharos_image in the sharedlib directory.

  cp $GSDL3SRCHOME/packages/tomcat/sharedlib/*.jar $GSDL3SRCHOME/web/WEB-INF/lib/
  ant restart