Greenstone Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Supporting running Greenstone in other locales on Windows 10 ======

Greenstone testing work tends to be done in an English locale, with English specific settings. However, Greenstone is meant to support language users around the world, whose PCs and laptops may be set to different locales and other language settings.

Months after Greenstone 3.09 was released, we discovered that on a machine running Windows 10 which was set to the Chinese locale and other related settings, different aspects of Greenstone 3 did not function out of the box.

**In particular, the following issues were detected, with their causes and //fixes// (which work on Windows 10) described below:**

**SYMPTOM:** Previewing http://localhost:8383/greenstone3/library after running GLI or the Greenstone 3 server still shows an empty page. If you try to investigate this issue on Windows, GS3's tomcat server output in the black DOS window can mention an XML parsing error to do with ''</displayItem>''. This comes from some strange characters or legitimate characters from other languages being randomly interspersed within a display language when interfaceConfig.xml is being parsed for loading in. The tomcat and greenstone logs, located in GS3/packages/tomcat/logs/catalina[...].out and GS3/web/logs/greenstone.log respectively, may contain similar output with multiple character sets wrongly interspersed.

This issue has been fixed since Dec 2019 and April 2020, months after Greenstone 3.09's release, and will be available from Greenstone 3.10 onwards when released. If you wish to implement this fix for yourself in Greenstone 3.09 or earlier or in any version of Greenstone 2, then read on.

====== COMPULSORY FIX PART 1 ======

**ADDITIONAL SYMPTOM:** If you were to now open a DOS prompt and try to run ''gs3-setup.bat'' inside your Greenstone 3 (or ''setup.bat ''inside your Greenstone2), there are foreign or strange characters that appear in the DOS output window, before gs3-setup.bat (or setup.bat) ends with an error message and fails to set up the Greenstone environment.

**CAUSE:** This occurs because the Windows PC code page needs to a particular code page in order for the characters in ''setup.bat'' and ''gs3-setup.bat'' (which itself also calls ''setup.bat'') to be interpreted correctly.

**FIX:**
  - Open up your Greenstone 3's gs2build/setup.bat file (or, for Greenstone 2, open up the top-level setup.bat file) in a text editor. On Windows use regular Notepad, not Notepad++ as the latter can end up changing the encoding in the file from ASCII to something else, in which case new encoding problems will be introduced.
  - Use the text editor to locate the section below near the top of the file:\\ <code>
@echo off

setlocal enabledelayedexpansion

pushd "%CD%"
CD /D "%~dp0"
set GSDLLANG=en

</code>
  - Then insert the following code after the above section:\\ <code>
:: This script's contents were created in Windows codepage 850 (Western European).
:: When this script is run in non-English locales like the Chinese locale, the 
:: character content/encoding of this script can get wrongly interpreted by the
:: different encoding of the locale. And then the GS environment gets set wrongly
:: or incompletely in those cases.
:: So we force this script to be run and interpreted in codepage 850.
:: However, the Russian display strings in this script were desgined to work in the
:: default Russian Windows codepage 866. Whereas, codepage 850 works for this
:: script's remaining English, French, Spanish display strings. So:
if ("!GSDLLANG!" == "ru") (
	chcp 866
) else (
	chcp 850
)</code>
  - Save and close the file.
  - If not launching Greenstone applications from the Windows Start Menu, then when you next run any part of the Greenstone suite hereafter to test the fix, make sure to use a fresh DOS prompt so that the updated greenstone environment will be used when you run ''setup.bat'' in greenstone 2, or ''gs3-setup.bat'' in Greenstone 3 which internally runs its ''gs2build\setup.bat''.


====== PART 2 OF FIX: TRY THESE 2 SOLUTIONS ======

**ONGOING SYMPTOM:** If you now run GLI, visiting ''http://localhost:8383/greenstone3/library'' still shows an empty page. And the output in the black DOS window can still mention an XML parsing error to do with ''</displayItem>'' and possible foreign or strange characters (which upon investigation will reveal that interfaceConfig.xml couldn't be parsed properly because the locale setting isn't as required).

**CAUSE:** Still an locale issue leading to encoding problems.

**FIX:** Try Solution 1 below first, and if that doesn't work, (optionally undo it and then) try Solution 2.

==== SOLUTION 1 ====
If you're using Greenstone 3:
  * Make sure Greenstone is not running and all Greenstone related DOS prompts have been exitted too.
  * Use a text editor to open your GS3's ''gs2build/setup.bat'' again.
  * At the very end of the file add a newline or two and paste the following, followed by another newline:\\ <code>set JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF-8</code>
  * Save and close the file ''setup.bat''.
  * Now use a text editor to open ''gs3-setup.bat'' again.
  * Go to the very bottom of this file, and just before the line that says '':end'', **replace** the line\\ <code>set JAVA_HOME=%JAVA_HOME%& set JRE_HOME=%JRE_HOME%& set ANT_HOME=%ANT_HOME%& set CLASSPATH=%CLASSPATH%</code> with the following, taking care to not add any newlines between it and the previous line nor any spaces at the end of it:\\ <code>set JAVA_HOME=%JAVA_HOME%& set JRE_HOME=%JRE_HOME%& set JAVA_TOOL_OPTIONS=%JAVA_TOOL_OPTIONS%& set ANT_HOME=%ANT_HOME%& set CLASSPATH=%CLASSPATH%</code>
  * Save and close the file ''gs3-setup.bat''.
  * Next, use your text editor to open your GS3's to open the file ''build.xml'' for editing.
  * Locate the line that contains:\\ <code><exec executable="${catalina.home}/bin/startup.bat" osfamily="windows" dir="${catalina.home}/bin" spawn="true"></code>\\ and paste the following in a new line immediately after:\\ <code><env key="JAVA_TOOL_OPTIONS" value="-Dfile.encoding=UTF-8"/></code>
  * Still in ''build.xml'', also locate the line containing:\\ <code><exec executable="${catalina.home}/bin/shutdown.bat" osfamily="windows" dir="${catalina.home}/bin" spawn="false"></code> and once more, paste the following in a new line immediately after it:\\ <code><env key="JAVA_TOOL_OPTIONS" value="-Dfile.encoding=UTF-8"/></code>
  * Save and close the file ''build.xml''.
  * Relaunch GLI or the Greenstone Server Interface and preview your collections. They should now hopefully work.


==== SOLUTION 2 ====
If the above didn't work, try the following to get previewing to work again.

For this, Windows will require system configuration changes followed by a restart.
     * POSSIBLY OPTIONAL: Rightclick on your Windows Desktop, choose ''Display Settings''. In the search box, search for "Language Settings". Then the ''Language'' configuration page opens, under ''Windows Display Language'' select the language setting appropriate for you.
     * POSSIBLY OPTIONAL: Go to ''Control Panel'' > ''Clock and Region'' > ''Region''.
        * In the ''Formats'' tab, under ''Format'', select your language/country's language format setting.
        * In the ''Administrative'' tab, under the ''Language for non-unicode programs'' section, press the ''Change system locale'' button. Your preferred locale may already be selected, if not select it here.
  * **Most importantly, available on Windows 10,**, return to Windows' ''Control Panel'' > ''Clock and Region'' > ''Region'' > ''Adminstrative'' tab > ''Change system locale'' button.\\ Here, ensure the tickbox option for ''Beta: Use Unicode UTF-8 for worldwide language support'' is ticked. (Details of what this setting does when you switch it on are [[https://stackoverflow.com/questions/56419639/what-does-beta-use-unicode-utf-8-for-worldwide-language-support-actually-do|here]].) While investigating the overall encoding issues described on this page in a Chinese locale, we found that this new setting available on Windows 10 more fully supports the chcp encoding fix listed in Part 1 of the solution. Earlier versions of the Windows OS may not provide complete support because this new setting is absent.
  * Restart your Windows machine.
  * Try running GLI or the Greenstone Server and previewing your collection now. If there are still issues, see whether the DOS prompt contains error messages mentioning that mention issues running ''pluginfo.pl''. If so, see step 2 below.

===== ANOTHER RELATED SYMPTOM AND ITS FIX =====
//We noticed the following symptom only once in the 3 test cases so far. It has not occurred in the remaining 2 test cases, so although we document its solution here, you may not need to carry out the
instructions for this solution until you actual notice the problem (the symptom) described here.//

**SYMPTOM:** Unable to successfully and completely run certain perl scripts successfully such as ''perl -S pluginfo.pl -describeall'', despite setting up the Greenstone environment by running gs3-setup.\\ As a result, GLI may fail to launch properly.

You can test if this is the case by opening a new DOS prompt, using it to navigate into your Greenstone 3 installation area and then running the following 2 commands in sequence there:\\<code>gs3-setup.bat
perl -S pluginfo.pl -describeall</code>

If you see error messages, proceed with the fix for this described below.

**CAUSE:**This is because the perl Locale.pm module in 3.09 and earlier releases was insufficient in its character encoding support.

**FIX:** Fixes for this have been committed for the future Greenstone 3.10 release and onwards, but at present they have not been made active yet.  

  * If you're working with Greenstone 2.87 or earlier or Greenstone 3.09 or earlier, then:
     - Use the Windows file explorer to go to ''perllib/cpan/Encode'' (which in a GS3 installation is located within its ''gs2build'' subfolder). Then rename the existing ''Locale.pm'' file to ''Locale.pm.old''. Next, download [[http://trac.greenstone.org/browser/main/trunk/greenstone2/perllib/cpan/Encode/Locale.pm.newForLocaleIssue|Locale.pm.forLocaleIssue]] from [[http://trac.greenstone.org/export/34105/main/trunk/greenstone2/perllib/cpan/Encode/Locale.pm.newForLocaleIssue|here]] and [[http://trac.greenstone.org/browser/main/trunk/greenstone2/perllib/cpan/Encode/Alias.pm.forLocaleIssue|Alias.pm.forLocaleIssue]] from [[http://trac.greenstone.org/export/34105/main/trunk/greenstone2/perllib/cpan/Encode/Alias.pm.forLocaleIssue|here]] into your ''perllib/cpan/Encode'' folder, and rename them to ''Locale.pm'' and ''Alias.pm'', respectively.
     - Use a text editor to open up ''perllib/plugins/DirectoryPlugin.pm'' (in a GS3 installation, this is located withing GS3's ''gs2build'' subfolder). Somewhere near the top of this file, find the line that says:<code>no strict 'subs';</code>\\ and insert the following line after it:<code>use Encode::Alias;</code>\\ Then save the file.
  * If you're working with a nightly ''caveat-emptor'' Greenstone binary obtained from www.greenstone.org/caveat-emptor, the fixes will be available in your Greenstone installation and you just have to manually activate the fixes as follows:
     - Use the Windows file explorer to go to ''perllib/cpan/Encode'' (which in a GS3 installation is located within its ''gs2build'' subfolder). Then rename the existing ''Locale.pm'' file to ''Locale.pm.old'', then you'll finally be able to rename the ''Locale.pm.forLocaleIssue'' file to ''Locale.pm'' and rename ''Alias.pm.forLocaleIssue'' to ''Alias.pm''.
     - Use a text editor to open up ''perllib/plugins/DirectoryPlugin.pm'' (in a GS3 installation, this is located withing GS3's ''gs2build'' subfolder). Somewhere near the top of this file, find the line that says:<code>#use Encode::Alias;</code>\\ and check out the ''#'' sign at the start of that line so that you get<code>use Encode::Alias;</code>\\ Then save the file with this change.