User Tools

Site Tools


en:supporting_other_locales

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
en:supporting_other_locales [2019/12/13 04:07] anupamaen:supporting_other_locales [2023/03/13 01:46] (current) – external edit 127.0.0.1
Line 1: Line 1:
-====== Supporting other locales on Windows 10 ======+ 
 + 
 + 
 +====== Supporting running Greenstone in other locales on Windows 10 ======
  
 Greenstone testing work tends to be done in an English locale, with English specific settings. However, Greenstone is meant to support language users around the world, whose PCs and laptops may be set to different locales and other language settings. Greenstone testing work tends to be done in an English locale, with English specific settings. However, Greenstone is meant to support language users around the world, whose PCs and laptops may be set to different locales and other language settings.
Line 7: Line 10:
 **In particular, the following issues were detected, with their causes and //fixes// (which work on Windows 10) described below:** **In particular, the following issues were detected, with their causes and //fixes// (which work on Windows 10) described below:**
  
-1. **SYMPTOM:** Visiting http://localhost:8383/greenstone3/library after running the Greenstone 3 server shows an empty page. If you try to investigate this issue on Windows, GS3's tomcat server output in the black DOS window can contain some strange characters or legitimate characters from other languages randomly interspersed within a display language when interfaceConfig.xml is being parsed for loading in. The tomcat and greenstone logs, located in GS3/packages/tomcat/logs/catalina[...].out and GS3/web/logs/greenstone.log respectively, may contain similar output with multiple character sets wrongly interspersed.+**SYMPTOM:** Previewing http://localhost:8383/greenstone3/library after running GLI or the Greenstone 3 server still shows an empty page. If you try to investigate this issue on Windows, GS3's tomcat server output in the black DOS window can mention an XML parsing error to do with ''</displayItem>''. This comes from some strange characters or legitimate characters from other languages being randomly interspersed within a display language when interfaceConfig.xml is being parsed for loading in. The tomcat and greenstone logs, located in GS3/packages/tomcat/logs/catalina[...].out and GS3/web/logs/greenstone.log respectively, may contain similar output with multiple character sets wrongly interspersed.
  
-**CAUSE:** This occurs because the PC'encoding setting does not correctly interpret the bytes of data of the interfaceConfig.xml file being loaded in, due to encoding differences.+This issue has been fixed since Dec 2019 and April 2020, months after Greenstone 3.09'release, and will be available from Greenstone 3.10 onwards when released. If you wish to implement this fix for yourself in Greenstone 3.09 or earlier or in any version of Greenstone 2then read on.
  
-**FIX:** This issue has been fixed since Dec 2019, several months after Greenstone 3.09's release, and will be available from Greenstone 3.10 onwards when released.+====== COMPULSORY FIX PART 1 ======
  
-If you wish to implement this fix for yourself in Greenstone 3.09 or earlier or in any version of Greenstone 2then:+**ADDITIONAL SYMPTOM:** If you were to now open a DOS prompt and try to run ''gs3-setup.bat'' inside your Greenstone 3 (or ''setup.bat ''inside your Greenstone2), there are foreign or strange characters that appear in the DOS output windowbefore gs3-setup.bat (or setup.bat) ends with an error message and fails to set up the Greenstone environment. 
 + 
 +**CAUSE:** This occurs because the Windows PC code page needs to a particular code page in order for the characters in ''setup.bat'' and ''gs3-setup.bat'' (which itself also calls ''setup.bat'') to be interpreted correctly. 
 + 
 +**FIX:**
   - Open up your Greenstone 3's gs2build/setup.bat file (or, for Greenstone 2, open up the top-level setup.bat file) in a text editor. On Windows use regular Notepad, not Notepad++ as the latter can end up changing the encoding in the file from ASCII to something else, in which case new encoding problems will be introduced.   - Open up your Greenstone 3's gs2build/setup.bat file (or, for Greenstone 2, open up the top-level setup.bat file) in a text editor. On Windows use regular Notepad, not Notepad++ as the latter can end up changing the encoding in the file from ASCII to something else, in which case new encoding problems will be introduced.
   - Use the text editor to locate the section below near the top of the file:\\ <code>   - Use the text editor to locate the section below near the top of the file:\\ <code>
Line 39: Line 46:
 ) else ( ) else (
  chcp 850  chcp 850
-)**</code>+)</code>
   - Save and close the file.   - Save and close the file.
   - If not launching Greenstone applications from the Windows Start Menu, then when you next run any part of the Greenstone suite hereafter to test the fix, make sure to use a fresh DOS prompt so that the updated greenstone environment will be used when you run ''setup.bat'' in greenstone 2, or ''gs3-setup.bat'' in Greenstone 3 which internally runs its ''gs2build\setup.bat''.   - If not launching Greenstone applications from the Windows Start Menu, then when you next run any part of the Greenstone suite hereafter to test the fix, make sure to use a fresh DOS prompt so that the updated greenstone environment will be used when you run ''setup.bat'' in greenstone 2, or ''gs3-setup.bat'' in Greenstone 3 which internally runs its ''gs2build\setup.bat''.
-  - Windows requires further configuration changes followed by a restart. The most important is the 3rd one below: + 
-     * Rightclick on your Windows Desktop, choose ''Display Settings''. In the search box, search for "Language Settings". Then the ''Language'' configuration page opens, under ''Windows Display Language'' select the language setting appropriate for you. + 
-     * Go to ''Control Panel'' > ''Clock and Region'' > ''Region''.+====== PART 2 OF FIX: TRY THESE 2 SOLUTIONS ====== 
 + 
 +**ONGOING SYMPTOM:** If you now run GLI, visiting ''http://localhost:8383/greenstone3/library'' still shows an empty page. And the output in the black DOS window can still mention an XML parsing error to do with ''</displayItem>'' and possible foreign or strange characters (which upon investigation will reveal that interfaceConfig.xml couldn't be parsed properly because the locale setting isn't as required). 
 + 
 +**CAUSE:** Still an locale issue leading to encoding problems. 
 + 
 +**FIX:** Try Solution 1 below first, and if that doesn't work, (optionally undo it and then) try Solution 2. 
 + 
 +==== SOLUTION 1 ==== 
 +If you're using Greenstone 3: 
 +  * Make sure Greenstone is not running and all Greenstone related DOS prompts have been exitted too. 
 +  * Use a text editor to open your GS3's ''gs2build/setup.bat'' again. 
 +  * At the very end of the file add a newline or two and paste the following, followed by another newline:\\ <code>set JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF-8</code> 
 +  * Save and close the file ''setup.bat''
 +  * Now use a text editor to open ''gs3-setup.bat'' again. 
 +  * Go to the very bottom of this file, and just before the line that says '':end'', **replace** the line\\ <code>set JAVA_HOME=%JAVA_HOME%& set JRE_HOME=%JRE_HOME%& set ANT_HOME=%ANT_HOME%& set CLASSPATH=%CLASSPATH%</code> with the following, taking care to not add any newlines between it and the previous line nor any spaces at the end of it:\\ <code>set JAVA_HOME=%JAVA_HOME%& set JRE_HOME=%JRE_HOME%& set JAVA_TOOL_OPTIONS=%JAVA_TOOL_OPTIONS%& set ANT_HOME=%ANT_HOME%& set CLASSPATH=%CLASSPATH%</code> 
 +  * Save and close the file ''gs3-setup.bat''
 +  * Next, use your text editor to open your GS3's to open the file ''build.xml'' for editing. 
 +  * Locate the line that contains:\\ <code><exec executable="${catalina.home}/bin/startup.bat" osfamily="windows" dir="${catalina.home}/bin" spawn="true"></code>\\ and paste the following in a new line immediately after:\\ <code><env key="JAVA_TOOL_OPTIONS" value="-Dfile.encoding=UTF-8"/></code> 
 +  * Still in ''build.xml'', also locate the line containing:\\ <code><exec executable="${catalina.home}/bin/shutdown.bat" osfamily="windows" dir="${catalina.home}/bin" spawn="false"></code> and once more, paste the following in a new line immediately after it:\\ <code><env key="JAVA_TOOL_OPTIONS" value="-Dfile.encoding=UTF-8"/></code> 
 +  * Save and close the file ''build.xml''
 +  * Relaunch GLI or the Greenstone Server Interface and preview your collections. They should now hopefully work. 
 + 
 + 
 +==== SOLUTION 2 ==== 
 +If the above didn't work, try the following to get previewing to work again. 
 + 
 +For this, Windows will require system configuration changes followed by a restart. 
 +     POSSIBLY OPTIONAL: Rightclick on your Windows Desktop, choose ''Display Settings''. In the search box, search for "Language Settings". Then the ''Language'' configuration page opens, under ''Windows Display Language'' select the language setting appropriate for you. 
 +     POSSIBLY OPTIONAL: Go to ''Control Panel'' > ''Clock and Region'' > ''Region''.
         * In the ''Formats'' tab, under ''Format'', select your language/country's language format setting.         * In the ''Formats'' tab, under ''Format'', select your language/country's language format setting.
         * In the ''Administrative'' tab, under the ''Language for non-unicode programs'' section, press the ''Change system locale'' button. Your preferred locale may already be selected, if not select it here.         * In the ''Administrative'' tab, under the ''Language for non-unicode programs'' section, press the ''Change system locale'' button. Your preferred locale may already be selected, if not select it here.
-        * **Most importantly**still in the Adminstrative tab, **on Windows 10,** there is a tickbox option for ''Beta: Use Unicode UTF-8 for worldwide language support''. Ensure you tick this. (Details of what this setting does when you switch it on are [[https://stackoverflow.com/questions/56419639/what-does-beta-use-unicode-utf-8-for-worldwide-language-support-actually-do|here]].) While investigating the overall encoding issues described on this page in a Chinese locale, we found that this new setting available on Windows 10 more fully supports the encoding fixes listed above. Earlier versions of the Windows OS may not provide complete support because this new setting is absent. +  * **Most importantly, available on Windows 10,**, return to Windows' ''Control Panel'' > ''Clock and Region'' > ''Region'' > ''Adminstrative'' tab > ''Change system locale'' button.\\ Here, ensure the tickbox option for ''Beta: Use Unicode UTF-8 for worldwide language support'' is ticked. (Details of what this setting does when you switch it on are [[https://stackoverflow.com/questions/56419639/what-does-beta-use-unicode-utf-8-for-worldwide-language-support-actually-do|here]].) While investigating the overall encoding issues described on this page in a Chinese locale, we found that this new setting available on Windows 10 more fully supports the chcp encoding fix listed in Part 1 of the solution. Earlier versions of the Windows OS may not provide complete support because this new setting is absent. 
-     Don't forget to restart your machine.+  Restart your Windows machine
 +  * Try running GLI or the Greenstone Server and previewing your collection now. If there are still issues, see whether the DOS prompt contains error messages mentioning that mention issues running ''pluginfo.pl''. If so, see step 2 below.
  
 +===== ANOTHER RELATED SYMPTOM AND ITS FIX =====
 +//We noticed the following symptom only once in the 3 test cases so far. It has not occurred in the remaining 2 test cases, so although we document its solution here, you may not need to carry out the
 +instructions for this solution until you actual notice the problem (the symptom) described here.//
  
-2. **SYMPTOM:** Unable to successfully and completely run certain perl scripts successfully such as ''perl -S pluginfo.pl -describeall'', despite setting up the Greenstone environment by running gs3-setup. +**SYMPTOM:** Unable to successfully and completely run certain perl scripts successfully such as ''perl -S pluginfo.pl -describeall'', despite setting up the Greenstone environment by running gs3-setup.\\ As a result, GLI may fail to launch properly. 
 + 
 +You can test if this is the case by opening a new DOS prompt, using it to navigate into your Greenstone 3 installation area and then running the following 2 commands in sequence there:\\<code>gs3-setup.bat 
 +perl -S pluginfo.pl -describeall</code> 
 + 
 +If you see error messages, proceed with the fix for this described below.
  
 **CAUSE:**This is because the perl Locale.pm module in 3.09 and earlier releases was insufficient in its character encoding support. **CAUSE:**This is because the perl Locale.pm module in 3.09 and earlier releases was insufficient in its character encoding support.
  
-**FIX:** Fixes for this have been committed for the future Greenstone 3.10 release and onwards, but at present they have not been made active yet. +**FIX:** Fixes for this have been committed for the future Greenstone 3.10 release and onwards, but at present they have not been made active yet.   
-  * If you're working with a nightly Greenstone binary obtained from www.greenstone.org/caveat-emptor, the fixes will be available in your Greenstone installation and you just have to manually activate the fixes as follows:+ 
 +  * If you're working with Greenstone 2.87 or earlier or Greenstone 3.09 or earlier, then: 
 +     - Use the Windows file explorer to go to ''perllib/cpan/Encode'' (which in a GS3 installation is located within its ''gs2build'' subfolder). Then rename the existing ''Locale.pm'' file to ''Locale.pm.old''. Next, download [[http://trac.greenstone.org/browser/main/trunk/greenstone2/perllib/cpan/Encode/Locale.pm.newForLocaleIssue|Locale.pm.forLocaleIssue]] from [[http://trac.greenstone.org/export/34105/main/trunk/greenstone2/perllib/cpan/Encode/Locale.pm.newForLocaleIssue|here]] and [[http://trac.greenstone.org/browser/main/trunk/greenstone2/perllib/cpan/Encode/Alias.pm.forLocaleIssue|Alias.pm.forLocaleIssue]] from [[http://trac.greenstone.org/export/34105/main/trunk/greenstone2/perllib/cpan/Encode/Alias.pm.forLocaleIssue|here]] into your ''perllib/cpan/Encode'' folder, and rename them to ''Locale.pm'' and ''Alias.pm'', respectively. 
 +     - Use a text editor to open up ''perllib/plugins/DirectoryPlugin.pm'' (in a GS3 installation, this is located withing GS3's ''gs2build'' subfolder). Somewhere near the top of this file, find the line that says:<code>no strict 'subs';</code>\\ and insert the following line after it:<code>use Encode::Alias;</code>\\ Then save the file
 +  * If you're working with a nightly ''caveat-emptor'' Greenstone binary obtained from www.greenstone.org/caveat-emptor, the fixes will be available in your Greenstone installation and you just have to manually activate the fixes as follows:
      - Use the Windows file explorer to go to ''perllib/cpan/Encode'' (which in a GS3 installation is located within its ''gs2build'' subfolder). Then rename the existing ''Locale.pm'' file to ''Locale.pm.old'', then you'll finally be able to rename the ''Locale.pm.forLocaleIssue'' file to ''Locale.pm'' and rename ''Alias.pm.forLocaleIssue'' to ''Alias.pm''.      - Use the Windows file explorer to go to ''perllib/cpan/Encode'' (which in a GS3 installation is located within its ''gs2build'' subfolder). Then rename the existing ''Locale.pm'' file to ''Locale.pm.old'', then you'll finally be able to rename the ''Locale.pm.forLocaleIssue'' file to ''Locale.pm'' and rename ''Alias.pm.forLocaleIssue'' to ''Alias.pm''.
      - Use a text editor to open up ''perllib/plugins/DirectoryPlugin.pm'' (in a GS3 installation, this is located withing GS3's ''gs2build'' subfolder). Somewhere near the top of this file, find the line that says:<code>#use Encode::Alias;</code>\\ and check out the ''#'' sign at the start of that line so that you get<code>use Encode::Alias;</code>\\ Then save the file with this change.      - Use a text editor to open up ''perllib/plugins/DirectoryPlugin.pm'' (in a GS3 installation, this is located withing GS3's ''gs2build'' subfolder). Somewhere near the top of this file, find the line that says:<code>#use Encode::Alias;</code>\\ and check out the ''#'' sign at the start of that line so that you get<code>use Encode::Alias;</code>\\ Then save the file with this change.
-  * If you're working with Greenstone 2.87 or earlier or Greenstone 3.09 or earlier, then: 
-     - Use the Windows file explorer to go to ''perllib/cpan/Encode'' (which in a GS3 installation is located within its ''gs2build'' subfolder). Then rename the existing ''Locale.pm'' file to ''Locale.pm.old''. Next, download [[http://trac.greenstone.org/browser/main/trunk/greenstone2/perllib/cpan/Encode/Locale.pm.newForLocalIssue?rev=33721|Locale.pm.forLocaleIssue]] and [[http://trac.greenstone.org/browser/main/trunk/greenstone2/perllib/cpan/Encode/Alias.pm.forLocalIssue?rev=33721|Alias.pm.forLocaleIssue]] into your ''perllib/cpan/Encode'' folder, and rename them to ''Locale.pm'' and ''Alias.pm'', respectively. 
-     - Use a text editor to open up ''perllib/plugins/DirectoryPlugin.pm'' (in a GS3 installation, this is located withing GS3's ''gs2build'' subfolder). Somewhere near the top of this file, find the line that says:<code>no strict 'subs';</code>\\ and insert the following line after it:<code>use Encode::Alias;</code>\\ Then save the file. 
- 
- 
  
en/supporting_other_locales.1576210056.txt.gz · Last modified: 2019/12/13 04:07 by anupama