en:user_advanced:command_line_building
Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
en:user_advanced:command_line_building [2014/04/13 23:52] – external edit 127.0.0.1 | en:user_advanced:command_line_building [2023/03/13 01:46] (current) – external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Command Line Building ====== | ||
- | <!-- id:26 -->It is possible to create and build collections directly from the command line. This | ||
- | page provides the basic information on building Greenstone collections on the command line. The full instructions | ||
- | are provided for Windows users. If you are on a MacOS/ | ||
- | commands themselves are slightly different. These differences are listed in the [[# | ||
- | This page goes through the complete steps for creating a new collection and building it on the command line. | ||
- | However, in some cases you may like to create a collection using the Librarian Interface and just rebuild it on the command line. In this case, you'll need to open the terminal, set up the environment, | ||
- | <TABAREA tabs=" | ||
- | <TAB> | ||
- | ===== Windows | + | ====== |
- | ==== Open a terminal | + | It is possible |
- | On Windows, there are several different ways to open a DOS terminal (a black console screen known as the DOS Prompt). Do one of the following: | + | page provides |
- | * '' | + | |
- | * Under the Start menu, type '' | + | |
- | * Hold down your keyboard' | + | |
- | * In any Windows Explorer, hold down Shift and right click in an empty area in the window. Select '' | + | |
+ | The first section shows how to rebuild a collection that has been created and edited in GLI. GLI doesn' | ||
- | ==== Setup the Environment ==== | + | The second part shows how to create, edit and build a collection entirely using the command line. |
- | In order to build collections in Greenstone (or run any other Greenstone scripts from the | + | ===== Using GLI to create a collection, then using command line for building ===== |
- | command line), you must first setup the terminal' | + | |
- | change into the directory where Greenstone has been installed. | + | |
- | Assuming Greenstone was installed in its default location (and your user name was " | + | |
- | < | + | If your collection will grow very large, it will save you time to build it using command line building tools. Initially, using GLI, you want to |
- | cd C: | + | |
- | </ | + | |
- | //**Note** if the path to your Greenstone installation includes spaces (e.g. Program Files), you **must** | + | |
- | put quotations around the path. For example: '' | + | |
- | Next, at the prompt type: | + | * Create a new collection |
+ | * Add a few documents and metadata | ||
+ | * Configure your collection. What indexes, plugin options, classifiers etc do you need? | ||
+ | * Build it in GLI and preview. Do you need to change configuration settings? | ||
- | < | + | Once you have the collection set up the way you want, then you can start adding the bulk of your documents. You can do this using GLI. And add metadata using GLI. |
- | gs3-setup | + | |
- | </ | + | |
- | This batch file (which | + | When its time to build, |
- | look for Greenstone programs. | + | |
- | //Note: On Windows 95/98 systems running '' | + | ==== Set up Greenstone |
- | may fail with an **Out of environment space** error. | + | |
- | If this happens, you should edit your system' | + | |
- | file (normally found at '' | + | |
- | '' | + | |
- | drive letter) to expand the size of the environment table. You' | + | |
- | need to reboot for this change to take effect, and then repeat the steps above for Greenstone.// | + | |
- | If, later on in your interactive session at the DOS prompt, | + | To begin, you will need to open a terminal window (see [[# |
- | you wish to return | + | Run the following command to setup the environment: |
- | '' | + | |
- | **//If you close your DOS window and start another one, | + | ^Greenstone version^Windows^Linux/Mac^ |
- | you will need to invoke '' | + | |2|setup|source setup.bash| |
+ | |3|gs3-setup|source | ||
- | <!-- id:33 -->Now you are in a position to make, build and rebuild collections. | + | Note, if you close your terminal window |
- | ==== Create a collection | + | ==== Build on the command line ==== |
- | The first program we will look at is the Perl program '' | + | Now you can build the collection. |
- | whose name stands for “make a collection.” Typing '' | + | |
- | the full list of options, which you can also view [[script_options# | + | |
- | //(If your Windows environment | + | The main command for rebuilding a collection |
- | files ending in '' | + | |
- | To create a new collection: | + | ^Greenstone version^Windows^Linux/ |
- | <code> | + | |2|perl -S full-rebuild.pl |
- | perl -S mkcol.pl [options] collection-name | + | |3|perl -S full-rebuild.pl -site localsite |
- | </code> | + | |
- | <!-- id:34 -->For example, to create a collection | + | Notes: |
- | with the creator' | + | * replace < |
+ | * If you have a custom site for Greenstone 3, replace | ||
+ | * There are options for full-rebuild.pl. View the list of options by running [perl -S] full-rebuild.pl -h | ||
+ | * For Linux and MacOS, you can leave off the perl -S for all the perl commands on this page. If your Windows environment is set up to associate the Perl application with | ||
+ | files ending in '' | ||
- | < | + | Running full-rebuild.pl will reimport and index all the documents. You will need to do this if you have changed plugin options, or other configuration options. If the configuration hasn't changed, and you just want to add new documents or update modified documents, then you should use incremental building. |
- | perl —S mkcol.pl -site localsite —creator [email protected].ac.nz dlpeople | + | |
- | </ | + | |
- | \\ | + | |
- | //(Since Greenstone3 allows you to have multiple [[en: | + | |
- | collection is in. The default site is called '' | + | |
- | <!-- id:36 -->To view the newly created files, move to the newly created | + | ==== Incremental building ==== |
- | collection directory by typing | + | |
- | < | + | Incremental building is where you only process the new or changed documents each time you build, thereby speeding up the build process. New and modified documents will be processed, and deleted documents will be removed from the collection. If metadata has changed, then documents will be reprocessed. |
- | cd %GSDL3HOME%\sites\localsite\collect\dlpeople | + | |
- | </ | + | |
+ | Important note for collection design: Greenstone can notice that metadata in a folder has been added/ | ||
+ | belongs to. Therefore, if metadata in a folder has changed (including new metadata being added), then all documents in that folder will be reimported. This means that if you have all your documents in the top level import folder, adding new metadata or changing any metadata for any document will result in all documents being reimported. If you intend to do incremental import, then please organize your documents into subfolders. That way modifying metadata for some documents won't result in all other documents being reimported. | ||
- | <!-- id:38 -->You can list the contents | + | Note 2: An empty metadata file in an import folder (including |
- | There should be six subdirectories: | + | |
- | * //etc// | + | |
- | * // | + | |
- | * // | + | |
- | * // | + | |
- | * //style// | + | |
- | ==== Add documents ==== | + | The main command for incremental rebuild is incremental-rebuild.pl. You can use this in place of full-rebuild.pl. |
- | <!-- id:39 -->Now we must populate the collection with sample documents. To do this, | + | ^Greenstone version^Windows^Linux/ |
- | we copy documents into the collections '' | + | |2|perl -S incremental-rebuild.pl |
- | '' | + | |3|perl |
- | <!-- id:40 --> | + | Indexer Note: only the Lucene |
- | and drag them into the '' | + | |
- | <!-- id:41 -->Or, you can type the command | + | ===== Finer control of the build process ===== |
- | < | + | The build process actually consists of several stages: |
- | xcopy /s C: | + | * **importing** the original documents into greenstone' |
- | </ | + | |
+ | * **activating** the collection in the live library (if necessary) | ||
- | ==== Edit the Config file ==== | + | These stages can all be run separately. Note, the greenstone environment must be set up in any terminal window before you can run these commands. |
- | <!-- id:42 -->In the collection' | + | ==== Importing |
- | Any modifications that you can make in the GLI, can also be achieved by manually editing the | + | |
- | '' | + | |
- | e.g. Notepad or Wordpad, make changes and save it. You can learn more about the Collection configuration file [[en: | + | |
- | ==== Import | + | This is the process of converting the original documents, which might be a mixture of file types, into a standardised XML based format - the Greenstone archive format. Original source documents live in the import folder of a collection, while the archive documents live in the archives folder. |
+ | The command to import a collection is '' | ||
- | <!-- id:43 -->Now you are ready to “import” the collection. | + | ^Greenstone |
- | This is the process of bringing the documents into the Greenstone | + | |2|perl -S import.pl |
- | standardizing the document format, the way that metadata is specified, | + | |3|perl -S import.pl |
- | and the file structure in which the documents are stored. | + | |
- | Type '' | + | |
- | or view them [[script_options# | + | |
- | < | + | As before, you need to put in your own collection name, and change the site name if you are using a custom greenstone3 site. |
- | perl —S import.pl -site localsite dlpeople | + | |
- | </ | + | |
- | <!-- id:44 -->Don't worry about all the text that scrolls past—it' | + | Don't worry about all the text that scrolls past—it' |
- | the progress of the import. Note that you do not have to be in either the | + | |
- | //collect// or // | + | |
- | because | + | |
the necessary files are. | the necessary files are. | ||
- | ==== Build the collection ==== | + | === Incremental import |
- | <!-- id:49 -->The next phase is to “build” the collection, | + | You can run just the import phase incrementally, |
- | which creates all the indexes and files that make the collection work. | + | |
- | Type '' | + | ==== Building a collection ==== |
- | collection-building options, which are also listed [[script_options# | + | |
+ | The next phase is to “build” the collection, which creates all the indexes and databases | ||
+ | Type '' | ||
For now, stick to the defaults by typing | For now, stick to the defaults by typing | ||
- | <code> | + | ^Greenstone version^Build command^ |
- | perl —S buildcol.pl -site localsite | + | |2|perl -S buildcol.pl |
- | </code> | + | |3|perl -S buildcol.pl -site localsite <collname>| |
- | <!-- id:50 -->Again, don't worry about the “progress report” text that scrolls past. | + | Again, don't worry about the “progress report” text that scrolls past. |
==== Make the collection live ==== | ==== Make the collection live ==== | ||
Finally, we need to make the collection " | Finally, we need to make the collection " | ||
- | with the contents of the '' | + | with the contents of the '' |
- | In an explorer window (i.e. outside of the terminal) simply select | + | Running activate.pl |
- | the contents of the // | + | |
- | directory and drag them into the '' | + | |
- | <!-- id:53 --> | + | ^Greenstone version^Activate |
- | (and all its contents) by typing the command | + | |2|perl -S activate.pl |
- | <code> | + | |3|perl -S activate.pl -site localsite |
- | rd /s index # on Windows NT/2000 | + | |
- | deltree /Y index # on Windows 95/98 | + | |
- | </code> | + | |
- | <!-- id:54 -->and then change | + | Or manually: |
+ | Delete | ||
+ | Note, the collection lives in the following location: | ||
- | <code> | + | ^Greenstone version^Collection location^ |
- | ren building index | + | |2|path-to-greenstone2/ |
- | </code> | + | |3|path-to-greenstone3/web/ |
- | It is important that these commands are issued from the correct directory | + | ==== Passing import/ |
- | (unlike | + | Import or buildcol options can be passed to full-rebuild and incremental-rebuild. If the option is shared between |
- | If the current working directory is not // | + | |
- | '' | + | |
- | '' | + | |
- | <!-- id:57 -->If your Greenstone server is already running, | + | ===== Creating and Editing a Collection on the command line ===== |
- | you should be able to access | + | |
- | from your Greenstone homepage. You will have to reload the page | + | |
- | if you already had it open in your browser, or perhaps even close | + | |
- | the browser and restart it (to prevent caching problems). | + | |
+ | ==== Create a collection ==== | ||
- | <!-- id:59 -->In summary then, the commands typed to produce | + | To create |
+ | the full list of options, which you can also view [[script_options# | ||
- | < | + | To create a new collection: |
- | cd C:\Users\jsmith\Greenstone3 # assuming default location | + | |
- | gs3-setup | + | |
- | perl —S mkcol.pl -site localsite —creator [email protected] dlpeople | + | |
- | cd %GSDL3HOME%\sites\localsite\collect\dlpeople | + | |
- | xcopy /s C: | + | |
- | perl —S import.pl -site localsite dlpeople | + | |
- | perl —S buildcol.pl -site localsite dlpeople | + | |
- | rd /s index # on Windows NT/2000 | + | |
- | deltree /Y index # on Windows 95/98 | + | |
- | ren building index | + | |
- | </ | + | |
- | <!-- LINUX ################################################################################### | + | ^Greenstone version^mkcol command^ |
+ | |2|perl -S mkcol.pl [options] | ||
+ | |3|perl | ||
- | =====MacOSX/Linux ===== | + | For example, to create a collection named //dlpeople// in '' |
+ | with the creator' | ||
- | Running | + | ^Greenstone |
- | on a Windows. Some of the commands are just a bit different. First change into the directory where Greenstone has been installed. | + | |2|perl -S mkcol.pl -creator me@cs.waikato.ac.nz < |
- | For example, if Greenstone is installed under its default name | + | |3|perl -S mkcol.pl -site localsite -creator [email protected] < |
- | at the top level of your user account you can move there by typing | + | |
- | < | ||
- | cd / | ||
- | </ | ||
- | \\ | ||
- | To set up the Greenstone environment: | ||
- | < | ||
- | source ./ | ||
- | </ | ||
- | \\ | + | //(Since Greenstone3 allows you to have multiple |
- | To create a collection: | + | |
- | < | + | |
- | mkcol.pl -site localsite —creator [email protected] dlpeople | + | |
- | </code> | + | |
- | \\ | + | |
- | To move to the newly created | + | |
- | collection directory: | + | |
- | < | + | |
- | cd $GSDL3HOME/ | + | |
- | </ | + | |
- | \\ | + | |
- | You can list the contents of this directory by typing '' | + | |
- | You can open and edit this using your favorite text editor — emacs is a popular editor on Linux. | + | |
+ | To view the newly created files, move to the newly created | ||
+ | collection directory by typing | ||
- | To copy the contents of the '' | + | ^Greenstone version^Windows^Linux/Mac^ |
- | directory into the '' | + | |2|cd %GSDL3HOME%\collect\dlpeople|cd $GSDL3HOME/collect/dlpeople| |
- | < | + | |3|cd %GSDL3HOME%\sites\localsite\collect\dlpeople|cd $GSDL3HOME/sites/localsite/collect/dlpeople| |
- | cp —r /home/documents/ | + | |
- | </ | + | |
- | \\ | + | |
- | To “import” the collection: | ||
- | < | ||
- | import.pl -site localsite dlpeople | ||
- | </ | ||
- | \\ | ||
- | |||
- | Next, “build” the collection: | ||
- | < | ||
- | buildcol.pl -site localsite dlpeople | ||
- | </ | ||
- | \\ | ||
- | |||
- | Finally, make the collection “live” by putting | ||
- | all the material that has just been put in the collection' | ||
- | // | ||
- | remove the old index: | ||
- | < | ||
- | rm —r index/* | ||
- | </ | ||
- | //(assuming you are in the '' | ||
- | |||
- | \\ | ||
- | And move the building directory to index: | ||
- | < | ||
- | mv building/* index/ | ||
- | </ | ||
- | |||
- | \\ | ||
- | In summary then, the commands typed to produced the // | ||
- | < | ||
- | cd / | ||
- | source ./ | ||
- | mkcol.pl —creator [email protected] dlpeople | ||
- | cd $GSDL3HOME/ | ||
- | cp —r / | ||
- | import.pl -site localsite dlpeople | ||
- | buildcol.pl -site localsite dlpeople | ||
- | rm -r index/* | ||
- | mv building/* index | ||
- | </ | ||
- | |||
- | |||
- | |||
- | ===== Additional Resources ===== | ||
- | |||
- | While this page only goes through the basics of building collections, | ||
- | are many other scripts that can be run from the command line | ||
- | (like [[en: | ||
- | the [[script_options|scripts and their options]] to get an idea of what else is available. | ||
- | </ | ||
- | <!-- ############################################################################################# | ||
- | ############################################################################################# | ||
- | ############################################################################################# | ||
- | <TAB> | ||
- | |||
- | ===== Windows ===== | ||
- | ==== Open a terminal ==== | ||
- | On Windows, there are several different ways to open a DOS terminal (a black console screen known as the DOS Prompt). Do one of the following: | ||
- | * '' | ||
- | * Under the Start menu, type '' | ||
- | * Hold down your keyboard' | ||
- | * In any Windows Explorer, hold down Shift and right click in an empty area in the window. Select '' | ||
- | |||
- | |||
- | ==== Setup the Environment ==== | ||
- | |||
- | In order to build collections in Greenstone (or run any other Greenstone scripts from the | ||
- | command line), you must first setup the terminal' | ||
- | change into the directory where Greenstone has been installed. | ||
- | Assuming Greenstone was installed in its default location (and your username is " | ||
- | |||
- | < | ||
- | cd C: | ||
- | </ | ||
- | //**Note** if the path to your Greenstone installation includes spaces (e.g. Program Files), you **must** | ||
- | put quotations around the path. For example: '' | ||
- | |||
- | Next, at the prompt type: | ||
- | |||
- | < | ||
- | setup | ||
- | </ | ||
- | |||
- | This batch file (which you can read if you like) tells the system where to look for Greenstone programs. | ||
- | |||
- | //Note: On Windows 95/98 systems running '' | ||
- | |||
- | If, later on in your interactive session at the DOS prompt, | ||
- | you wish to return to the top level Greenstone directory you can accomplish this by typing | ||
- | '' | ||
- | |||
- | **//If you close your DOS window and start another one, you will need to invoke '' | ||
- | |||
- | <!-- id:33 -->Now you are in a position to make, build and rebuild collections. | ||
- | |||
- | ==== Create a collection ==== | ||
- | |||
- | The first program we will look at is the Perl program '' | ||
- | whose name stands for “make a collection.” Typing '' | ||
- | the full list of options, which you can also view [[script_options# | ||
- | |||
- | //(If your Windows environment is set up to associate the Perl application with | ||
- | files ending in '' | ||
- | |||
- | To create a new collection: | ||
- | < | ||
- | perl -S mkcol.pl [options] collection-name | ||
- | </ | ||
- | |||
- | <!-- id:34 -->For example, to create a collection named // | ||
- | with the creator' | ||
- | |||
- | < | ||
- | perl —S mkcol.pl —creator [email protected] dlpeople | ||
- | </ | ||
- | \\ | ||
- | //Please substitute your email address for mine!// | ||
- | |||
- | <!-- id:36 -->To view the newly created files, move to the newly created | ||
- | collection directory by typing | ||
- | |||
- | < | ||
- | cd %GSDLHOME%\collect\dlpeople | ||
- | </ | ||
+ | You can list the contents of this directory by typing '' | ||
+ | There should be several subdirectories, | ||
- | <!-- id:38 -->You can list the contents of this directory by typing //dir//. | ||
- | There should be six subdirectories: | ||
* //etc// | * //etc// | ||
* //images// | * //images// | ||
* //import// | * //import// | ||
- | * //macros// | + | * // |
* //script// | * //script// | ||
* //style// | * //style// | ||
- | ==== Add documents ==== | + | ==== Add documents |
- | <!-- id:39 -->Now we must populate | + | To add documents into the collection, |
- | we copy documents | + | |
- | '' | + | |
- | + | ||
- | <!-- id:40 --> | + | |
- | and drag them into the '' | + | |
- | + | ||
- | <!-- id:41 -->Or, you can type the command | + | |
- | + | ||
- | < | + | |
- | xcopy /s C:\Users\jsmith\dldocuments\* import | + | |
- | </ | + | |
==== Edit the Config file ==== | ==== Edit the Config file ==== | ||
- | <!-- id:42 -->In the collection' | + | In the collection' |
- | Open it using your favorite text editor, e.g. Notepad or Wordpad. Any modifications that you | + | Any modifications that you can make in the GLI, can also be achieved by manually editing this file. Simply open it using your favorite text editor, |
- | can make in the GLI, can also be achieved by manually editing this | + | e.g. Notepad or Wordpad, make changes and save it. You can learn more about the Collection configuration file [[en:user:configuration files# |
- | collection configuration | + | |
- | e.g. Notepad or Wordpad, make changes and save it. | + | |
- | You can learn more about the Collection configuration file [[configuration files# | + | |
- | ==== Build the collection | + | ==== Build the Collection |
- | Building a collection | + | Now you can build the collection |
- | standardizing the document format, the way that metadata is specified, | + | |
- | and the file structure | + | |
- | The building stage generates the indexes, databases and other auxiliary files that are needed to make the collection work in Greenstone. | + | |
- | These processes can be run separately, or, in later Greenstone versions, a single script can be run which invokes both processes (see [[# | + | ===== Additional information ===== |
- | === Importing | + | ==== Opening a terminal on Windows ==== |
- | <!-- id:43 --> | + | |
- | Type //perl —S import.pl// at the prompt to get a list of all the options for the import program, | + | |
- | or view them [[script_options# | + | |
- | < | + | On Windows, there are several different ways to open a DOS terminal (a black console screen known as the DOS Prompt). Do one of the following: |
- | perl —S import.pl dlpeople | + | |
- | </ | + | * Under the Start menu, type '' |
- | + | * Hold down your keyboard's Windows | |
- | <!-- id:44 --> | + | * In any Windows Explorer, hold down Shift and right click in an empty area in the window. Select |
- | the progress of the import. Note that you do not have to be in either the | + | |
- | //collect// or // | + | |
- | | + | |
- | the necessary files are. | + | |
- | + | ||
- | === Building === | + | |
- | + | ||
- | <!-- id:49 --> | + | |
- | Type '' | + | |
- | | + | |
- | For now, stick to the defaults by typing: | + | |
- | + | ||
- | < | + | |
- | perl —S buildcol.pl dlpeople | + | |
- | </ | + | |
- | + | ||
- | <!-- id:50 --> | + | |
- | + | ||
- | === Make the collection live === | + | |
- | + | ||
- | Finally, we need to make the collection " | + | |
- | with the contents | + | |
- | + | ||
- | In an explorer window (i.e. outside of the terminal) simply select | + | |
- | the contents of the // | + | |
- | directory and drag them into the '' | + | |
- | + | ||
- | <!-- id:53 -->Alternatively, | + | |
- | (and all its contents) by typing the command | + | |
- | < | + | |
- | rd /s index # on Windows NT/2000 | + | |
- | deltree /Y index # on Windows 95/98 | + | |
- | </ | + | |
- | + | ||
- | <!-- id:54 -->and then change the name of the '' | + | |
- | + | ||
- | < | + | |
- | ren building index | + | |
- | </ | + | |
- | + | ||
- | It is important that these commands are issued from the correct directory | + | |
- | (unlike the Greenstone commands '' | + | |
- | If the current working directory is not // | + | |
- | '' | + | |
- | '' | + | |
- | + | ||
- | <!-- id:57 --> | + | |
- | from your Greenstone homepage. You will have to reload the page | + | |
- | if you already had it open in your browser, or perhaps even close | + | |
- | the browser and restart it (to prevent caching problems). Alternatively, | + | |
- | if you are using the “local library” version of Greenstone you | + | |
- | will have to restart the library program. | + | |
- | + | ||
- | ==== Build the collection in one easy step ==== | + | |
- | + | ||
- | An alternative to running import, then build, then deleting the old index and renaming building to index, is to run a single command, full-rebuild.pl. | + | |
- | + | ||
- | < | + | |
- | perl -S full-rebuild.pl dlpeople | + | |
- | </ | + | |
- | + | ||
- | This will run import.pl, buildcol.pl and then remove the old indexes and copy the new ones into the index folder. | + | |
- | + | ||
- | Import or buildcol options can be passed to full-rebuild. If the option is shared between import.pl and buildcol.pl then it can appear as is, such as -verbosity 5. This value will be passed to both programs. If an option is specific to one of the programs in particular, then prefix it with 'import:' | + | |
- | + | ||
- | Remember, you can run 'perl -S import.pl' | + | |
- | + | ||
- | ==== Summary ==== | + | |
- | <!-- id:59 -->In summary then, the commands typed to produce the // | + | |
- | + | ||
- | To set up the collection: | + | |
- | < | + | |
- | cd C: | + | |
- | setup.bat | + | |
- | perl —S mkcol.pl —creator [email protected] dlpeople | + | |
- | cd %GSDLHOME%\collect\dlpeople | + | |
- | xcopy /s d: | + | |
- | </ | + | |
- | + | ||
- | To build the collection: | + | |
- | < | + | |
- | perl -S full-rebuild.pl dlpeople | + | |
- | </ | + | |
- | or | + | |
- | < | + | |
- | perl —S import.pl dlpeople | + | |
- | perl —S buildcol.pl dlpeople | + | |
- | rd /s index # on Windows | + | |
- | deltree /Y index # on Windows 95/98 | + | |
- | ren building index | + | |
- | </ | + | |
- | + | ||
- | =====MacOSX/ | + | |
- | + | ||
- | Running Greenstone from the command line on MacOSX | + | |
- | on a Windows. Some of the commands are just a bit different. Please read through the Windows | + | |
- | + | ||
- | First change into the directory where Greenstone has been installed. | + | |
- | For example, if Greenstone | + | |
- | at the top level of your user account you can move there by typing | + | |
- | + | ||
- | < | + | |
- | cd / | + | |
- | </ | + | |
- | \\ | + | |
- | To set up the Greenstone environment: | + | |
- | < | + | |
- | source ./ | + | |
- | </ | + | |
- | //If you are unsure of the shell type you are using, enter '' | + | |
- | command-line prompt —it will print out the sought information. | + | |
- | If you are using a different shell contact your system administrator for advice.// | + | |
- | + | ||
- | \\ | + | |
- | To create a collection: | + | |
- | < | + | |
- | mkcol.pl —creator [email protected] dlpeople | + | |
- | </ | + | |
- | \\ | + | |
- | To move to the newly created | + | |
- | collection directory: | + | |
- | < | + | |
- | cd $GSDLHOME/ | + | |
- | </ | + | |
- | \\ | + | |
- | You can list the contents of this directory by typing '' | + | |
- | You can open and edit this using your favorite text editor — emacs is a popular editor on Linux. | + | |
- | + | ||
- | + | ||
- | To copy the contents of the ''/ | + | |
- | directory into the '' | + | |
- | < | + | |
- | cp —r / | + | |
- | </ | + | |
- | \\ | + | |
- | + | ||
- | To build the collection in one step: | + | |
- | < | + | |
- | full-rebuild.pl dlpeople | + | |
- | </ | + | |
- | + | ||
- | Or, to build it step by step manually: | + | |
- | + | ||
- | To “import” the collection: | + | |
- | < | + | |
- | import.pl dlpeople | + | |
- | </ | + | |
- | \\ | + | |
- | + | ||
- | Next, “build” the collection: | + | |
- | < | + | |
- | buildcol.pl dlpeople | + | |
- | </ | + | |
- | \\ | + | |
- | + | ||
- | Finally, make the collection “live” by putting | + | |
- | all the material that has just been put in the collection's | + | |
- | // | + | |
- | remove the old index: | + | |
- | < | + | |
- | rm —r index/* | + | |
- | </ | + | |
- | //(assuming you are in the '' | + | |
- | + | ||
- | \\ | + | |
- | And move the building directory to index: | + | |
- | < | + | |
- | mv building/* index/ | + | |
- | </ | + | |
- | + | ||
- | \\ | + | |
- | In summary then, the commands typed to produced the // | + | |
- | < | + | |
- | cd / | + | |
- | source ./ | + | |
- | mkcol.pl —creator [email protected] dlpeople | + | |
- | cd $GSDLHOME/ | + | |
- | cp —r / | + | |
- | </ | + | |
- | To build the collection: | + | |
- | < | + | |
- | full-rebuild.pl dlpeople | + | |
- | </ | + | |
- | or | + | |
- | < | + | |
- | import.pl dlpeople | + | |
- | buildcol.pl dlpeople | + | |
- | rm -r index/* | + | |
- | mv building/* index | + | |
- | </ | + | |
- | + | ||
- | + | ||
- | ===== Incremental Building ===== | + | |
- | + | ||
- | Incremental building is where you only process | + | |
- | + | ||
- | **Incremental importing**: | + | |
- | + | ||
- | Important note for collection design: Greenstone can notice that metadata | + | |
- | belongs to. Therefore, if metadata in a folder has changed (including new metadata being added), then all documents in that folder will be reimported. This means that if you have all your documents in the top level import folder, adding new metadata or changing any metadata for any document will result in all documents being reimported. If you intend to do incremental import, then please organize your documents into subfolders. That way modifying metadata for some documents won't result in all other documents being reimported. | + | |
- | + | ||
- | **Incremental indexing**: Currently only the Lucene indexer can do incremental indexing. If you are using MG/MGPP then a full buildcol pass will be done, even if incremental-buildcol.pl is used. | + | |
- | + | ||
- | If collection design has changed, then you will need to do a full rebuild. Changes to plugin options, and some import options will necessitate a full import. Changes to search indexes, partition indexes, browsing classifiers will necessitate a full buildcol. | + | |
- | + | ||
- | If you are doing incremnetal building, a full rebuild every now and then can be a good idea, in case something hasn't gone quite right in the incremental process. If you notice anything weird after an incremental build, then a full rebuild is a good idea then too. | + | |
- | + | ||
- | On the command | + | |
- | + | ||
- | Note that running incremental-buildcol.pl when you are not using Lucene for your indexer will be the same as running buildcol.pl. Without any -builddir option, incremental-buildcol.pl will do the indexing into the existing index directory, so you don't need to rename building to index. | + | |
===== Additional Resources ===== | ===== Additional Resources ===== | ||
Line 642: | Line 204: | ||
(like [[en: | (like [[en: | ||
the [[script_options|scripts and their options]] to get an idea of what else is available. | the [[script_options|scripts and their options]] to get an idea of what else is available. | ||
- | </ | ||
- | </ | ||
- | |||
en/user_advanced/command_line_building.txt · Last modified: 2023/03/13 01:46 by 127.0.0.1