====== Downloading from the command line======

Greenstone allows you to download files from the internet using a variety of protocols:
  * **[[en:filetype:web|Web]]:** downloads web pages and files via HTTP and FTP.
  * **[[en:filetype:mediawiki|MediaWiki]]:** downloads web pages and files via HTTP from a MediaWiki website. 
  * **[[en:user_advanced:oai|OAI]]:** downloads metadata records (and optionally documents) from an [[http://www.openarchives.org/pmh/|OAI-PMH (Open Archives Initiative)]] server. 
  * **[[en:user_advanced:z3950#Download through Z39.50|Z39.50]]:** downloads MARC records that match a particular search criterion from a Z39.50 server. 
  * **[[en:user_advanced:z3950#Download through SRU|SRW/SRU]]:** downloads MARCXML records that match a particular search criterion from a Search/Retrieve via URL (SRU) server. 

This can be done from either the [[en:gli:download_panel|Download panel]] of the GLI, or directly from the command line. 


<tabbox Greenstone3>
On Windows:
<code>
gs3-setup
</code>
\\
On Linux/Mac:
<code>
source gs3-setup.bash 
</code>
<tabbox Greenstone2>
On Windows:
<code>
setup
</code>
\\
On Linux/Mac:
<code>
source setup.bash 
</code>
</tabbox>


GLI uses a perl script, **downloadfrom.pl**, to download files. This can be run on the command line,
 outside of GLI. The following options are available for all 
methods of download (Web, MediaWiki, OAI, Z3950, and SRW) using ''downloadfrom.pl'':

^Option^Description^
|''-download_mode <enum>''|(REQUIRED) The type of server to download from; allowable values: ''Web'', ''MediaWiki'', ''OAI'', ''Z3950'', and ''SRW''|
|''-cache_dir <string>''|The location of the cache directory|
|''-gli''|   |
|''-info''|Print information about the server, rather than downloading|
|//This information is also available from the command line: ''perl -S downloadfrom.pl -h''//||

There are also several options available if you are using a proxy:

^Option^Description^
|''-proxy_on''|Indicates you are using a proxy connection|
|''-proxy_host <string>''|Proxy host|
|''-proxy_port <string>''|Proxy port|
|''user_name <string>''|Proxy username|
|''user_password <string>''|Proxy password|

Each download mode also has its own set of additional options, which are outlined on 
their respective pages in the documentation ([[en:filetype:web|Web]], [[en:filetype:mediawiki|MediaWiki]], [[en:user_advanced:oai|OAI]], [[en:user_advanced:z3950#Download through Z39.50|Z39.50]], [[en:user_advanced:z3950#Download through SRU|SRW/SRU]]) . These options are the same as 
those available on the GLI Download panel and can also be viewed by running 
''perl -S downloadinfo.pl <download-module>''. For example, to get information 
and options for downloading via OAI protocol, you would run:
<code>
perl -S downloadinfo.pl OAIDownload
</code>

The download modules names are:
  * WebDownload
  * MediaWikiDownload
  * OAIDownload
  * Z3950Download
  * SRWDownload (//for SRU/SRW downloads//)

Once aware of all of the options, you are able to run the download script. An example download would be:
<code>
 perl -S downloadfrom.pl -document_mode OAI -url http://www.nzdl.org/cgi-bin/oaiserver.cgi -set demo -max_records 5
</code>

This will try to download 5 records from the set //demo// at the nzdl.org's OAI server.

The records (and optionally documents) will be downloaded into the folder the script is run from. To change this, use the ''-cache_dir <full-path-to-folder>'' option.




===== Additional Resources =====
There are several tutorials on downloading files using various protocols:

<tabbox Greenstone3>
  * [[http://wiki.greenstone.org/wiki/gsdoc/tutorial/gs3-current/en/downloading_from_internet.htm|Downloading files from the web]]
  * [[http://wiki.greenstone.org/wiki/gsdoc/tutorial/gs3-current/en/OAI_downloading.htm|Downloading over OAI]]
<tabbox Greenstone2>
  * [[http://wiki.greenstone.org/wiki/gsdoc/tutorial/gs2-current/en/downloading_from_internet.htm|Downloading files from the web]]
  * [[http://wiki.greenstone.org/wiki/gsdoc/tutorial/gs2-current/en/OAI_downloading.htm|Downloading over OAI]]
</tabbox>