Downloading Files with wget

Downloading Files with wget

The wget command can download files from web servers (HTTP and HTTPS) and FTP servers. With a server that doesn’t require authentication, a wget command can be as simple as the wget command and the location of the download file:

$ wget http://releases.ubuntu.mirror.dkm.cz/releases/10.04.1/ubuntu-10.04.1-desktop-i386.iso
--------------------------------------------------------------------------------------------------
If you have ever had a large file download (DVD image file) disconnect before it completed, you may find the -c option to wget to be a lifesaver. Using
-c, wget resumes where it left off, continuing an interrupted file download. For example:

$ wget http://releases.ubuntu.mirror.dkm.cz/releases/10.04.1/ubuntu-10.04.1-desktop-i386.iso

Begin downloading large file
...
21% [=======> ] 155,208,064 292K/s eta 25m 27s
Download killed before completion
--------------------------------------------------------------------------------------------------
Resume download where stopped

$ wget -c http://releases.ubuntu.mirror.dkm.cz/releases/10.04.1/ubuntu-10.04.1-desktop-i386.iso
--------------------------------------------------------------------------------------------------
If, for example, an FTP server requires a login and password, you can enter that information on the wget command line in the following forms:

$ wget ftp://user:password@ftp.ubuntu.com/path/to/file
$ wget --user=user --password=password ftp://ftp.ubuntu.com/path/to/file
--------------------------------------------------------------------------------------------------
You can use wget to download a single web page as follows:

$ wget http://www.ubuntu.com
--------------------------------------------------------------------------------------------------
Download only the Web page
If you open the resulting index.html, you’ll have all sorts of broken links. To download all the images and other elements required to render the page properly, use the
-p option:

$ wget -p http://www.ubuntu.com
--------------------------------------------------------------------------------------------------
Download Web page and other elements
But if you open the resulting index.html in your browser, chances are you will still
have all the broken links even though all the images were downloaded. That’s because
the links need to be translated to point to your local files. So instead, do this:

$ wget -pk http://www.ubuntu.com
--------------------------------------------------------------------------------------------------
Download pages and use local file names
And if you’d like wget to keep the original file and also do the translation, type this:

$ wget -pkK http://www.ubuntu.com
--------------------------------------------------------------------------------------------------
Rename to local names, keep original
Sometimes an HTML file you download does not have an.html extension, but ends
in .asp or .cgi instead. That may result in your browser not knowing how to open
your local copy of the file. You can have wget append .html to those files using the
-E option:

$ wget -E http://www.ubuntu.com
--------------------------------------------------------------------------------------------------
Append .html to downloaded files
With the wget command, you can recursively mirror an entire web site. While copying
files and directories for the entire depth of the server’s file structure, the -m option adds
timestamping and keeps FTP directory listings. (Use this with caution, because it can
take a lot of time and space.)

$ wget -m http://www.ubuntu.com
--------------------------------------------------------------------------------------------------
Using some of the options just described, the following command line results in the
most usable local copy of a web site:

$ wget -mEkK http://www.ubuntu.com
--------------------------------------------------------------------------------------------------