Wget Command in Linux with Examples

0 Shares

Wget is a command-line utility created by the GNU Project for downloading files from the web. The name is a combination of World Wide Web and the word get.

With the help of Wget you can download files using HTTP, HTTPS, FTP and SFTP protocols.

Wget is created in portable C and usable on any Unix system and provides a number of options allowing you to download multiple files, limit the bandwidth, resume downloads, recursive downloads, mirror a website, download in the background and much more.

This page explains how to use the wget command with examples and complete explanations of the most common options.

Table of Contents

Installing Wget

First, check whether the wget utility is already installed or not in your Linux operating system, using the following command.

## On Debian, Ubuntu and Mint ##

$ dpkg -l | grep wget

## On RHEL/CentOS/Fedora ##

$ rpm -q wget

Note: The wget package is pre-installed on most Linux distributions today.

If wget is not installed, you can easily install it using the package manager of your distro.

Installing Wget on Redhat, CentOS and Fedora

$ sudo yum install wget

Installing Wget on Debian and Ubuntu

$ sudo apt install wget

Wget Syntax

You must follow the syntax given below to use the wget command.

$ wget url
$ wget [options] [url]

options : wget options
url : URL of the file or directory you want to download or synchronize.

1. Download a File with Wget

The command will download a single file and store it in a current directory.

It also shows Download Progress, Date and Time, Size while downloading.

$ wget https://github.com/BoostIO/boost-releases/releases/download/v0.16.1/boostnote_0.16.1_amd64.deb

2. Saving the Downloaded File Under Different Name

To save the downloaded file under a different name, pass the -O option followed by the chosen name.

$ wget -O boostnote_latest.deb https://github.com/BoostIO/boost-releases/releases/download/v0.16.1/boostnote_0.16.1_amd64.deb

The command above will save the boostnote_0.16.1_amd64.deb file from GitHub as boostnote_latest.deb instead of its original name.

3. Download Files in Background

With -b option you can send download in the background immediately after download start and logs are written in wget.log file.

$ wget -b wget.log https://github.com/BoostIO/boost-releases/releases/download/v0.16.1/boostnote_0.16.1_amd64.deb

4. Downloading a File to a Specific Directory

By default, wget will save the downloaded file in the current working directory. To save the file to a specific location, use the -P option.

$ wget -P mydata/files/ https://github.com/BoostIO/boost-releases/releases/download/v0.16.1/boostnote_0.16.1_amd64.deb

The command above tells wget to save the file to the mydata/files/ directory.

5. Download Multiple Files From a File

To download multiple files at once, use the -i option with the location of the file that contains the list of URLs to be downloaded.

Each URL needs to be added on a separate line as shown. For example, the following file named download_files.txt contains the list of URLs to be downloaded.

$ cat download_files.txt 
https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-11.2.0-amd64-DVD-1.iso
https://github.com/gohugoio/hugo/archive/master.zip
https://releases.ubuntu.com/20.04.3/ubuntu-20.04.3-desktop-amd64.iso
https://github.com/BoostIO/boost-releases/releases/download/v0.16.1/boostnote_0.16.1_amd64.deb
https://download.rockylinux.org/pub/rocky/8/isos/x86_64/Rocky-8.5-x86_64-dvd1.iso
https://download.fedoraproject.org/pub/fedora/linux/releases/28/Server/x86_64/iso/Fedora-Server-dvd-x86_64-28-1.1.iso

$ wget -i download_files.txt

6. Skipping Certificate Check

If you want to download a file over HTTPS from a host that has an invalid SSL certificate, use the --no-check-certificate option.

$ wget --no-check-certificate https://download.fedoraproject.org/pub/fedora/linux/releases/28/Server/x86_64/iso/Fedora-Server-dvd-x86_64-28-1.1.iso

7. Mirror Entire Website

To create a mirror of a website with wget, use the -m option.

This will create a complete local copy of the website by following and downloading all internal links as well as the website resources (JavaScript, Images, CSS).

$ wget -m https://edumotivation.com

If you want to use the downloaded website for local browsing, you will need to pass a few extra arguments to the command above.

$ wget -m -k -p https://edumotivation.com

The -k option will cause wget to convert the links in the downloaded documents to make them suitable for local viewing.
The -p option will tell wget to download all necessary files for displaying the HTML page.

8. Resume Uncompleted Download

In case of large file download, it may happen sometimes to stop download.

In that case we can resume downloading the same file where it was left off with -c option but when you start downloading files without specifying -c an option wget will add .1 extension at the end of the file, considered as a fresh download.

So, it’s good practice to add -c switch when you download large files.

$ wget -c https://download.fedoraproject.org/pub/fedora/linux/releases/28/Server/x86_64/iso/Fedora-Server-dvd-x86_64-28-1.1.iso

9. Downloading via FTP (Password Protected)

To download a file from a password protected FTP server, specify the username and password as shown below.

$ wget --ftp-user=FTP_USERNAME --ftp-password=FTP_PASSWORD ftp://ftp.demo.com/file_name.tar.gz

10. Downloading via HTTP (Password Protected)

To download a file from a password protected HTTP server, you can use the options --http-user=username and --http-password=password as shown.

$ wget --http-user=helpdesk --http-password=pass@123 http://http.demo.com/file_name.tar.gz

11. Download Multiple Files with HTTP and FTP Protocol

Type the following command to download multiple files at once using HTTP and FTP protocol with the help of wget.

$ wget http://ftp.gnu.org/gnu/wget/wget2-2.0.0.tar.gz ftp://ftp.demo.com/file_name.tar.gz

12. Downloading to the Standard Output

In the following example, wget will quietly ( flag -q) download and output the latest WordPress version to stdout ( flag -O -) and pipe it to the tar utility, which will extract the archive to the /var/www directory.

$ wget -q -O - "http://wordpress.org/latest.tar.gz" | tar -xzf - -C /var/www

13. Set File Download Speed Limits

To limit the download speed, use the --limit-rate option.

By default, the speed is measured in bytes/second. With option --limit-rate=500k, the download speed limit is restricted to 500k and the logs will be created under wget.log as shown below.

$ wget -c --limit-rate=500k -b wget.log https://download.fedoraproject.org/pub/fedora/linux/releases/28/Server/x86_64/iso/Fedora-Server-dvd-x86_64-28-1.1.iso

14. Changing the Wget User-Agent

Sometimes when downloading a file, the remote server may be set to block the wget User-Agent.

In situations like this, to emulate a different browser, pass the -U option.

$ wget --user-agent="Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" http://wget-forbidden.com/

The above command will emulate Firefox 60 requesting the page from wget-forbidden.com.

Conclusion

I hope you have learned something from this article.

I have tried my best to include all the features of wget command in this guide.

Now I’d like to hear your thoughts.

Was this guide useful to you?

Or maybe you have some queries.

Have I not included any command in this guide?