Wget command is a Linux utility that used to download the files from the web. Basically, we are using web borwser to download file. I will cover wget command in linux with examples in this article.
You can use wget command, for now, to download the files from web servers using HTTP, HTTPS, and FTP protocols. Wget command work similarly to curl command in Linux.
Wget package is a freely available package under GNU GPL License.
You can install wget utility any Linux/Unix based distribution including windows and MAC OS.
It is a non-interactive command-line tool. You don’t need to have eyes open and stare at the screen at the time of downloading files.
If your downloading process stucked due network issue wget command automatically strat download where it was left off.
You would not trust but you can be mirroring complete website from the webserver on your system, you can enjoy using the website offline.
Let’s have some interesting discussion with examples of Wget command, Are you ready?? Go ahead…
How to use wget command to download a file
When you use wget command without any option, wget will download the resource mentioned in the URL to current directory. The download file will be saved in the current directory. You must mention the URL after wget command.
In the following example, I am downloading the TeamViewer tool.
[root@localhost ~]# wget https://download.teamviewer.com/download/linux/teamviewer.x86_64.rpm --2020-05-03 13:20:16-- https://download.teamviewer.com/download/linux/teamviewer.x86_64.rpm Resolving download.teamviewer.com (download.teamviewer.com)... 104.16.63.16, 104.16.62.16, 2606:4700::6810:3f10, ... Connecting to download.teamviewer.com (download.teamviewer.com)|104.16.63.16|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://dl.teamviewer.com/download/linux/version_15x/teamviewer_15.5.3.x86_64.rpm [following] --2020-05-03 13:20:17-- https://dl.teamviewer.com/download/linux/version_15x/teamviewer_15.5.3.x86_64.rpm Resolving dl.teamviewer.com (dl.teamviewer.com)... 104.16.63.16, 104.16.62.16, 2606:4700::6810:3e10, ... Connecting to dl.teamviewer.com (dl.teamviewer.com)|104.16.63.16|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 14565386 (14M) [application/x-redhat-package-manager] Saving to: ‘teamviewer.x86_64.rpm’ teamviewer.x86_64.r 100%[===================>] 13.89M 354KB/s in 2m 58s 2020-05-03 13:23:17 (79.9 KB/s) - ‘teamviewer.x86_64.rpm’ saved [14565386/14565386] [root@localhost ~]#
As you can see in the above example, First, Wget resolves the IP address of the domain, then connects to the remote server and starts the transfer data from a remote server to the local computer.
Also, you can see the progress bar alongside with the file name during donwloding file, file size, download speed, and the estimated time to complete the download as per current data transfer speed.
Once the download is complete, you can find the downloaded file in your current working directory.
You can use the ls command to view the downloaded file in the current directory.
[root@localhost ~]# ls anaconda-ks.cfg leafpad-0.8.9-2.fc8.rf.x86_64.rpm initial-setup-ks.cfg teamviewer.x86_64.rpm [root@localhost ~]#
If the file already exists with the same name, Wget will add .N (number) at the end of the file name. As you have seen in other downloading software.
Using Wget Command to Download a File to a Specific Directory
As you have seen in above section, By default, Wget will save the downloaded file in the current working directory.
If you want to save the file in to a different location or specific location, then use the -P option followed by specific path of the locaton. Where you want to save download file.
The syntax will be as below:
wget -P /opt/ http://centos.excellmedia.net/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.torrent
With the command above we are telling Wget to save the CentOS 8 torrent file into /opt directory.
[root@localhost ~]# wget -P /opt/ http://centos.excellmedia.net/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.torrent --2020-05-05 18:59:33-- http://centos.excellmedia.net/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.torrent Resolving centos.excellmedia.net (centos.excellmedia.net)... 175.101.1.2, 202.153.32.16 Connecting to centos.excellmedia.net (centos.excellmedia.net)|175.101.1.2|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 288772 (282K) [application/x-bittorrent] Saving to: ‘/opt/CentOS-8.1.1911-x86_64-dvd1.torrent’ CentOS-8.1.1911-x86 100%[===================>] 282.00K 235KB/s in 1.2s 2020-05-05 18:59:35 (235 KB/s) - ‘/opt/CentOS-8.1.1911-x86_64-dvd1.torrent’ saved [288772/288772] [root@localhost ~]#
Use wget command to download file and save with different name
You can use wget command followed -O (uppercase) option and NewFileName before putting the resource address. The file will be downloaded and save with different file names. The basic syntax will be as bellow:
wget -O newfilename.extension https://resource_address/currentfile.extension
In the follwoing example:
[root@localhost ~]# wget -O centOS-8-64bit.torrent /opt/ http://centos.excellmedia.net/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.torrent /opt/: Scheme missing. --2020-05-05 19:01:41-- http://centos.excellmedia.net/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.torrent Resolving centos.excellmedia.net (centos.excellmedia.net)... 202.153.32.16, 175.101.1.2 Connecting to centos.excellmedia.net (centos.excellmedia.net)|202.153.32.16|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 288772 (282K) [application/x-bittorrent] Saving to: ‘centOS-8-64bit.torrent’ centOS-8-64bit.torr 100%[===================>] 282.00K 442KB/s in 0.6s 2020-05-05 19:01:42 (442 KB/s) - ‘centOS-8-64bit.torrent’ saved [288772/288772] FINISHED --2020-05-05 19:01:42-- Total wall clock time: 0.9s Downloaded: 1 files, 282K in 0.6s (442 KB/s) [root@localhost ~]#
Use ls command to view downloaded file as follow:
[root@localhost ~]# ls anaconda-ks.cfg leafpad-0.8.9-2.fc8.rf.x86_64.rpm centOS-8-64bit.torrent teamviewer.x86_64.rpm initial-setup-ks.cfg [root@localhost ~]#
Resume incomplete or interrupted download by wget command
In case, you are downloading a big file, it may happen sometime to interrupt the downloading process due to the network error or something else.
And in this case, you can resume downloading the same file where it was left off.
You can use wget command followed by -c option to resume uncompleted download.
But when you start download file without specifying -c option wget will add .1 extension at the end of the file, considering as a fresh download.
So, you will find -c switch very helpful and it’s recommended when you download big files.
Basic syntax as follows:
$wget -c file_source_address.
Set downloading speed limit by wget command
Its amazing feature is inbuilt with wget command. You can set the limit of the download speed, use the –limit-rate option.
By default, the data transfer (downloading) speed is measured in bytes/second. You can append k for kilobytes, m for megabytes, and g for gigabytes.
This option is helpful if you don’t want to share all bandwidth with wget. In the other hand you don’t want wget to consume all the available bandwidth even you have some other work.
In the following example, wget command will download the file with limit the download speed to 1mb, You can replace ‘1m’ whatever you like.
wget –limit-rate=1m http://centos.excellmedia.net/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.iso
Send download in Background with wget command
When you are donwloading a big file, it is not possible to open terminal and stare on it untill completing download file. Is it possible. In mine words it is not possible.
So better idea send this downloading process in background. You can continue your work wihtout interruption.
You can use -b option to download in the background. In the following example, I am downloading the CentOS 8 iso file in the background.
$wget -b http://centos.excellmedia.net/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.iso
By default, the output is redirected to wget-log file in the current directory. To view the status of the curent download, then use the tail command as follows:
tail -f wget-log
Use wget command to download multiple files
Basically, you can specify multiple files one by one after wget command. You must give space between the files. So, the syntax will be as below.
$wget https://file_resource/file1.gz https://file_resource/file2.gz https://file_resource/file3.gz
But this is not a better idea. Every file has a long source address, You assume easily the length of 3 files. But If I am going to download 10 files by a single command, It is unbelievable to use in a single command.
Here is another better method, If you want to download multiple files at once, Then first create a .txt file and write down every file source address one by one. Each URL needs to be on a separate line.
After you must use the -i option followed by the path to a local or external file containing a list of the URLs to be downloaded.
In the following example I am going to download the Arch Linux, Debian, and Fedora iso files with URLs specified in the linux-distros.txt file:
wget -i linux-distros.txt
Download via FTP by using wget command
If you are downloading file via anonymous FTP, It does’t require FTP logins to download the files. So you can use basic syntax to download file via FTP. See the example below:
$wget ftp://ftp.example.com/file.tar.gz
In another case, if you want to download a file from a password-protected FTP server, specify the username and password as shown in below example:
$wget –ftp-user=FTP_USERNAME –ftp-password=FTP_PASSWORD ftp://ftp.example.com/filename.tar.gz
Mirror a Website using wget command
This feature will surprise you because lots of tech guys are using httrack to copy a website for testing purposes. But you can use wget command in Linux to perform this amazing task.
If you want to create a mirror of a website with Wget command, then add -m option with the command.
This will create a complete local copy of the website by following and downloading all internal links as well as the website resources including JavaScript, CSS, Images.
So syntax will be as below:
wget -m https://example.com
But If you want a perfect feel by local browsing of the downloaded website, then you will need to use a few extra arguments with get command.
It would be -k and -p, and the syntax will be as follows:
wget -m -k -p https://example.com
The -k option will cause Wget to convert the links in the downloaded documents to make them suitable for local viewing. The -p option will tell wget to download all necessary files for displaying the HTML page
Conclusion
I’ve tried my best to cover most of the basic uses of wget command in Linux to download files over HTTP, https, and FTP protocols.
For more detailed information on wget, you can check the manual page of this command. To display the manual page use man command from the terminal.
If I’ve missed any important command, please do share it with me via the comment section.