How to Download Files From The Web Using Wget [Linux CLI Tips]

In this post: a little about the WGET command:

Download a zip archive:

wget http://website.com/archive.zip

and archive.zip will be downloaded. But we can download files using a lot of parameters. Read on!

wget -r http://website.com

This download all files recursively: images, html files, etc. But this could get us banned by the server for sending too many download requests so to avoid this:

wget --random-wait --limit-rate=20k -r http://website.com

--random-wait means to download a file and then wait for a random period of time, then download the next file and so on.

--limit-rate=20k indicates that you want to download at a maximum speed of 20k so you don't get banned.

Or you could also do:

wget --wait=20 --limit-rate=20K -r -p -U Mozilla http://website.com

--wait=20 to wait 20 seconds between each file download, but I think it's better to download with --random-wait

-p indicates that the files should be displayed as HTML, as if you were actually looking at the page

-U Mozilla will make the website believe you are using a Mozilla browser.

And here is how to download all images, videos or whatever you want, from a website:

wget -r -A=.jpg,.png http://website.com

With this command, you download all jpg and png files from website.com. If you want to download all mp3s, then you would use -A=.mp3

You can also use a GUI for wget if you want. It's called Gwget and should be in your distribution repositories. For Ubuntu, do:

sudo apt-get install gwget

Credits: paraisolinux