Downloading files using Wget

Or how to use up a lot of disk space real quick…!

I used the following command to create a local copy of all the files needed to install the 32-bit version of Cygwin (with out the source code) onto my hard disk as I wanted to install it on an old machine that I don’t connect to the internet.

wget --mirror --reject '*-src.tar*' -nv \
ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/cygwin/x86/ \
-w 1 --random-wait --limit-rate=1536K

Explaniation

The command is broken into three lines with a continuation character at the end of the first two lines.

  • Line 1 – Invokes wget
    • ‑‑mirror
      Instructs wget to create a directory structure that mirrors the source.
    • ‑‑reject '*‑src.tar*' (optional)
      Tells wget not to download any files that match '*‑src.tar*', which in this case will exclude any source packages. To only download one type of file use ‑‑accept instead of ‑‑reject.
    • ‑nv
      Be less verbose which causes wget to just print out the URL and path name of each file downloaded.
  • Line 2 – The URL where the files should be downloaded from.
  • Line 3 (optional) – Tries to make wget behave nicely..
    • ‑w 5
      Causes wget to wait (for five seconds) between each download.
    • ‑‑random-wait
      Makes the actual length of the wait a random number of seconds between 0.5 and 1.5 times the specified delay.
    • ‑‑limit-rate=1536K
      Tell wget to attempt to limit the bandwidth used.

This entry was posted in Linux and tagged , . Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.