wget


With a handful of links to image files that needed downloading, I decided to see if there was a better way to download all the files rather than manually load them all up and save them - in fact, I was positive there was an easier way. Lo and behold, this easier way was called wget.

I created a download.txt file by copying out the rows of links and pasting it in a new text file, which ensured that one link was on each line of the file. Then I ran the following command:

    % wget -i download.txt

But I kept getting a 404 error, as the command was reading all the links in the file as a single link. I thought I was going crazy as I feverishly researched how I should have formatted the download.txt file - it would have been easier to just download them all individually at this point.

Turns out if you copy and paste right out of Microsoft Excel into a plain text editor, for some reason it must copy in some hidden characters. I’ve never had an issue using this editor in stripping formatting from text, but in this case it eluded me. Removing those characters somehow, the command then worked.

But left me with a list of files like this:

[...]
01.jpg.6
01.jpg.5
01.jpg.4
01.jpg.3
01.jpg.2
01.jpg.1
01.jpg

Due to the original filenames of each link, which were all ‘01.jpg’. Who would do that?

Unfortunately I had a deadline to meet and needed to rename the files to something more unique anyway so didn’t spend the time trying to understand what flags or additional syntax to use to rename the files. An activity for another time.

Nice wget resource here: ChristophChamp’s Wiki, StackOverflow, wget cs curl on The Geek Stuff.

Would curl have been better for this? I don’t know.