Download all files from a web directory which requires login ID and password by using command line (Linux).

Question

[eDNAuRNA]

I have a web directory which requires login and password for access. I need to download multiple data files from the web directory by using linux command line but I don't know how to get the web links to all files in the directory as well as how to use a list of web links in wget command. Any help will be appreciated.

Thanks

[GenoMax]

In addition to @Alex's solution below you may need to use -r and -A option to get all files (of a kind).

[cpad0112]

and -np and -nd

[eDNAuRNA]

Thanks a lot Alex.

For getting the list of links, can we have a solution in Linux?

[GenoMax]

Please use ADD COMMENT/ADD REPLY when responding to existing posts to keep threads logically organized. This comment belongs under @Alex's answer.

SUBMIT ANSWER is for new answers to original question.

Answer 1

[Alex Reynolds]

If the site uses basic authentication, you could use wget --user=<user> --password=<password> "<url>" to download the specified url.

The wget tool is for requests. It won't give you a list of links. You could use Python requests with scraping libraries: https://blog.hartleybrody.com/web-scraping-cheat-sheet/