Download a website with "wget".

[hack3rcon]

Hello,
I used below command to download a website completely but some parts like images on the "index.html" not work:

Code: Select all

# wget -m https://ctf101.org/

Any idea?

Thank you.

[kedaha]

Code: Select all

# wget -m https://ctf101.org/

Any idea?

Hi,
Not really but I think, for a start, it should be downloaded as normal user, not as root; otherwise there'll be permissions issues.

[andreathome]

In the past there were quite good tools that could download quite a big part of htm or html based websites.
Nowadays a lot of website are database driving, using CMS front tools.
In my humble opinion it is not possible to just simply download such CMS based websites, certainly not with simple tools.

[hack3rcon]

In the past there were quite good tools that could download quite a big part of htm or html based websites.
Nowadays a lot of website are database driving, using CMS front tools.
In my humble opinion it is not possible to just simply download such CMS based websites, certainly not with simple tools.

Any tools?

[oswaldkelso]

https://packages.debian.org/buster/httrack

[andreathome]

Also in this link:

HTTrack is an offline browser utility, allowing you to download a World Wide website from the Internet to a local directory, building recursively all directories, getting html, images, and other files from the server to your computer.

So no word there on CMS website which work completely different, with databases, sometimes even located elsewhere, so I think most of current websites cannot be downloaded with this tool, only fully classical htm/html based types.

[reinob]

Hello,
I used below command to download a website completely but some parts like images on the "index.html" not work:

Code: Select all

# wget -m https://ctf101.org/

Any idea?

Thank you.

I just tested with wget (check here: https://www.guyrutenberg.com/2014/05/02 ... sing-wget/) and it worked quite nicely. There may be some elements that are still downloaded (such as web fonts) but a quick test with firefox in offline mode shows the page just fine.

(You really really should read the manual of the tools you think are useful, as well as improve your google-fu.)

[andre@home]

For CMS based web sites as Joomla you need to tweak it to get a reasonable result:
https://vaasa.hacklab.fi/2013/11/28/how ... with-wget/
Needs quite some knowledge on how to convert it into the right way, how to apply the tweaks..