Download a website with "wget".

If none of the more specific forums is the right place to ask

Download a website with "wget".

Postby hack3rcon » 2019-08-24 06:47

Hello,
I used below command to download a website completely but some parts like images on the "index.html" not work:
Code: Select all
# wget -m https://ctf101.org/

Any idea?

Thank you.
hack3rcon
 
Posts: 298
Joined: 2015-02-16 09:54

Re: Download a website with "wget".

Postby kedaha » 2019-08-24 07:32

hack3rcon wrote:
Code: Select all
# wget -m https://ctf101.org/

Any idea?

Hi,
Not really :wink: but I think, for a start, it should be downloaded as normal user, not as root; otherwise there'll be permissions issues.
Mate DE & OSSv4.
FreedomBox in Debian
ispmail
Debian Stable

Words, as is well known, are the great foes of reality. Joseph Conrad.
Kedaha's Conjecture
User avatar
kedaha
 
Posts: 3017
Joined: 2008-05-24 12:26

Re: Download a website with "wget".

Postby andreathome » 2019-08-25 23:00

In the past there were quite good tools that could download quite a big part of htm or html based websites.
Nowadays a lot of website are database driving, using CMS front tools.
In my humble opinion it is not possible to just simply download such CMS based websites, certainly not with simple tools.
andreathome
 
Posts: 9
Joined: 2019-08-25 08:06

Re: Download a website with "wget".

Postby hack3rcon » 2019-08-26 05:32

andreathome wrote:In the past there were quite good tools that could download quite a big part of htm or html based websites.
Nowadays a lot of website are database driving, using CMS front tools.
In my humble opinion it is not possible to just simply download such CMS based websites, certainly not with simple tools.

Any tools?
hack3rcon
 
Posts: 298
Joined: 2015-02-16 09:54

Re: Download a website with "wget".

Postby oswaldkelso » 2019-08-26 13:08

Ash init durbatulûk, ash init gimbatul,
Ash init thrakatulûk agh burzum-ishi krimpatul.
User avatar
oswaldkelso
 
Posts: 1184
Joined: 2005-07-26 23:20
Location: UK

Re: Download a website with "wget".

Postby andreathome » 2019-09-01 19:15

Also in this link:
HTTrack is an offline browser utility, allowing you to download a World Wide website from the Internet to a local directory, building recursively all directories, getting html, images, and other files from the server to your computer.

So no word there on CMS website which work completely different, with databases, sometimes even located elsewhere, so I think most of current websites cannot be downloaded with this tool, only fully classical htm/html based types.
andreathome
 
Posts: 9
Joined: 2019-08-25 08:06

Re: Download a website with "wget".

Postby reinob » 2019-09-02 07:52

hack3rcon wrote:Hello,
I used below command to download a website completely but some parts like images on the "index.html" not work:
Code: Select all
# wget -m https://ctf101.org/

Any idea?

Thank you.


I just tested with wget (check here: https://www.guyrutenberg.com/2014/05/02 ... sing-wget/) and it worked quite nicely. There may be some elements that are still downloaded (such as web fonts) but a quick test with firefox in offline mode shows the page just fine.

(You really really should read the manual of the tools you think are useful, as well as improve your google-fu.)
reinob
 
Posts: 743
Joined: 2014-06-30 11:42

Re: Download a website with "wget".

Postby andre@home » 2019-09-02 19:29

For CMS based web sites as Joomla you need to tweak it to get a reasonable result:
https://vaasa.hacklab.fi/2013/11/28/how ... with-wget/
Needs quite some knowledge on how to convert it into the right way, how to apply the tweaks..
andre@home
 
Posts: 347
Joined: 2011-10-02 08:00


Return to General Questions

Who is online

Users browsing this forum: No registered users and 17 guests

fashionable