The Navigatrix has been updated. The new website can be found at navigatrix.net.




Post new topic Reply to topic  [ 1 post ] 
Author Message
 Post subject: Download Entire Websites In A Single Bound

Joined: 04 Nov 2010, 20:51
Posts: 1062
Say there is a website, or a part of a website, that you want to read...later.

For example, you are going back to the boat in the next 15 minutes and will set sail in the morning, but you want the well written and enjoyable Navigatrix Documentation to read when you have time, but maybe not an internet connection.

Download it and read it off the clock on your own time offline.

It's easy.
Code:
mkdir offline && cd offline
to make a directory to hold your treasure and change to it. (or wherever your hearts desire)
Code:
wget \
     -w 20 --random-wait \
     --recursive \
     --no-clobber \
     --page-requisites \
     --convert-links \
     --domains navigatrix.net \
     --no-parent  navigatrix.net/nxdoc/
This command downloads the Web site http://navigatrix.net/nxdoc.

The options are:

  • -w 20 --random-wait: to avoid getting banned for too many requests in too short a time. The number (20) is time in seconds. This can be reduced, or the line eliminated if you laugh in the face of danger.

  • --recursive: download the entire Web site from entry point on down.

  • --no-clobber: don't overwrite any existing files (used in case the download is interrupted and
    resumed).

  • --page-requisites: get all the elements that compose the page (images, CSS and so on).

  • --convert-links: convert links so that they work locally, off-line.

  • --domains navigatrix.net: don't follow links outside navigatrix.net.

  • --no-parent: don't follow links outside the directory nxdoc. and inadvertantly download the entire site
Fire away and it will
Quote:
FINISHED --2013-09-12 15:21:43--
Total wall clock time: 23s <----no random wait time; 22m 23s with random wait.
Downloaded: 65 files, 2.4M in 7.1s (347 KB/s)
Converting navigatrix.net/nxdoc/index.html... 22-0
Converting navigatrix.net/nxdoc/02_getting_nx.html... 9-0
Converting navigatrix.net/nxdoc/01_nx_what_why.html... 4-0
Converting navigatrix.net/nxdoc/04_applications.html... 17-0
Converting navigatrix.net/nxdoc/05_gps.html... 4-0
Converting navigatrix.net/nxdoc/06_config_nx.html... 38-0
Converting navigatrix.net/nxdoc/08_nx_support_db.html... 3-0
Converting navigatrix.net/nxdoc/03_installing_nx.html... 22-0
Converting navigatrix.net/nxdoc/07_hidden_treasures.html... 4-0
Converting navigatrix.net/nxdoc/nx_doc_stylesheet.css... 2-0
Converted 10 files in 0.003 seconds.
Then you can either navigate to the directory with the File Manager or
Code:
wadda@mini:~/offline$ firefox -new-window file:////home/$USER/offline/navigatrix.net/nxdoc/index.html
Then again you could just use the Scrapbook extension that is pre-installed in Firefox. It's just as effective, but not as much fun.


Top
   
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 1 post ] 


Search for:
cron

Credits © 2010 - 2024