TIL: Mirroring Sites With wget Recursive Mode

Internet goes down. Documentation disappears. wget -r downloads an entire site for offline access.

wget -r --level=2 https://developers.example.com/docs/

Downloads the docs directory recursively to a depth of 2 pages. No external links, no infinite crawling.

wget -r -np -nH --cut-dirs=1 https://developers.example.com/docs/

-np prevents following parent directories. -nH strips the hostname directory. –cut-dirs removes URL path segments from the local folder structure.

wget -r -l inf -k -p https://docs.example.com/

-k converts links to work locally. -p downloads every asset (CSS, images, JS). The result is a fully offline mirror that opens from the filesystem.

Is wget on macOS?

macOS ships with a BSD version of wget. Install the full GNU version with brew install wget.

What is the difference between wget and curl?

wget is built for recursive downloads. curl is built for single requests. Use wget for mirrors and curl for API calls.