managing wget's server load?

Hi folks,

I want to create a static mirror of a dynamically generated website, using wget. My site has about 2,000 pages, and I'm concerned about putting an undue load on the DH server when wget spiders its way through the site. I don't want to annoy other shared users on my server.

Does anyone have any experience with this? Do you recommend using the -w ("wait") option to slow down wget? What's a reasonable wait value?

Many thanks!

- john

A more efficient approach would be to 'nice' the process. Check out the manpage for 'nice.'

-Scott

Thanks, sdayman.

I just checked out the man page for nice, but now I'm not sure what "niceness" value would be appropriate in my case. Would a niceness of +19 keep my neighbors happy, while still allowing wget to process all 2,000 pages in a reasonable time?

(I can experiment with this, of course; I just thought I could learn from the experience of others.)

- john

I'd go with +19. Everybody else will be happy, and maybe even you.

-Scott

Thanks, Scott. I'll start with +19 and see how it goes.

I appreciate your help.

- john