Site snapshots over time

Source Control should be able to resolve your request in the home. Be sure to label it correctly and have an internal server to deploy this shortcut and you shouldn't have a problem. If you have an auto-deploy tool and choose your labels wisely, it should be relatively straightforward to write an application that checks your origin on label X and deploys it only if the user enters a label. Now, if your tags we have are similar to the date, they just have to enter the date in the correct format and wait 5 minutes for deployment.

Similar to what others have suggested (assuming a dynamic website), I would use egress caching to generate the webpage code and then use Subversion to track changes.

Using a WayBack machine is probably only a last resort, such as if a user asks to view a web page before you install the system. You can't rely on everything you need at the WayBack Machine.

My suggestion was to just run wget through the site every night and store it on archive.yourdomain.com

. Add a control to each page for those with the appropriate permissions that pass the url of the current page to the date picker. When the date is selected, upload archive.yourdomain.com/YYYYMMDD/original_url

.

Allowing users to browse the entire site without broken links to archive.yourdomain.com

, might need to rewrite or copy the URL to rewrite or rewrite the archived copy of the site from some repository to the root archive.yourdomain.com

. For saving disk space, this may be the best option. Keep wget

copies encrypted then retrieve the date the user asks for. There are some problems with this, like how you deal with multiple users who want to view multiple archive pages at the same time from different dates, etc.

I suggest running wget

your site every night out of the original control, as you would end up with a page as shown to WWW visitors, complete with any dynamically served content, bugs, random rotated ads, etc.

EDIT: You can keep the output wget

in the original control, I'm not sure if you would buy it by pinning it to the filesystem somewhere outside of the source control. Also note that this plan will use large amounts of disk space over time, assuming a website of any size.

As Grant says, you can combine wget with version control to save space. I'm actually trying to write a script to do this for normal browsing, since I don't trust the internet archive or website indefinitely (and they're not very searchable).

The script will look something like this: cd to directory; call the correct command wget --mirror

or whatever; run darcs add $(find .)

to check out any new files in the repository; then darcs record --all

.

Wget should overwrite any modified files with an updated version; darcs add writes any new files / directories; darcs entry will save the changes.

To get a view from date X, you simply pull from your repo all revisions up to date X.

You don't keep unlimited duplicate copies because DVCS do not keep history unless there are actual changes to the file's content. You will end up with "garbage" in the sense of modifying pages to no longer require CSS or JS or images that you previously loaded, but you can just periodically delete everything and write it as a patch, and the next wget call will only pull what is needed for the latest version of the web page. (And you can still do full-text searches, you are now viewing history, not files on disk.)

(If large media files are being downloaded, you can do something like rm $(find . -size +2M)

to remove them before they get darcs add

ed.)

EDIT: I didn't bother with explicit version control, but let wget create duplicates and sometimes weed them out with fdupes

. See http://www.gwern.net/Archiving%20URLs

WayBackMachine can help.