How to transfer resources from your own CMS?
I need to migrate our site from a proprietary CMS that uses active server pages. Is there a tool or technique to help download assets from an existing site? I am probably looking for a tool that scans and cleans the entire site.
Another issue is that the site uses SSL and is protected by forms based authentication. I have the required credentials and can grab the cookie that validates the session, but I'm not sure where to go from here and I don't want to reinvent the wheel if existing tools can help me.
EDIT - I am using Windows OS
a source to share
wget can be a good tool to use
wget --load-cookies cookies.txt --mirror --page-requisites http://example.com/
add --convert-links if you want to make it more suitable for a local archive rather than something you can re-download somewhere.
Windows version of wget is available from the gnuwin32 project at sourceforge.net http://gnuwin32.sourceforge.net/packages/wget.htm
a source to share
wget --http-user:username --http-pass:password -r http://yoursite.com
This will fetch the entire site (recursively). If you are on windows you will want to install cygwin or something similar in order to use it, although I believe there are Windows versions / wget clones you can download.
a source to share
If you know Perl, you might like WWW :: Mechanize . Depending on the level of automation you are trying to achieve wget
, it will probably be very good for some cases.
a source to share
You have many options. Consider how complex authentication is. Besides wget you can look at curl (a very robust option with links for different languages), Python urllib , Apache HttpClient , WWW-Mechanize , etc.
a source to share