webCapturelog

Archive-It documentation provides directions for crawling password protected sites but notes that the technology remains "incompatible with log-in processes that require two-step authentication, split username and password fields across webpages, or require a CAPTCHA." The New School email is configured with G-Suite and all Google Docs created with a @newschool.edu account require 2-step verification. Our solution was to download the content and divide access to the site between two sources.

Two links to external #Vimeo videos provided another challenge. Though Archive-it should theoretically be able to handle these pages, the System Status section notes: "Archived Vimeo seeds might not replay in Wayback. We are working actively to resolve this."  

Capture Notes

Wayback Link https://wayback.archive-it.org/9699/20210107172350/https://www.newschool.edu/university-planning/

Length of Time 7 days

Data 8.1GB

Crawling Technology Archive-It, Standard

Media Google Docs, Vimeo links  

Capture Extent One page

Seed Scope Rules To prevent capture of the header and footer links, this was a one page crawl with a seed scope rule to accept URLs from blogs.newschool.edu—the only external links of interest on the page (aside from two Vimeo pages, which were added as separate seeds).

Remaining Questions