# File: robots.txt for http://trs.mt.gov/ # Date: 7/11/2007 # Author: Bill Hallinan # # Note: Very important to excluded all robots except archive.org_bot. # Make sure with archive.org_bot that only public, not senstive directories are revealed. # Archive service for the State of Montana. Uses www.waybackmachine.org for revealing contents of # site. Be careful not to save or otherwise post information to web directories that archive.org_bot # will duplicate for universal browsing. User-agent: archive.org_bot Disallow: Disallow: /cgi-bin Disallow: /cgi-bin-test Disallow: /BugTracker Disallow: /BugTrackerUploads Disallow: /Help/TestApps Disallow: /Members/BenefitEstimator Disallow: /PhWebReports Disallow: /Test # For all other robots (except archive.org_bot) User-agent: * Disallow: /