User-agent: * Disallow: /account/ Disallow: /errors/ #this stops Internet Archive (archive.org) robots archiving pages but others use same software to scrape sites User-agent: ia_archiver Disallow: / #others (inc can copy and archive all pages and show it on their sites as a "webpage resource" to "help" people User-agent: AboutUsBot Disallow: /