# robots.txt for http://www.lofcom.com/ # MSIE 4.0 go away User-agent: MSIECrawler Disallow: / # go away User-agent: WiseWire Disallow: / # go away User-agent: netcraft Disallow: / # go away User-agent: grub-client Disallow: / # Alexa... User-agent: ia_archiver Disallow: / #Disallow: /archive/ #Disallow: /bin/ #Disallow: /cgibin/ #Disallow: /configuration/ #Disallow: /forms/ #Disallow: /incoming/ #Disallow: /lists/ #Disallow: /lists2/ #Disallow: /logs/ #Disallow: /packets/ #Disallow: /private/ #Disallow: /temp/ #Disallow: /userpackets/ #Disallow: /lof/ #Disallow: /personal/ #Disallow: /nostalgia/ # charlotte.betaspider.com - really hammering the Edwards site User-Agent: Charlotte Disallow: / # All others... User-agent: * Disallow: /archive/ Disallow: /bin/ Disallow: /cgibin/ Disallow: /configuration/ Disallow: /forms/ Disallow: /incoming/ Disallow: /lists/ Disallow: /lists2/ Disallow: /logs/ Disallow: /nostalgia/newshow.phtml Disallow: /nostalgia/eplogs.phtml Disallow: /nostalgia/whysschange.phtml Disallow: /nostalgia/xtarsearch.phtml Disallow: /packets/ Disallow: /private/ Disallow: /temp/ Disallow: /userpackets/