# # This is a robots.txt file that can be used to indicate to # WWW robots which parts of a web site should not be indexed. # # Following page has details on how to setup a robots.txt file and # related security issues: # # http://www.chami.com/tips/internet/010198I.html # # 0. Sitemap - Indicate full URL of XML Sitemap location # Sitemap: http://www.csmd.edu/sitemap.xml Sitemap: http://www.csmd.edu/clc/sitemap.xml Sitemap: http://www.csmd.edu/Foundation/sitemap.xml # 1. Uncomment following two lines by removing "#" at the beginning to # indicate that all robots should stay away from the /cgi-bin/ folder User-agent: * Disallow: /cgi-bin/ Disallow: /_private/ Disallow: /_themes/ Disallow: /_vti_cnf/ Disallow: /_vti_pvt/ Disallow: /_derived/ Disallow: /_private/ Disallow: /_fpclass/ Disallow: /_scripts/ Disallow: /acad/ Disallow: /bus/ Disallow: /Cert/ Disallow: /Connections/OLD/ Disallow: /continuingeducation/ Disallow: /continuingeducation/newsite/ Disallow: /cookies/ Disallow: /CareerServices/09 templates/ Disallow: /CareerServices/Redesign/ Disallow: /csmboard/ Disallow: /Current/StudentLife/ Disallow: /design/ Disallow: /DistanceLearning/ Disallow: /Express-services/ Disallow: /fad/ Disallow: /FamilyFun/ Disallow: /footer/ Disallow: /forms/ Disallow: /gwstart/ Disallow: /images/ Disallow: /Library/infolit/ Disallow: /Library/infolitdesign/ Disallow: /new Disallow: /OAR/ Disallow: /OAR_Surveys/ Disallow: /old/ Disallow: /OnlineExpressServices/ Disallow: /OnlineServicesDown/ Disallow: /OnlineServicesUp/ Disallow: /napftds/ Disallow: /PeriodicReview/ Disallow: /PHPsecinfo/ Disallow: /privacy/ Disallow: /Realize/ Disallow: /rideshare/ Disallow: /Social/ Disallow: /search/adminstuff/ Disallow: /StudentAdvising/ Disallow: /styles/ Disallow: /StyleGuide/ Disallow: /Surveys/ Disallow: /test/ Disallow: /Training/PersonalEnrichment/ Disallow: /Training/WorkforceDevelopment/ Disallow: /tmp/ Disallow: /Web/ Disallow: /Webct/ Disallow: /webmaster/ Disallow: /wfs/ Disallow: /ws/ Disallow: /_pri.css Disallow: /_vti_inf.html Disallow: /googleform.html Disallow: /GoogleRedirect.html Disallow: /hdr.html Disallow: /index1.html Disallow: /index2.html Disallow: /index_20090220.html Disallow: /index_20090120.html Disallow: /index_20090706.html Disallow: /index_20091223.html Disallow: /index_timed_spotlight.txt Disallow: /indexnew.html Disallow: /printpage.php Disallow: /quicklinkjump.php Disallow: /quicklinks_old.html Disallow: /photos_old.html Disallow: /redirect.html Disallow: /redirect_HTTP301.html Disallow: /redirect_HTTP302.html Disallow: /siteindex2.html Disallow: /topnav_20090825.html Disallow: /wdt-oldfiles.php Disallow: /wdt-details.php # 2. Following two lines indicate that robots named "WWW_Robot_Name" # should stay away from this site. #User-agent: WWW_Robot_Name #Disallow: / # 3. Following two lines indicate that robots named "WWW_Wanderer_Name" # should not index files in the /private/ folder. #User-agent: WWW_Wanderer_Name #Disallow: /private/