# Dont disallow anything from Sharepoint crawler User-agent: Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+NT;+MS+Search+4.0+Robot)+Microsoft Disallow: # disallow the following from everyone else #User-agent: * #Disallow: /NR/rdonlyres/ #Disallow: /en-GB/Judicial+Decisions/