User-agent: * Disallow: /test.html/ Disallow: /test1.html/ Disallow: /cgi-bin/ Disallow: /nav/ # updated 2002-03-22 (created disallow links)