User-agent: * Disallow: /apps/ Disallow: /cgi-bin/ Disallow: /css/ Disallow: /custom/ Disallow: /favicon.ico Disallow: /images/ Disallow: /issues/cgi-bin/ Disallow: /javascript/ Disallow: /js/ Disallow: /newspics/ Disallow: /profile/ Disallow: /pub/cgi-bin/ Disallow: /scripts/ Disallow: /secure/ Disallow: /udt/ Crawl-delay: 10 User-agent: * Disallow: /*? Disallow: *cgi* Disallow: /cgi-bin Disallow: /issues/cgi-bin Disallow: /pub/cgi-bin Disallow: /script Disallow: /javascript Disallow: /images Disallow: /newspics Disallow: /profile Disallow: /css Disallow: /favicon.ico