User-agent: * Disallow: /not-published/ Disallow: /html files/ Disallow: /$Updated Files/ Disallow: /js/ Disallow: /flashdata/ Disallow: /framework/ Disallow: /css/ Disallow: /main/ Disallow: /video/ Disallow: /old files/ Disallow: /common/