User-agent: * Allow: /js Disallow: /bin/ Disallow: /client_scripts/ Disallow: /ClientBin/ Disallow: /aspnet_client/ Disallow: /App_Data/ Disallow: /emailScripts/ Disallow: /fckeditor/ Disallow: /ckeditor/ Disallow: /flash/ Disallow: /flashJs/ Disallow: /flashSQL/ Disallow: /xml/ Disallow: /ssl/ Disallow: /typography.aspx User-agent: EtaoSpider Disallow: / User-agent: baiduspider Disallow: / User-agent: BrandVerity Disallow: / User-agent: Seekport Crawler Disallow: / User-agent: SeekportBot Disallow: / User-agent: BLEXBot Crawl-delay: 30 User-agent: MJ12bot Crawl-delay: 60 User-agent: AhrefsBot Crawl-delay: 30 User-agent: AhrefsSiteAudit Crawl-delay: 30 User-agent: SemrushBot Crawl-delay: 30 User-agent: DataForSeoBot Crawl-delay: 30 User-agent: pricebot Crawl-delay: 30 User-agent: dotbot Crawl-delay: 30 User-agent: Charlotte Crawl-delay: 30 Request-rate: 1/30s User-agent: bingbot Crawl-delay: 1 User-agent: Pinterestbot Crawl-delay: 1 User-agent: SeznamBot Crawl-delay: 30 Request-rate: 1/30s