Log In Sign Up

cnc-motorsports.com
robots.txt

Robots Exclusion Standard data for cnc-motorsports.com

Resource Scan

Scan Details

Site Domain cnc-motorsports.com
Base Domain cnc-motorsports.com
Scan Status Ok
Last Scan2024-06-12T06:17:18+00:00
Next Scan 2024-07-12T06:17:18+00:00

Last Scan

Scanned2024-06-12T06:17:18+00:00
URL https://cnc-motorsports.com/robots.txt
Domain IPs 104.26.6.100, 104.26.7.100, 172.67.68.155, 2606:4700:20::681a:664, 2606:4700:20::681a:764, 2606:4700:20::ac43:449b
Response IP 172.67.68.155
Found Yes
Hash ca9e13643687c21a220e3bd11e59ca0264046c861bf983833f8eddea7e7e950b
SimHash 41724f154d77

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60
Back to top

Comments

  • Enable robots.txt rules for all crawlers
  • Crawl-delay parameter: number of seconds to wait between successive requests to the same server.
  • Set a custom crawl rate if you're experiencing traffic problems with your server.
Back to top