Log In Sign Up

cyclevolta.com
robots.txt

Robots Exclusion Standard data for cyclevolta.com

Resource Scan

Scan Details

Site Domain cyclevolta.com
Base Domain cyclevolta.com
Scan Status Ok
Last Scan2024-11-16T13:11:17+00:00
Next Scan 2024-11-23T13:11:17+00:00

Last Scan

Scanned2024-11-16T13:11:17+00:00
URL https://cyclevolta.com/robots.txt
Redirect https://www.cyclevolta.com:443/robots.txt
Redirect Domain www.cyclevolta.com
Redirect Base cyclevolta.com
Domain IPs 15.197.174.213, 3.33.166.34
Redirect IPs 23.209.46.30, 23.209.46.31, 2600:1413:b000:13::b857:c18e, 2600:1413:b000:13::b857:c196
Response IP 42.99.140.217
Found Yes
Hash 496b663e43e0c5b0cf82a6f31a84835aef1a52910a423bf05300bb5ed8b66686
SimHash 8c04daf2a523

Groups

gigabot

Rule Path
Disallow /

scrubby

Rule Path
Disallow /

nutch

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

asterias

Rule Path
Disallow /

*

Rule Path
Disallow /au/
Disallow /ca/
Disallow /fr/
Disallow /ca/
Disallow /fr/
Disallow /de/
Disallow /in/
Disallow /it/
Disallow /jp/
Disallow /mx/
Disallow /es/
Disallow /uk/

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.cyclevolta.com/arcio/sitemap-index/index/
sitemap https://www.cyclevolta.com/arcio/fronts-sitemap/
Back to top

Comments

  • Disallow the following spiders
Back to top