cicadamania.com
robots.txt

Robots Exclusion Standard data for cicadamania.com

Resource Scan

Scan Details

Site Domain cicadamania.com
Base Domain cicadamania.com
Scan Status Ok
Last Scan2024-10-02T13:13:29+00:00
Next Scan 2024-11-01T13:13:29+00:00

Last Scan

Scanned2024-10-02T13:13:29+00:00
URL https://cicadamania.com/robots.txt
Redirect https://www.cicadamania.com/robots.txt
Redirect Domain www.cicadamania.com
Redirect Base cicadamania.com
Domain IPs 75.119.203.87
Redirect IPs 75.119.203.87
Response IP 75.119.203.87
Found Yes
Hash 5cd34e0d6f1f8c179e45289f7d73b6d48a4622b04194e53674b0d21c59f47ec2
SimHash 085f9c680ebb

Groups

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

*

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.cicadamania.com/urllist.txt
sitemap https://www.cicadamania.com/cicadas/feed/

Comments

  • Robot roll call!
  • Sitemap files