budgetandinvest.com
robots.txt

Robots Exclusion Standard data for budgetandinvest.com

Resource Scan

Scan Details

Site Domain budgetandinvest.com
Base Domain budgetandinvest.com
Scan Status Ok
Last Scan2024-05-31T19:16:09+00:00
Next Scan 2024-06-07T19:16:09+00:00

Last Scan

Scanned2024-05-31T19:16:09+00:00
URL https://budgetandinvest.com/robots.txt
Redirect https://www.budgetandinvest.com/robots.txt
Redirect Domain www.budgetandinvest.com
Redirect Base budgetandinvest.com
Domain IPs 104.21.75.50, 172.67.213.198, 2606:4700:3030::6815:4b32, 2606:4700:3037::ac43:d5c6
Redirect IPs 104.18.37.69, 172.64.150.187, 2606:4700:4400::6812:2545, 2606:4700:4400::ac40:96bb
Response IP 104.18.37.69
Found Yes
Hash 5512e73e62448416253bf96729e2342d8ced51801acc732a24f635c90044acb1
SimHash 00149932ee42

Groups

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /trackback/
Disallow */comments$
Disallow */trackback
Disallow */trackback$
Disallow /wp-comments
Disallow /wp-trackback
Disallow */replytocom%3D

Comments

  • Block Ahrefs Crawler
  • Block Semrush Crawler
  • Block Moz Crawler
  • Block Chatgpt
  • Block Spam Directories