cleaningbeasts.com
robots.txt

Robots Exclusion Standard data for cleaningbeasts.com

Resource Scan

Scan Details

Site Domain cleaningbeasts.com
Base Domain cleaningbeasts.com
Scan Status Ok
Last Scan2024-06-30T22:46:52+00:00
Next Scan 2024-07-07T22:46:52+00:00

Last Scan

Scanned2024-06-30T22:46:52+00:00
URL https://cleaningbeasts.com/robots.txt
Domain IPs 104.21.23.247, 172.67.214.183, 2606:4700:3032::6815:17f7, 2606:4700:3037::ac43:d6b7
Response IP 172.67.214.183
Found Yes
Hash 4b63546a5581075e5020f4ceeeaf9e9aa90ac1eacc97dffc6b1de92523778706
SimHash 6b08c8e0a333

Groups

*

Rule Path
Allow /wp-content/uploads/
Allow /wp-admin/admin-ajax.php
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-content/plugins/
Disallow /wp-admin/
Disallow /readme.html
Disallow /refer/

bottraffic032.xyz

Rule Path
Disallow /

*

Rule Path
Disallow /trafficart.club

*

Rule Path
Disallow /cutt.ly/yxlHTUi

Comments

  • crawl-delay: 10
  • Crawl-delay: 30

Warnings

  • 1 invalid line.