rdn-team.com
robots.txt

Robots Exclusion Standard data for rdn-team.com

Resource Scan

Scan Details

Site Domain rdn-team.com
Base Domain rdn-team.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-04-21T00:17:06+00:00
Next Scan 2025-06-20T00:17:06+00:00

Last Successful Scan

Scanned2025-02-21T00:11:31+00:00
URL https://rdn-team.com/robots.txt
Domain IPs 104.21.93.239, 172.67.216.179, 2606:4700:3030::6815:5def, 2606:4700:3031::ac43:d8b3
Response IP 172.67.216.179
Found Yes
Hash 3e722bd88007e54c4c5475c09d6c571695f53120f916141f2a3b1eef8653e227
SimHash 5d59dc20c691

Groups

*

Rule Path
Disallow /engine/
Disallow /*do%3D*
Disallow /*newsid%3D*
Disallow /user/

msnbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap /sitemap.xml

Comments

  • Crawl-delay: 2

Warnings

  • `host` is not a known field.