Log In Sign Up

rutubet.info
robots.txt

Robots Exclusion Standard data for rutubet.info

Resource Scan

Scan Details

Site Domain rutubet.info
Base Domain rutubet.info
Scan Status Ok
Last Scan5/18/2025, 2:44:57 AM
Next Scan 6/17/2025, 2:44:57 AM

Last Scan

Scanned5/18/2025, 2:44:57 AM
URL https://rutubet.info/robots.txt
Redirect https://www.rutubet.info/robots.txt
Redirect Domain www.rutubet.info
Redirect Base rutubet.info
Domain IPs 104.21.21.159, 172.67.199.89, 2606:4700:3035::6815:159f, 2606:4700:3036::ac43:c759
Redirect IPs 104.21.21.159, 172.67.199.89, 2606:4700:3035::6815:159f, 2606:4700:3036::ac43:c759
Response IP 104.21.21.159
Found Yes
Hash a87b9065bcd965f8065248e91f01768d06f71836b2c47660ba59d3b4c2a206eb
SimHash 501c4848c0b0

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /wp-admin/

ia_archiver

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

bytedance

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.rutubet.info/sitemap.xml
Back to top