Log In Sign Up

rshld.eu
robots.txt

Robots Exclusion Standard data for rshld.eu

Resource Scan

Scan Details

Site Domain rshld.eu
Base Domain rshld.eu
Scan Status Ok
Last Scan2025-11-21T01:55:45+00:00
Next Scan 2025-12-21T01:55:45+00:00

Last Scan

Scanned2025-11-21T01:55:45+00:00
URL https://rshld.eu/robots.txt
Domain IPs 104.26.6.157, 104.26.7.157, 172.67.73.119
Response IP 104.26.7.157
Found Yes
Hash 7893ead1f0d46d6f68fae01b0e78d8636eba750ae7e909d94c330ba76aa5b6f4
SimHash 5116dc9077b1

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

slurp

Product Comment
slurp Yahoo
Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

facebot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /
Back to top

Comments

  • robots.txt for https://www.example.com/
  • Market Research & Online Surveys Business
  • Comprehensive and Protective setup blocking all bots including search engines
  • Block all bots from crawling any content on the site
  • Explicitly block major search engines and known bots (redundant but clear)
  • No sitemap is specified because crawling is fully blocked
  • Sitemap: https://www.example.com/sitemap.xml
Back to top