Log In Sign Up

sheriahub.com
robots.txt

Robots Exclusion Standard data for sheriahub.com

Resource Scan

Scan Details

Site Domain sheriahub.com
Base Domain sheriahub.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-04T13:06:21+00:00
Next Scan 2024-11-03T13:06:21+00:00

Last Successful Scan

Scanned2024-06-14T13:04:55+00:00
URL https://sheriahub.com/robots.txt
Domain IPs 104.21.33.215, 172.67.192.214, 2606:4700:3030::ac43:c0d6, 2606:4700:3037::6815:21d7
Response IP 172.67.192.214
Found Yes
Hash dc22809b1bb471c924f18fdc2295f4855a792713267de7b7933c13c04dc63545
SimHash 580577c1c795

Groups

*

Rule Path
Disallow /dashboard/*

*

Rule Path
Disallow /js/improvement.js*

*

Rule Path
Disallow /cgi-bin/*

ia_archiver

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow

applebot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

bingbot

Rule Path
Allow /

discordbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

naverbot

Rule Path
Allow /

slurp

Rule Path
Allow /

telegrambot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

yandex

Rule Path
Allow /

yeti

Rule Path
Allow /

gptbot

Rule Path
Disallow /

*

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://sheriahub.com/sitemap.xml
Back to top

Comments

  • Notice: Collection of data on Sheriahub through automated means is
  • prohibited unless you have express written permission from Sheriahub
  • and may only be conducted for the limited purpose contained in said
  • permission.
Back to top