history.sf-fandom.com
robots.txt

Robots Exclusion Standard data for history.sf-fandom.com

Resource Scan

Scan Details

Site Domain history.sf-fandom.com
Base Domain sf-fandom.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2025-06-16T18:27:54+00:00
Next Scan 2025-06-23T18:27:54+00:00

Last Successful Scan

Scanned2025-06-08T18:27:39+00:00
URL https://history.sf-fandom.com/robots.txt
Domain IPs 192.96.218.77
Response IP 192.96.218.77
Found Yes
Hash 4b487b7045e7fe1a1043a4b46ee4d6c73c090963d865c8bf5d79047249a03df0
SimHash 4a2c5bd07751

Groups

baiduspider
bingbot
duckduckbot
exabot
facebookexternalhit
feedfetcher-google
google-inspectiontool
google-site-verification
google-speakr
googlebot
googlebot-image
googlebot-news
googlebot-video
mediapartners-google
msnbot
qwantify
speedyspider
twitterbot
yandexantivirus/2.0
yandexbot/3.0
yandeximageresizer/2.0
yandeximages/3.0
yandexmedia/3.0
yandexpagechecker/1.0
yandexwebmaster/2.0
yandexzakladki/3.0

Rule Path
Disallow /?s
Disallow /cgi-bin/
Disallow /wp-admin/

ahrefsbot
ccbot/2.0
chatgpt-user
google-cloudvertexbot
google-extended
googleother
mj12bot
mojeekbot
petalbot
semrushbot
similarweb
yepbot
youbot
*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://history.sf-fandom.com/history-sitemap.txt

Comments

  • This Alternate robots.txt file was created by the Alternate Robots.txt WordPress plugin.
  • Allow the good bots in
  • Block the well-behaved unwanted bots