Log In Sign Up

furarchiver.net
robots.txt

Robots Exclusion Standard data for furarchiver.net

Resource Scan

Scan Details

Site Domain furarchiver.net
Base Domain furarchiver.net
Scan Status Ok
Last Scan2024-06-09T01:07:45+00:00
Next Scan 2024-07-09T01:07:45+00:00

Last Scan

Scanned2024-06-09T01:07:45+00:00
URL https://furarchiver.net/robots.txt
Domain IPs 104.21.45.69, 172.67.211.14, 2606:4700:3032::ac43:d30e, 2606:4700:3033::6815:2d45
Response IP 104.21.45.69
Found Yes
Hash bdb02652dbf0bfa14eb37eafcd35c8072c9f5c4f908e5cf2aba00888f0e848ed
SimHash 9874797bdd59

Groups

*

Rule Path
Disallow /Account/

obot

Rule Path
Disallow /

x28-job-bot

Rule Path
Disallow /

facrbot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

intelx.io_bot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

datanyze

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

netpeakcheckerbot

Rule Path
Disallow /

netpeakchecker

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

zoombot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://furarchiver.net/sitemap.xml
Back to top

Comments

  • IBM content checker bot
  • Unnecessary bots
  • Not providing an URL
  • Unknown/Unavailable
  • Below is a list of random SEO Bots we don't benefit from
  • Usually they sell your data but don't give your own data to you for free.
  • Bots in this list are usually automatically IP blocked if they try to access anything else than this file
Back to top