Log In Sign Up

sciforums.com
robots.txt

Robots Exclusion Standard data for sciforums.com

Resource Scan

Scan Details

Site Domain sciforums.com
Base Domain sciforums.com
Scan Status Ok
Last Scan2024-10-05T07:10:40+00:00
Next Scan 2024-10-12T07:10:40+00:00

Last Scan

Scanned2024-10-05T07:10:40+00:00
URL https://sciforums.com/robots.txt
Domain IPs 162.0.209.68
Response IP 162.0.209.68
Found Yes
Hash 2cd60f0aa9dd3a9941bfd6827e3a7599ebff6087e6ea591af90e5ad29317cc08
SimHash 381c0dc4ea88

Groups

*

Rule Path
Disallow /members/

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

data2lifebot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

genieo

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

izsearch.com

Rule Path
Disallow /

spbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

xintellibot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

wesee:ads/picturebot

Rule Path
Disallow /

wesee:ads/pagebot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

um-ic/1.0

Rule Path
Disallow /

mozilla/5.0 (compatible) semanticscholarbot (+https://www.semanticscholar.org/crawler)

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

startmebot

Rule Path
Disallow /

obot

Rule Path
Disallow /

linkdex.com/v2.0

Rule Path
Disallow /

linkdex.com/v2.1

Rule Path
Disallow /

linkdex.com/v2.2

Rule Path
Disallow /

feedlybot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

cocolyzebot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

mbcrawler

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

scrapy

Rule Path
Disallow /
Back to top