freetexthost.net
robots.txt

Robots Exclusion Standard data for freetexthost.net

Resource Scan

Scan Details

Site Domain freetexthost.net
Base Domain freetexthost.net
Scan Status Ok
Last Scan2025-03-24T15:23:00+00:00
Next Scan 2025-04-23T15:23:00+00:00

Last Scan

Scanned2025-03-24T15:23:00+00:00
URL https://freetexthost.net/robots.txt
Domain IPs 104.21.22.25, 172.67.202.13, 2606:4700:3032::6815:1619, 2606:4700:3036::ac43:ca0d
Response IP 172.67.202.13
Found Yes
Hash 3b12a47ef900c5007b08a38c3e8d8c739d431b445c697f85f755fca0c5839e89
SimHash 3275c462d118

Groups

*

Rule Path
Disallow /admin/
Disallow /includes/
Disallow /theme/
Disallow /langs/
Disallow /tmp/
Disallow /oauth/
Disallow /mail/
Disallow /page/privacy
Disallow /page/terms
Disallow /page/contact

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

lipperhey spider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ncbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

pagesinventory

Rule Path
Disallow /

spbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

majestic-12

Rule Path
Disallow /

majestic-seo

Rule Path
Disallow /

dsearch

Rule Path
Disallow /

mj12

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

tineye

Rule Path
Disallow /

serpstat

Rule Path
Disallow /

spyfu

Rule Path
Disallow /