linejournal.net
robots.txt

Robots Exclusion Standard data for linejournal.net

Resource Scan

Scan Details

Site Domain linejournal.net
Base Domain linejournal.net
Scan Status Ok
Last Scan2024-06-27T06:49:07+00:00
Next Scan 2024-07-04T06:49:07+00:00

Last Scan

Scanned2024-06-27T06:49:07+00:00
URL https://linejournal.net/robots.txt
Domain IPs 183.111.174.9
Response IP 183.111.174.9
Found Yes
Hash 850ef26f7b56def2459145682fc842d6171bea0d67d7528b00eed5134366b6e3
SimHash 52549060af18

Groups

*

Rule Path
Allow /ads.txt

ahrefsbot
amazonbot
arachni
baiduspider
baiduspider
baiduspider+
bbot
blexbot
brands-bot
dataforseo-bot
dotbot
exabot
eyeotabot
megaindex
mj12bot
petalbot
semrushbot
wordpress

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /do-not-crawl/

*

Rule Path
Disallow /not-allowed/

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow