internallinkjuicer.com
robots.txt

Robots Exclusion Standard data for internallinkjuicer.com

Resource Scan

Scan Details

Site Domain internallinkjuicer.com
Base Domain internallinkjuicer.com
Scan Status Ok
Last Scan2026-01-31T05:32:18+00:00
Next Scan 2026-03-02T05:32:18+00:00

Last Scan

Scanned2026-01-31T05:32:18+00:00
URL https://internallinkjuicer.com/robots.txt
Redirect https://www.internallinkjuicer.com/robots.txt
Redirect Domain www.internallinkjuicer.com
Redirect Base internallinkjuicer.com
Domain IPs 104.21.57.51, 172.67.141.246, 2606:4700:3034::ac43:8df6, 2606:4700:3035::6815:3933
Redirect IPs 104.21.57.51, 172.67.141.246, 2606:4700:3034::ac43:8df6, 2606:4700:3035::6815:3933
Response IP 104.21.57.51
Found Yes
Hash 2bb9f3a25704a3c41b88c9f7499df0ed006c7fbe5a2f33fec1fe3e4d4316e7f5
SimHash 4beccdc27cb3

Groups

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

companybook-crawler

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot/7~bl

Rule Path
Disallow /

linguee

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

carsnipbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

*

Rule Path
Disallow /*?wc_country_preselect
Disallow /forums/ticket/
Disallow /forums/reply/

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /?s=
Disallow /page/*/?s=
Disallow /search/
Disallow /wp-json/
Disallow /?rest_route=

adsbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.internallinkjuicer.com/sitemap_index.xml

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK

Warnings

  • 2 invalid lines.