hometheater.co.il
robots.txt

Robots Exclusion Standard data for hometheater.co.il

Resource Scan

Scan Details

Site Domain hometheater.co.il
Base Domain hometheater.co.il
Scan Status Ok
Last Scan2024-06-10T10:38:19+00:00
Next Scan 2024-06-17T10:38:19+00:00

Last Scan

Scanned2024-06-10T10:38:19+00:00
URL https://hometheater.co.il/robots.txt
Domain IPs 104.21.39.157, 172.67.146.184, 2606:4700:3030::6815:279d, 2606:4700:3037::ac43:92b8
Response IP 104.21.39.157
Found Yes
Hash c71e4c5b182dffedc9ae20204fa51e9c5bf5d4405871a8c495c7bb5007ec136b
SimHash 3a08c551ca33

Groups

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot/7~bl

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

siteauditbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot-ba

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot-si

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot-swa

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot-ct

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

splitsignalbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot-coub

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot/2~bl

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot/3~bl

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot/4~bl

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot/5~bl

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot/6~bl

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

neticle crawler

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

*

Rule Path
Disallow /files/*
Disallow /images/*
Disallow /pir/*
Disallow /modules.php?name=Your_Account*
Disallow /phpbb/memberlist.php?*
Disallow /phpbb/posting.php?*
Disallow /phpbb/ucp.php?*
Disallow /wideyoutube.php?*
Disallow /phpbb/app.php/thankslist/*
Disallow /phpbb/search.php?*

googlebot-image

Rule Path
Disallow /wp-content/uploads/*
Disallow /phpbb/*
Allow /*.ico
Allow /*.png

Other Records

Field Value
sitemap https://htmag.co.il/wp-sitemap.xml
sitemap https://htmag.co.il/feed
sitemap https://htmag.co.il/phpbb/app.php/feed