Log In Sign Up

leem.com
robots.txt

Robots Exclusion Standard data for leem.com

Resource Scan

Scan Details

Site Domain leem.com
Base Domain leem.com
Scan Status Ok
Last Scan2024-09-10T20:20:44+00:00
Next Scan 2024-09-24T20:20:44+00:00

Last Scan

Scanned2024-09-10T20:20:44+00:00
URL https://leem.com/robots.txt
Redirect https://www.leem.com/robots.txt
Redirect Domain www.leem.com
Redirect Base leem.com
Domain IPs 13.33.28.123, 13.33.28.36, 13.33.28.53, 13.33.28.82
Redirect IPs 13.226.2.14, 13.226.2.44, 13.226.2.46, 13.226.2.81, 2600:9000:21f8:1600:2:a395:1100:93a1, 2600:9000:21f8:6a00:2:a395:1100:93a1, 2600:9000:21f8:7c00:2:a395:1100:93a1, 2600:9000:21f8:8e00:2:a395:1100:93a1, 2600:9000:21f8:9a00:2:a395:1100:93a1, 2600:9000:21f8:c600:2:a395:1100:93a1, 2600:9000:21f8:e00:2:a395:1100:93a1, 2600:9000:21f8:f800:2:a395:1100:93a1
Response IP 18.165.140.18
Found Yes
Hash ee5f719e6d846a06130dc2b08bc9a422f542d9989270e456e77b3abd5778317b
SimHash 424cdd726e31

Groups

*

Rule Path
Allow *

Other Records

Field Value
crawl-delay 10

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

optimizer

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.leem.com/sitemap.xml
Back to top

Comments

  • sitemap
  • crawling
Back to top