almanahj.com
robots.txt

Robots Exclusion Standard data for almanahj.com

Resource Scan

Scan Details

Site Domain almanahj.com
Base Domain almanahj.com
Scan Status Ok
Last Scan2024-06-13T19:15:45+00:00
Next Scan 2024-06-20T19:15:45+00:00

Last Scan

Scanned2024-06-13T19:15:45+00:00
URL https://almanahj.com/robots.txt
Domain IPs 199.85.209.22
Response IP 199.85.209.22
Found Yes
Hash 7af9c463df39ad1efc1fa6449038f1f849b0b1df6e79a22e47f56d36403e7dd9
SimHash 4334115757bb

Groups

*

Rule Path
Disallow /aafolder
Disallow /bots
Disallow /test
Allow /chat/css
Disallow /chat
Disallow /.pdf$

rogerbot

Rule Path
Disallow /

siteauditbot disallow: /
semrushbot-ba disallow: /
semrushbot-si disallow: /
semrushbot-swa disallow: /
semrushbot-ct disallow: /
semrushbot-bm disallow: /
splitsignalbot disallow: /
semrushbot-coub disallow: /
ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

nbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

facebot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://almanahj.com/ae/sitemap.xml
sitemap https://almanahj.com/bh/sitemap.xml
sitemap https://almanahj.com/sa/sitemap.xml
sitemap https://almanahj.com/eg/sitemap.xml
sitemap https://almanahj.com/kw/sitemap.xml
sitemap https://almanahj.com/om/sitemap.xml
sitemap https://almanahj.com/ae/posts_sitemap.xml
sitemap https://almanahj.com/bh/posts_sitemap.xml
sitemap https://almanahj.com/om/posts_sitemap.xml
sitemap https://almanahj.com/kw/posts_sitemap.xml
sitemap https://almanahj.com/sa/posts_sitemap.xml
sitemap https://almanahj.com/qa/posts_sitemap.xml
sitemap https://almanahj.com/us/posts_sitemap.xml

Warnings

  • 1 invalid line.
  • `x-robots-tag` is not a known field.