albayan.ae
robots.txt

Robots Exclusion Standard data for albayan.ae

Resource Scan

Scan Details

Site Domain albayan.ae
Base Domain albayan.ae
Scan Status Ok
Last Scan2024-06-13T19:16:29+00:00
Next Scan 2024-06-20T19:16:29+00:00

Last Scan

Scanned2024-06-13T19:16:29+00:00
URL https://albayan.ae/robots.txt
Redirect https://www.albayan.ae/robots.txt
Redirect Domain www.albayan.ae
Redirect Base albayan.ae
Domain IPs 3.250.251.107
Redirect IPs 23.50.232.200, 23.50.232.235, 2600:1413:5000:12::1737:27f2, 2600:1413:5000:12::1737:27fa
Response IP 23.59.168.136
Found Yes
Hash 989ceb0acac2d9751e7271b0bb51160bde8787277d38d8ce0526e531244fda8f
SimHash 6d580843b3b3

Groups

*

Rule Path
Disallow /res/html
Disallow /captcha
Disallow /logger
Disallow /happiness-meter
Allow /

Other Records

Field Value
sitemap https://www.albayan.ae/news-sitemaps-1.1402252?ot=ot.AjaxPageLayout
sitemap https://www.albayan.ae/generic-sitemaps-1.1381814?ot=ot.AjaxPageLayout

Comments

  • News Sitemaps
  • Generic Sitemaps
  • Sitemap: https://www.albayan.ae/sitemaps/index.xml
  • Disallow: /search-7.124
  • Disallow: /search-7.3262750
  • Disallow: /search-7.3263153