aeg.com.es
robots.txt

Robots Exclusion Standard data for aeg.com.es

Resource Scan

Scan Details

Site Domain aeg.com.es
Base Domain aeg.com.es
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-08-22T02:12:42+00:00
Next Scan 2024-11-20T02:12:42+00:00

Last Successful Scan

Scanned2024-04-02T01:29:16+00:00
URL https://aeg.com.es/robots.txt
Redirect https://www.aeg.com.es/robots.txt
Redirect Domain www.aeg.com.es
Redirect Base aeg.com.es
Domain IPs 23.50.94.113, 2600:1413:b000:384::395d, 2600:1413:b000:393::395d
Redirect IPs 23.50.89.97
Response IP 23.15.111.114
Found Yes
Hash 39394218d63248d9bb813f1a208a1120fc23e199e440ed96bcb99f4e1b086bca
SimHash 7302784a6f12

Groups

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /find/*
Disallow /compare/*
Disallow /search/*
Disallow /link/*
Disallow /overlays/send-to-a-friend/*
Disallow /recycle-bin/*
Disallow /util/*
Disallow /secui/*
Disallow /additions/*
Disallow /templates/*
Disallow /global-pages/global-menu/store-locator/
Disallow /global-pages/search-page/
Disallow /global/images/icons/
Disallow /localfiles/*
Disallow /documents/*
Disallow /pagefiles/*
Disallow /webresource.*
Disallow /node*.*
Disallow /my-pages-container.*
Disallow /my-pages/*
Disallow /*/modallogin*
Disallow /handlers/*
Disallow /image/
Disallow /downloadpdf/

Other Records

Field Value
sitemap https://www.aeg.com.es/globalassets/sitemaps/www.aeg.com.es/sitemapindex.xml

Comments

  • new
  • Directories
  • Resources