makito.es
robots.txt

Robots Exclusion Standard data for makito.es

Resource Scan

Scan Details

Site Domain makito.es
Base Domain makito.es
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-15T00:51:59+00:00
Next Scan 2024-09-13T00:51:59+00:00

Last Successful Scan

Scanned2023-01-26T02:41:03+00:00
URL https://makito.es/robots.txt
Domain IPs 23.44.4.144, 23.44.4.178, 2600:1413:b000:6::17d5:2bcf, 2600:1413:b000:6::17d5:2bde
Response IP 104.88.70.155
Found Yes
Hash 0c93e01e281515d919ee749ae4c9509aa2c90f8f23dfb10c3270a523611473cc
SimHash 0c7d531ccf90

Groups

*

Rule Path
Disallow /epages/Site.admin/
Disallow /epages/*ViewAction%3DSF-ViewJQueryConfig
Disallow /epages/*ViewAction%3DMobileSF-ViewJQueryConfig
Disallow /WebRoot/StoreTypes/*/lib/*.js
Disallow /epages/*ViewAction%3DFacetedSearchProducts
Disallow /epages/*ViewAction%3DRemoteSearchProducts
Disallow /epages/*ViewAction%3DViewRemoteSearchProducts
Disallow /epages/*ViewAction%3DMobileSF-ViewDetailRemoteSearchProducts
Disallow /epages/*ViewAction%3DMobileSF-DetailRemoteSearchProducts
Disallow /epages/*ViewAction%3DMobileSF-ViewDetailFacetedSearchProducts
Disallow /epages/*ViewAction%3DMobileSF-DetailFacetedSearchProducts

Comments

  • robots.txt for epages
  • applies to all robots
  • disallow TBO, PBO
  • disallow javascript
  • prevent mass indexing of search pages