maplebear.vn
robots.txt

Robots Exclusion Standard data for maplebear.vn

Resource Scan

Scan Details

Site Domain maplebear.vn
Base Domain maplebear.vn
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-04-19T17:52:36+00:00
Next Scan 2025-04-26T17:52:36+00:00

Last Successful Scan

Scanned2025-04-11T17:52:29+00:00
URL https://maplebear.vn/robots.txt
Redirect https://derbyshireuk.net/robots.txt
Redirect Domain derbyshireuk.net
Redirect Base derbyshireuk.net
Domain IPs 104.21.12.136, 172.67.152.137, 2606:4700:3032::ac43:9889, 2606:4700:3034::6815:c88
Redirect IPs 104.26.12.30, 104.26.13.30, 172.67.72.250, 2606:4700:20::681a:c1e, 2606:4700:20::681a:d1e, 2606:4700:20::ac43:48fa
Response IP 172.67.72.250
Found Yes
Hash d61cab956623b579d4d6bdd7da92ab9bdb621edb610fb129948c305290cff9cf
SimHash 6b6adc04cba2

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /author/
Disallow /*/trackback
Disallow /img/
Disallow /tag/
Disallow /feed
Disallow /*/feed
Disallow /comments/feed
Disallow /?s=*
Disallow /attachment/

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
sitemap https://derbyshireuk.net/sitemap.xml

Comments

  • Allow Facebook scraper