sascustomcarts.com
robots.txt

Robots Exclusion Standard data for sascustomcarts.com

Resource Scan

Scan Details

Site Domain sascustomcarts.com
Base Domain sascustomcarts.com
Scan Status Ok
Last Scan2025-05-10T06:53:41+00:00
Next Scan 2025-06-09T06:53:41+00:00

Last Scan

Scanned2025-05-10T06:53:41+00:00
URL https://sascustomcarts.com/robots.txt
Domain IPs 104.26.2.140, 104.26.3.140, 172.67.72.158, 2606:4700:20::681a:28c, 2606:4700:20::681a:38c, 2606:4700:20::ac43:489e
Response IP 172.67.72.158
Found Yes
Hash 008aac286348f0538dee2f46910003e23aea2d6408c02ea94fb6de5e6b3e9a48
SimHash 683cdc8161a0

Groups

chatgptbot

Rule Path
Allow /

*

Rule Path
Disallow /feed/
Disallow */feed
Disallow */feed$
Disallow /feed/$
Disallow /comments/feed
Disallow /?feed=
Disallow /wp-feed
Disallow /path-to-block/
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 10

google-extended

Rule Path
Allow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot-mobile

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

Other Records

Field Value
sitemap https://sascustomcarts.com/sitemaps.xml