Log In Sign Up

thecakeboutiquect.com
robots.txt

Robots Exclusion Standard data for thecakeboutiquect.com

Resource Scan

Scan Details

Site Domain thecakeboutiquect.com
Base Domain thecakeboutiquect.com
Scan Status Ok
Last Scan2024-09-22T03:14:10+00:00
Next Scan 2024-09-29T03:14:10+00:00

Last Scan

Scanned2024-09-22T03:14:10+00:00
URL https://thecakeboutiquect.com/robots.txt
Domain IPs 104.219.248.120
Response IP 104.219.248.120
Found Yes
Hash b71732e21e47313ab1ade15ba7f11383d2c1a36825c14e005c78c353d4d8ea33
SimHash d37d5462e881

Groups

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

amazonbot

Rule Path
Disallow /

amazon cloudfront

Rule Path
Disallow /

s[ee][mm]rushbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

vulpixitbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Back to top