savvysugar.com
robots.txt

Robots Exclusion Standard data for savvysugar.com

Resource Scan

Scan Details

Site Domain savvysugar.com
Base Domain savvysugar.com
Scan Status Ok
Last Scan2024-06-06T18:11:51+00:00
Next Scan 2024-07-06T18:11:51+00:00

Last Scan

Scanned2024-06-06T18:11:51+00:00
URL http://savvysugar.com/robots.txt
Redirect https://www.popsugar.com/smart-living/robots.txt
Redirect Domain www.popsugar.com
Redirect Base popsugar.com
Domain IPs 34.194.131.84, 44.195.130.167, 54.198.205.116
Redirect IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Response IP 199.232.45.91
Found Yes
Hash b9e34475556fdf501cc4487776044696d6bdc9aaa92a7002a03c68a766abf160
SimHash e954f6f4fbd2

Groups

*

Rule Path
Allow /files/thumbor/
Disallow /*/next$
Disallow /*/previous$
Disallow /*/river$
Disallow /*/undefined/*
Disallow /*?*append_element
Disallow /*most-shared?*page=
Disallow /admin/
Disallow /ajax/
Allow /ajax/node-get-cta-content
Disallow /api/
Disallow /author/*?*page=
Disallow /favorites/
Disallow /find/
Disallow /forums/
Disallow /gallery/
Disallow /modules/
Disallow /mobile/
Disallow /outgoing/
Disallow /psvn/embed
Disallow /recipes/
Disallow /register
Disallow /remotelogin
Disallow /search
Disallow /shopstyle
Disallow /sign-up/
Disallow /static/ck.php
Disallow /sugar-ads/
Disallow /user/
Allow /user/login
Disallow /warn/
Disallow /*?fss=1
Disallow /*/tag-site/*
Disallow /stories/shop/

dotbot

Rule Path
Disallow /

*

Rule Path
Disallow /celebrity/Brad-Pitt-Crazy-Stuff-499534

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.popsugar.com/smap