superdown.com
robots.txt

Robots Exclusion Standard data for superdown.com

Resource Scan

Scan Details

Site Domain superdown.com
Base Domain superdown.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-09-08T09:55:16+00:00
Next Scan 2025-11-07T09:55:16+00:00

Last Successful Scan

Scanned2025-07-03T05:20:14+00:00
URL https://superdown.com/robots.txt
Redirect https://www.superdown.com/robots.txt
Redirect Domain www.superdown.com
Redirect Base superdown.com
Domain IPs 23.58.144.144, 23.58.144.153
Redirect IPs 23.52.171.208, 23.52.171.57
Response IP 23.32.39.149
Found Yes
Hash a73ac489f547275d786c694dca99537c7d1cd7acd67d1b8370c907ac82e840a6
SimHash 45246ef10233

Groups

gptbot

Rule Path
Disallow /

*

Rule Path
Disallow /WEB-INF/
Disallow /BrowserInfo.jsp
Disallow /beta/
Disallow /productpages/
Disallow /hgg/
Disallow /classic/
Disallow /images/
Disallow /Unsubscribe.jsp
Disallow /luckybreaks/
Disallow /brandinfopages/
Disallow /beta3/
Disallow /fwd/Shipping.jsp
Disallow /fwd/AboutUs.jsp
Disallow /fwd/CustomerService.jsp
Disallow /fwd/Guarantee.jsp
Disallow /fwd/ReturnPolicy.jsp
Disallow /fwd/FAQ.jsp
Disallow /fwd/MyAccount.jsp
Disallow /fwd/LoginOrCreateAccount.jsp
Disallow /fwd/PrivacyPolicy.jsp
Disallow /blog/ViewFullPost.jsp
Disallow /fw/
Disallow /r/PageTrack.jsp
Disallow /r/mobile/PageTrack.jsp
Disallow /edit/
Disallow /blognew/
Disallow /images/emailpics/
Disallow /f/
Disallow /magazine/
Disallow /w/
Disallow /mobile/Checkout.jsp
Disallow /r/ShoppingBag.jsp

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.superdown.com/scripts/sd_sitemap/sd_sitemap_index.xml

Comments

  • Block GPT crawler
  • Block GPT crawler