Log In Sign Up

shop.pbs.org
robots.txt

Robots Exclusion Standard data for shop.pbs.org

Resource Scan

Scan Details

Site Domain shop.pbs.org
Base Domain pbs.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-22T08:01:33+00:00
Next Scan 2024-10-20T08:01:33+00:00

Last Successful Scan

Scanned2024-03-02T06:46:06+00:00
URL https://shop.pbs.org/robots.txt
Domain IPs 23.215.7.14, 23.215.7.16
Response IP 23.32.29.104
Found Yes
Hash f3d948a2102f1e20e3acb31287d54b03589d3b80ef7d42cccb5ec19e924f0e2b
SimHash edc872574777

Groups

*

Rule Path
Allow /?action=ADD&ITEM=*
Disallow /tools/no_results.hzml?q=*
Disallow /cgi-bin/hazel.cgi?ACTION=find*
Disallow /*filter%3A*
Disallow /*sort%3A*
Disallow /*perpage%3A48*
Disallow /*perpage%3A64*
Disallow /*perpage%3A499*
Disallow /*category-filter%3D
Disallow /*sort%3D
Disallow /*rows%3D48
Disallow /*rows%3D96
Disallow /find?q=*
Disallow /cgi-bin/hazel.cgi?ACTION=search*
Disallow /cgi-bin/hazel.cgi?action=search*
Disallow /cgi-bin/hazel.cgi?ACTION=ADD*
Disallow /cgi-bin/hazel.cgi?action=ADD*
Disallow /cgi-bin/hazel.cgi?action=CHECKOUT*
Disallow /cgi-bin/hazel.cgi?ACTION=CHECKOUT*
Disallow /cgi-bin/hazel.cgi?ACTION=VIEW
Disallow /cgi-bin/hazel.cgi?action=VIEW*
Disallow /cgi-bin/hazel.cgi?action=VIEW*
Disallow /?ACTION=search*
Disallow /?action=search*
Disallow /?ACTION=ADD*
Disallow /?action=ADD*
Disallow /?action=CHECKOUT*
Disallow /?ACTION=CHECKOUT*
Disallow /?ACTION=VIEW
Disallow /?action=VIEW*
Disallow /?action=VIEW*
Disallow /?ACTION=FIND*
Disallow /?action=find*
Disallow /members/
Disallow /lakenameshirts/
Disallow /lakeart/
Disallow /insert
Disallow /cart
Disallow /checkout
Disallow /account
Disallow /searchresults
Disallow /noresults
Disallow /search?
Disallow /find?
Disallow /confirmation
Disallow /billing
Disallow /shipping
Disallow /logout
Disallow /forgetme
Disallow /wishlistupdate
Disallow /shared-wishlist
Disallow /wishlistremove
Disallow /cartsummary
Disallow /minicart
Disallow /resetpassword
Disallow /default
Disallow /keepalive
Disallow /info/shopkids_modal.html
Disallow /tools/email_capture.html
Disallow /tools/email_capture.hzml
Disallow /tools/email_capture_drawer.html
Disallow /tools/rory.hzml
Disallow /tools/rory2.hzml
Disallow /info/flip_catalog_modal.hzml
Disallow /welcome_email/welcome_email.html

twengabot

Rule Path
Disallow /

twenga2.com

Rule Path
Disallow /

twenga.com

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://shop.pbs.org/sitemap.xml
Back to top

Comments

  • Disallow: /add
Back to top