teacherspayteachers.com
robots.txt

Robots Exclusion Standard data for teacherspayteachers.com

Resource Scan

Scan Details

Site Domain teacherspayteachers.com
Base Domain teacherspayteachers.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-22T15:22:39+00:00
Next Scan 2024-11-21T15:22:39+00:00

Last Successful Scan

Scanned2024-07-02T15:05:51+00:00
URL https://teacherspayteachers.com/robots.txt
Redirect https://www.teacherspayteachers.com:443/robots.txt
Redirect Domain www.teacherspayteachers.com
Redirect Base teacherspayteachers.com
Domain IPs 100.24.147.142, 34.202.30.203, 35.153.218.21, 54.81.175.16
Redirect IPs 104.18.37.229, 172.64.150.27, 2606:4700:4400::6812:25e5, 2606:4700:4400::ac40:961b
Response IP 172.64.150.27
Found Yes
Hash 0ba132604aa6062b2a858cb792a0ca58ae00f56f9596caa90bc570cbe233ac60
SimHash b82cda1389b3

Groups

*

Rule Path
Disallow /*.pdf$
Disallow /*.xls$
Disallow /*.ppt$
Disallow /*.doc$
Disallow /*.rtf$
Disallow /*.swf$
Disallow /errors/
Disallow /store/*?*
Disallow /Store/*/Search
Disallow /Store/*/Search*
Disallow /Store/*/Order*
Disallow /Download/*
Disallow /carts/ajax_index
Disallow /Cart/Add*
Disallow /Login/*
Disallow /Login?*
Disallow /browse?order=
Disallow /graph/graphql*
Disallow /graph/graphiql.html*
Disallow /users
Disallow /users/*
Disallow /Cart
Disallow /Cart/*
Disallow /Wish-List
Disallow /Wish-List/*
Disallow /University/
Disallow /University/*
Disallow /university/
Disallow /university/*
Disallow /widgets/show/*
Disallow /widgets/get/*

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

twitterbot
applebot-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.teacherspayteachers.com/Sitemap_index.xml