utph.org
robots.txt

Robots Exclusion Standard data for utph.org

Resource Scan

Scan Details

Site Domain utph.org
Base Domain utph.org
Scan Status Ok
Last Scan2025-11-04T16:26:54+00:00
Next Scan 2025-12-04T16:26:54+00:00

Last Scan

Scanned2025-11-04T16:26:54+00:00
URL https://utph.org/robots.txt
Redirect https://www.utph.org/robots.txt
Redirect Domain www.utph.org
Redirect Base utph.org
Domain IPs 104.18.40.8, 172.64.147.248
Redirect IPs 104.18.40.8, 172.64.147.248
Response IP 172.64.147.248
Found Yes
Hash 6b57b1122fc86407354ad996ffe6874667ac24ab3ab44469bb5cc58c42792b96
SimHash 7903d151c533

Groups

screaming frog seo spider

Rule Path
Disallow /index/test/

*

Rule Path
Disallow /index/test/
Disallow /index-new.htm
Disallow /kitchen-sink
Disallow /about/
Disallow /test-folders/
Disallow /about/leadership
Disallow /about/publications
Disallow /about/strategy

Other Records

Field Value
sitemap https://www.utph.org/utph-sitemap.xml

Comments

  • Allow Screaming Frog crawler
  • Allow all other crawlers