Log In Sign Up

occ.treas.gov
robots.txt

Robots Exclusion Standard data for occ.treas.gov

Resource Scan

Scan Details

Site Domain occ.treas.gov
Base Domain treas.gov
Scan Status Ok
Last Scan2024-10-19T18:50:54+00:00
Next Scan 2024-11-18T18:50:54+00:00

Last Scan

Scanned2024-10-19T18:50:54+00:00
URL https://occ.treas.gov/robots.txt
Domain IPs 199.83.40.54
Response IP 199.83.40.54
Found Yes
Hash 1a4462a0914aed16f53ec97771c58b1afcbdf660b73be346741719e5bbe1e4b9
SimHash f41752a19813

Groups

gsa-crawler

Rule Path
Disallow

applebot

Rule Path
Disallow /occsearch/
Disallow /static/community-affairs/_private/
Disallow /static/community-affairs/includes/
Disallow /static/community-affairs/community-developments-investments/cdvc-ezine2007/_private/
Disallow /static/community-affairs/community-developments-investments/fall08/includes/
Disallow /static/community-affairs/community-developments-investments/spring06b/headers/
Disallow /static/community-affairs/community-developments-investments/spring06b/templates/
Disallow /static/community-affairs/community-developments-investments/spring09/includes/
Disallow /static/community-affairs/community-developments-investments/summer05/styles/
Disallow /static/community-affairs/community-developments-investments/summer06/_private/
Disallow /static/community-affairs/community-developments-investments/summer06/Templates/
Disallow /static/community-affairs/community-developments-newsletter/fall08/includes/
Disallow /static/community-affairs/community-developments-newsletter/spring08/includes/
Disallow /static/community-affairs/community-developments-newsletter/spring10/_includes/
Disallow /static/community-affairs/community-developments-newsletter/summer05/_private/
Disallow /static/community-affairs/community-developments-newsletter/summer05/Templates/
Disallow /static/community-affairs/other-publications/FinLit/
Disallow /static/css/
Disallow /static/header-footer/
Disallow /static/training/
Disallow /static/weekly-bulletin/
Disallow /404-error-page-not-found.html
Disallow /about/who-we-are/history/160th/index-160th-anniversary-celebration.html
Disallow /about/managers-conference-agenda.pdf
Disallow /topics/supervision-and-examination/compliance-conference-2023.html
Disallow /topics/supervision-and-examination/financial-technology/digital-assets/symposium-on-tokenization-of-real-world-assets-liabilities-panel.html
Disallow /about/occ-all-staff-conference-agenda.html
Disallow /about/all-staff-conference/
Back to top

Other Records

Field Value
sitemap https://www.occ.gov/sitemap.xml
Back to top

Warnings

  • `noindex` is not a known field.
Back to top