charlottejcc.org
robots.txt

Robots Exclusion Standard data for charlottejcc.org

Resource Scan

Scan Details

Site Domain charlottejcc.org
Base Domain charlottejcc.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-02T01:57:27+00:00
Next Scan 2024-10-31T01:57:27+00:00

Last Successful Scan

Scanned2023-06-17T00:19:16+00:00
URL https://charlottejcc.org/robots.txt
Redirect http://www.charlottejcc.org/robots.txt
Redirect Domain www.charlottejcc.org
Redirect Base charlottejcc.org
Domain IPs 104.26.6.16, 104.26.7.16, 172.67.70.60, 2606:4700:20::681a:610, 2606:4700:20::681a:710, 2606:4700:20::ac43:463c
Redirect IPs 104.26.6.16, 104.26.7.16, 172.67.70.60, 2606:4700:20::681a:610, 2606:4700:20::681a:710, 2606:4700:20::ac43:463c
Response IP 172.67.70.60
Found Yes
Hash 033267f999f7bfacadbb79a388e3a2ec7953c0dbd9a0b8d9a354671a680a5c92
SimHash adc89ac0e5b6

Groups

*

Rule Path
Disallow /*print%3Dpdf*

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.charlottejcc.org/sitemap.xml

Comments

  • ROBOTS.TXT
  • asoft200231.accrisoft.com
  • Google
  • User-agent: Googlebot
  • Disallow:
  • Yahoo
  • User-agent: Slurp
  • Disallow:
  • Alta-Vista
  • User-agent: Scooter
  • Disallow:
  • Excite
  • User-agent: ArchitextSpider
  • Disallow:
  • InfoSeek
  • User-agent: UltraSeek
  • Disallow:
  • Lycos
  • User-agent: Lycos_Spider_(T-Rex)
  • Disallow:
  • LookSmart
  • User-agent: MantraAgent
  • Disallow:
  • Alltheweb
  • User-agent: FAST-WebCrawler
  • Disallow: