stjobs.sg
robots.txt

Robots Exclusion Standard data for stjobs.sg

Resource Scan

Scan Details

Site Domain stjobs.sg
Base Domain stjobs.sg
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-28T22:43:38+00:00
Next Scan 2024-09-26T22:43:38+00:00

Last Successful Scan

Scanned2024-05-07T22:42:16+00:00
URL https://stjobs.sg/robots.txt
Redirect https://www.stjobs.sg/robots.txt
Redirect Domain www.stjobs.sg
Redirect Base stjobs.sg
Domain IPs 108.156.133.115, 108.156.133.66, 108.156.133.8, 108.156.133.90
Redirect IPs 152.195.56.190
Response IP 152.195.56.190
Found Yes
Hash 6e08ed6ae91c92e83a841755e0b537e99868d06041939d0a7efd0a00b64d6b47
SimHash 7040f90e4337

Groups

*

Rule Path
Disallow /admin/
Disallow /api/
Disallow /template/
Disallow /mob/

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.stjobs.sg/sitemaps/sitemap-index.xml

Comments

  • For new training only
  • Not for training, only for user requests
  • Marker for disabling Bard and Vertex AI
  • Multi-purpose, commercial uses; including LLMs