serv.co.za
robots.txt

Robots Exclusion Standard data for serv.co.za

Resource Scan

Scan Details

Site Domain serv.co.za
Base Domain serv.co.za
Scan Status Ok
Last Scan2025-11-18T10:35:10+00:00
Next Scan 2025-11-25T10:35:10+00:00

Last Scan

Scanned2025-11-18T10:35:10+00:00
URL https://serv.co.za/robots.txt
Domain IPs 66.147.237.173
Response IP 66.147.237.173
Found Yes
Hash 586e283146942925559ab13864dc1cd808ed0512a0d5e64719e1331c4d9f57de
SimHash 0013f8915577

Groups

oai-searchbot
chatgpt-user

Rule Path
Allow /

gptbot

Rule Path Comment
Disallow / Optional - Disallow for training

googlebot
googlebot-image
mediapartners-google
google-structured-data-testing-tool
bingbot
bing preview bot
baiduspider
yandexbot
yahoo-mmcrawler
yahoo-slurp
slurp
msnbot
msnbot-media

Rule Path
Allow /

twitterbot

Rule Path
Allow /

Comments

  • AI Bots
  • Main Search Engine Bots
  • Other Bots
  • END