cgsteam.io
robots.txt

Robots Exclusion Standard data for cgsteam.io

Resource Scan

Scan Details

Site Domain cgsteam.io
Base Domain cgsteam.io
Scan Status Ok
Last Scan2025-06-03T06:27:40+00:00
Next Scan 2025-07-03T06:27:40+00:00

Last Scan

Scanned2025-06-03T06:27:40+00:00
URL https://cgsteam.io/robots.txt
Domain IPs 104.21.45.168, 172.67.217.3, 2606:4700:3030::6815:2da8, 2606:4700:3032::ac43:d903
Response IP 104.21.45.168
Found Yes
Hash 0654450bdaf936a1d1e5d9881502af30d2127cc27433f2d817ebc3e81f8373e0
SimHash 0805452081f3

Groups

*

Rule Path
Allow /
Disallow /cv
Disallow /Admin
Disallow /blog?page=

applebot

Rule Path
Allow /
Disallow /cv
Disallow /Admin
Disallow /blog?page=

openai-gpt3

Rule Path
Allow /
Disallow /cv
Disallow /Admin
Disallow /blog?page=

gpt-4-bot

Rule Path
Allow /
Disallow /cv
Disallow /Admin
Disallow /blog?page=

Other Records

Field Value
sitemap https://cgsteam.io/sitemap.xml