sbf-sg.social
robots.txt

Robots Exclusion Standard data for sbf-sg.social

Resource Scan

Scan Details

Site Domain sbf-sg.social
Base Domain sbf-sg.social
Scan Status Ok
Last Scan5/24/2025, 12:53:13 AM
Next Scan 6/23/2025, 12:53:13 AM

Last Scan

Scanned5/24/2025, 12:53:13 AM
URL https://sbf-sg.social/robots.txt
Domain IPs 104.21.44.234, 172.67.204.251, 2606:4700:3037::6815:2cea, 2606:4700:3037::ac43:ccfb
Response IP 104.21.44.234
Found Yes
Hash 2a7151806e2cbafd9fde314e9de3a2d52db94260ee06a60b697be725b0ea79de
SimHash 673cd060eb50

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

amazonbot

Rule Path
Disallow /do-not-crawl/