forgebox.io
robots.txt

Robots Exclusion Standard data for forgebox.io

Resource Scan

Scan Details

Site Domain forgebox.io
Base Domain forgebox.io
Scan Status Ok
Last Scan2025-09-10T04:27:34+00:00
Next Scan 2025-10-10T04:27:34+00:00

Last Scan

Scanned2025-09-10T04:27:34+00:00
URL https://forgebox.io/robots.txt
Domain IPs 104.26.12.201, 104.26.13.201, 172.67.75.122, 2606:4700:20::681a:cc9, 2606:4700:20::681a:dc9, 2606:4700:20::ac43:4b7a
Response IP 104.26.13.201
Found Yes
Hash e2949bbfe378c94eb2d00469f523cbf7c7301ae619399c6c7bc4863c8df810eb
SimHash 6842d9f14471

Groups

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 100

gsa-crawler-www

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

mediapartners-google

Rule Path
Disallow

yahoo-newscrawler

Rule Path
Disallow

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 100

*

Rule Path
Disallow /config/
Disallow /handlers/
Disallow /includes/
Disallow /interceptors/
Disallow /layouts/
Disallow /lib/
Disallow /logs/
Disallow /models/
Disallow /modules/
Disallow /modules_app/
Disallow /views/
Disallow /download
Allow /

Other Records

Field Value
crawl-delay 10