gematsu.com
robots.txt

Robots Exclusion Standard data for gematsu.com

Resource Scan

Scan Details

Site Domain gematsu.com
Base Domain gematsu.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-04T03:48:05+00:00
Next Scan 2025-01-03T03:48:05+00:00

Last Successful Scan

Scanned2024-09-05T20:22:59+00:00
URL https://gematsu.com/robots.txt
Redirect https://www.gematsu.com/robots.txt
Redirect Domain www.gematsu.com
Redirect Base gematsu.com
Domain IPs 104.26.12.127, 104.26.13.127, 172.67.68.54, 2606:4700:20::681a:c7f, 2606:4700:20::681a:d7f, 2606:4700:20::ac43:4436
Redirect IPs 104.26.12.127, 104.26.13.127, 172.67.68.54, 2606:4700:20::681a:c7f, 2606:4700:20::681a:d7f, 2606:4700:20::ac43:4436
Response IP 104.26.12.127
Found Yes
Hash 9c8b86a4ac88ac03d30f4191d5e76bc8708a619465f13eb6056a2d3b6cb3fbff
SimHash 6518d454e331

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5