dailynewsonline.jp
robots.txt

Robots Exclusion Standard data for dailynewsonline.jp

Resource Scan

Scan Details

Site Domain dailynewsonline.jp
Base Domain dailynewsonline.jp
Scan Status Ok
Last Scan2024-09-24T15:25:34+00:00
Next Scan 2024-10-01T15:25:34+00:00

Last Scan

Scanned2024-09-24T15:25:34+00:00
URL https://dailynewsonline.jp/robots.txt
Domain IPs 108.157.254.10, 108.157.254.25, 108.157.254.51, 108.157.254.81
Response IP 108.157.254.10
Found Yes
Hash b6febcb79492b5da2011ab877e7943d24d1ee826ab96c26251768c54dc0added
SimHash 4540dc60ef90

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Warnings

  • 1 invalid line.
  • `host` is not a known field.