newming.io
robots.txt

Robots Exclusion Standard data for newming.io

Resource Scan

Scan Details

Site Domain newming.io
Base Domain newming.io
Scan Status Ok
Last Scan2024-11-16T16:30:17+00:00
Next Scan 2024-12-16T16:30:17+00:00

Last Scan

Scanned2024-11-16T16:30:17+00:00
URL https://newming.io/robots.txt
Domain IPs 13.225.4.102, 13.225.4.103, 13.225.4.15, 13.225.4.99
Response IP 13.225.4.15
Found Yes
Hash accc9032d8f8a578869e68e83fbab060518e0917440a554120e726ec35795014
SimHash c03698800766

Groups

*

Rule Path
Disallow /

googlebot-image

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3600

mediapartners-google

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3600

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3600

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3600

naverbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

adsbot-naver

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

yeti

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

Comments

  • robots.txt for Common Domain
  • $Id: robots.txt,v 1.0 2013/06/30 13:37:40 Exp $