pme-legend.com
robots.txt

Robots Exclusion Standard data for pme-legend.com

Resource Scan

Scan Details

Site Domain pme-legend.com
Base Domain pme-legend.com
Scan Status Ok
Last Scan2024-11-16T13:04:50+00:00
Next Scan 2024-11-30T13:04:50+00:00

Last Scan

Scanned2024-11-16T13:04:50+00:00
URL https://pme-legend.com/robots.txt
Redirect https://www.pme-legend.com/robots.txt
Redirect Domain www.pme-legend.com
Redirect Base pme-legend.com
Domain IPs 104.26.8.197, 104.26.9.197, 172.67.68.169
Redirect IPs 104.26.8.197, 104.26.9.197, 172.67.68.169
Response IP 172.67.68.169
Found Yes
Hash ba75fa8c896db51fa34f7f4a102dbdaf693f2b71111722e8490949367672ec99
SimHash ac10dd310691

Groups

*

Rule Path
Disallow /apollo-graphql

googlebot

Rule Path
Disallow /apollo-graphql

googlebot-image

Rule Path
Disallow /apollo-graphql

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://www.pme-legend.com/sitemap.xml

Comments

  • robots.txt
  • Website Sitemap
  • Crawlers Setup
  • Disallow baidu (chinse SEO)