mpdeluxe.mingpao.com
robots.txt

Robots Exclusion Standard data for mpdeluxe.mingpao.com

Resource Scan

Scan Details

Site Domain mpdeluxe.mingpao.com
Base Domain mingpao.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2024-06-12T14:37:51+00:00
Next Scan 2024-07-12T14:37:51+00:00

Last Successful Scan

Scanned2024-04-21T06:55:03+00:00
URL https://mpdeluxe.mingpao.com/robots.txt
Domain IPs 202.80.6.130
Response IP 202.80.6.130
Found Yes
Hash 07e924bcbbef7b26567876f52942b4a6884b71033b038f1983765dd2bf0a27aa
SimHash 22104370cb83

Groups

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow

googlebot-mobile

Rule Path
Allow /

mediapartners-google

Rule Path
Allow

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

alexa

Rule Path
Allow /

indeedbot

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

jobdiggerspider

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

teoma

Rule Path
Disallow /

fast-webcrawler

Rule Path
Disallow /

gurujibot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

exabot

Rule Path
Disallow /

soso spider

Rule Path
Disallow /

dotbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

siteliner

Rule Path
Disallow /

curious george

Rule Path
Disallow /

grapeshot

Rule Path
Disallow

*

Rule Path
Disallow /cgi-bin/
Disallow /htm/dummy/
Disallow /m/htm/dummy/
Disallow /wp-admin/
Disallow /private*
Disallow /cgi-bin
Disallow /dat
Disallow /php/manage
Disallow /*/feed/
Allow /m/?display=wide
Allow /m/wp-content/uploads/
Disallow /m/wp-content/plugins/
Disallow /m/readme.html
Disallow /m/refer/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://mpdeluxe.mingpao.com/sitemap.xml

Warnings

  • 2 invalid lines.