Log In Sign Up

mattanews.co
robots.txt

Robots Exclusion Standard data for mattanews.co

Resource Scan

Scan Details

Site Domain mattanews.co
Base Domain mattanews.co
Scan Status Ok
Last Scan2024-09-29T12:58:14+00:00
Next Scan 2024-10-06T12:58:14+00:00

Last Scan

Scanned2024-09-29T12:58:14+00:00
URL https://mattanews.co/robots.txt
Domain IPs 2a02:4780:6:1513:0:1679:db3f:2, 93.127.220.73
Response IP 93.127.220.73
Found Yes
Hash 9e916469f20634c059c793a4a6f98ab2bd9c8a9d9894fa5d77982b8e263907bd
SimHash 66a0d363001c

Groups

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /wp-content/uploads/

applebot

Rule Path
Allow /

yandex

Rule Path
Allow /

yandeximages

Rule Path
Allow /wp-content/uploads/

slurp

Rule Path
Allow /

*

Rule Path
Allow /ads.txt

*

Rule Path
Allow /app-ads.txt

*

Rule Path
Disallow /2020/12/wisata-tangkuban-perahu-bikin-deg-degan

Other Records

Field Value
crawl-delay 7
Back to top

Comments

  • Allow/Disallow Ads.txt
  • Allow/Disallow App-ads.txt
  • Manual rules with Better Robots.txt Post Meta Box
  • This robots.txt file was created by Better Robots.txt (Index & Rank Booster by Pagup) Plugin. https://www.better-robots.com/
Back to top

Warnings

  • 13 invalid lines.
Back to top