macmagazine.com.br
robots.txt

Robots Exclusion Standard data for macmagazine.com.br

Resource Scan

Scan Details

Site Domain macmagazine.com.br
Base Domain macmagazine.com.br
Scan Status Ok
Last Scan2024-11-16T19:59:16+00:00
Next Scan 2024-11-23T19:59:16+00:00

Last Scan

Scanned2024-11-16T19:59:16+00:00
URL https://macmagazine.com.br/robots.txt
Domain IPs 104.26.0.126, 104.26.1.126, 172.67.74.114, 2606:4700:20::681a:17e, 2606:4700:20::681a:7e, 2606:4700:20::ac43:4a72
Response IP 104.26.1.126
Found Yes
Hash 6a85515fb133551d61939fb5d445206e7d7c5d2dfddf52741e7f12e2ccd74612
SimHash 4a2a5ec2a633

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-login.php
Disallow /xmlrpc.php
Disallow /*wp-login.php*
Disallow /*/trackback
Disallow /*/feed/
Disallow /*/comments
Disallow /stats/
Disallow /dh_
Disallow /login/
Disallow /?log=
Disallow /?s=
Disallow /*.php$
Disallow /*.cgi$
Disallow /*.xhtml$
Disallow /*.inc$
Disallow /*.gz$
Disallow /*.wmv$
Allow /wp-content/cache/
Allow /wp-content/plugins/
Allow /wp-content/uploads/
Allow /wp-includes/js/
Allow /*.js
Allow /*.css

duggmirror

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /wp-includes/
Allow /

mediapartners-google

Rule Path
Disallow
Allow /

yandex

Rule Path
Disallow /

yandexblogs

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

Other Records

Field Value
sitemap https://macmagazine.com.br/sitemap_index.xml

Comments

  • Sitemap
  • Global rules
  • Specific user-agent rules