webyes.com.br
robots.txt

Robots Exclusion Standard data for webyes.com.br

Resource Scan

Scan Details

Site Domain webyes.com.br
Base Domain webyes.com.br
Scan Status Ok
Last Scan2024-05-19T08:05:06+00:00
Next Scan 2024-05-26T08:05:06+00:00

Last Scan

Scanned2024-05-19T08:05:06+00:00
URL https://webyes.com.br/robots.txt
Domain IPs 104.21.3.85, 172.67.130.129, 2606:4700:3031::6815:355, 2606:4700:3033::ac43:8281
Response IP 172.67.130.129
Found Yes
Hash 7b1b3b27c8d0f32a7301f914a5d5960d635a41cc556f3890798a057837dafba4
SimHash 4955fa53e674

Groups

scrapy

Rule Path
Allow /

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /cgi-bin
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /wp-content/themes
Disallow /wusage
Disallow /trackback
Disallow /feed
Disallow /comments
Disallow /category/*/*
Disallow */trackback
Disallow */feed
Disallow /*/feed/rss/$
Disallow /category/*
Disallow */comments
Disallow /*?*
Disallow /*?
Allow /wp-content/uploads

googlebot-image

Rule Path
Disallow
Allow /*

duggmirror

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

Comments

  • Google Image
  • digg mirror