gdesobiraut.com
robots.txt

Robots Exclusion Standard data for gdesobiraut.com

Resource Scan

Scan Details

Site Domain gdesobiraut.com
Base Domain gdesobiraut.com
Scan Status Ok
Last Scan2024-11-13T01:44:09+00:00
Next Scan 2024-11-20T01:44:09+00:00

Last Scan

Scanned2024-11-13T01:44:09+00:00
URL https://gdesobiraut.com/robots.txt
Domain IPs 185.68.16.197, 2a00:7a60:0:10c5::1
Response IP 185.68.16.197
Found Yes
Hash 304cfe562a930d4eb840258216da91b0aff5a84138d8c22c80a5b85fdf589924
SimHash 285cc910c390

Groups

*

Rule Path
Disallow /page/
Disallow /comments/feed/
Disallow /comments/
Disallow /plugins/
Disallow /attachment/
Disallow /attachment
Disallow /wp-content
Disallow */attachment/

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

semrushbot-sa

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

alphaseobot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

alphaseobot-sa

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://gdesobiraut.com/sitemap.xml

Warnings

  • `host` is not a known field.