borjak.net
robots.txt

Robots Exclusion Standard data for borjak.net

Resource Scan

Scan Details

Site Domain borjak.net
Base Domain borjak.net
Scan Status Ok
Last Scan2024-09-26T16:22:45+00:00
Next Scan 2024-10-03T16:22:45+00:00

Last Scan

Scanned2024-09-26T16:22:45+00:00
URL https://borjak.net/robots.txt
Domain IPs 104.21.51.29, 172.67.220.6, 2606:4700:3032::ac43:dc06, 2606:4700:3036::6815:331d
Response IP 172.67.220.6
Found Yes
Hash 97f30ccca27efd69ebec93a0f3af6f7293136d6ad4a916e0b25836864e5570cb
SimHash 47004b865695

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Allow /wp-admin/admin-ajax.php

baiduspider

Rule Path
Disallow

baiduspider-image

Rule Path
Disallow

baiduspider-mobile

Rule Path
Disallow

baiduspider-news

Rule Path
Disallow

baiduspider-video

Rule Path
Disallow

bingbot

Rule Path
Disallow

msnbot

Rule Path
Disallow

msnbot-media

Rule Path
Disallow

adidxbot

Rule Path
Disallow

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

googlebot-video

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

slurp

Rule Path
Disallow

yandex

Rule Path
Disallow

facebot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

petalbot

Rule Path
Disallow

Other Records

Field Value
sitemap https://borjak.net/sitemap_index.xml