adjaranet.to
robots.txt

Robots Exclusion Standard data for adjaranet.to

Resource Scan

Scan Details

Site Domain adjaranet.to
Base Domain adjaranet.to
Scan Status Ok
Last Scan2025-07-24T09:09:44+00:00
Next Scan 2025-07-31T09:09:44+00:00

Last Scan

Scanned2025-07-24T09:09:44+00:00
URL https://adjaranet.to/robots.txt
Redirect https://adjaranets.to/robots.txt
Redirect Domain adjaranets.to
Redirect Base adjaranets.to
Domain IPs 104.21.50.28, 172.67.155.219, 2606:4700:3033::6815:321c, 2606:4700:3037::ac43:9bdb
Redirect IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.112.1
Found Yes
Hash 5fec669b9eca66c0edf9032ab353c603665ad8ff4ab2ad93477bb3331d6f5216
SimHash 5d0d9470ab31

Groups

*

Rule Path
Disallow /engine/go.php
Disallow /user/
Disallow /newposts/
Disallow /statistics.html
Disallow /*subaction%3Duserinfo
Disallow /*subaction%3Dnewposts
Disallow /*do%3Dlastcomments
Disallow /*do%3Dfeedback
Disallow /*do%3Dregister
Disallow /*do%3Dlostpassword
Disallow /*do%3Daddnews
Disallow /*do%3Dstats
Disallow /*do%3Dpm
Disallow /*do%3Dsearch
Disallow /*do%3Ddownload
Disallow /*do%3Dgo

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

barkrowler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

Other Records

Field Value
sitemap https://adjaranets.to/sitemap.xml
sitemap https://adjaranets.to/news_pages.xml
sitemap https://adjaranets.to/category_pages.xml
sitemap https://adjaranets.to/tags_pages.xml