howtodo.rocks
robots.txt

Robots Exclusion Standard data for howtodo.rocks

Resource Scan

Scan Details

Site Domain howtodo.rocks
Base Domain howtodo.rocks
Scan Status Ok
Last Scan2024-11-14T13:52:26+00:00
Next Scan 2024-11-21T13:52:26+00:00

Last Scan

Scanned2024-11-14T13:52:26+00:00
URL https://howtodo.rocks/robots.txt
Domain IPs 104.21.27.53, 172.67.141.116, 2606:4700:3033::ac43:8d74, 2606:4700:3035::6815:1b35
Response IP 172.67.141.116
Found Yes
Hash 4a998254ce9ea1098e874b8234ec56f14ed754637bc35a718fefb158d3e21614
SimHash 489e5d5a7790

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /wp-admin/
Disallow /forum/admin/
Disallow /forum/auth/
Disallow /forum/assets/browser-update*.js
Disallow /forum/email/
Disallow /forum/session/
Disallow /forum/user-api-key/
Disallow /forum/*?api_key*
Disallow /forum/*?*api_key*
Disallow /forum/badges/
Disallow /forum/u/
Disallow /forum/my/
Disallow /forum/search/
Disallow /forum/tag/*/l/
Disallow /forum/g/
Disallow /forum/t/*/*.rss
Disallow /forum/c/*.rss
Disallow /cdn-cgi/

mauibot

Rule Path
Disallow /forum/

semrushbot

Rule Path
Disallow /forum/

ahrefsbot

Rule Path
Disallow /forum/

blexbot

Rule Path
Disallow /forum/

seo spider

Rule Path
Disallow /forum/

googlebot

Rule Path
Disallow /forum/admin/
Disallow /forum/auth/
Disallow /forum/assets/browser-update*.js
Disallow /forum/email/
Disallow /forum/session/
Disallow /forum/user-api-key/
Disallow /forum/*?api_key*
Disallow /forum/*?*api_key*

mj12bot

Rule Path
Disallow /*

ia_archiver

Rule Path
Disallow /*

dotbot

Rule Path
Disallow /*

Other Records

Field Value
sitemap https://howtodo.rocks/sitemap.xml