smartoptions.io
robots.txt

Robots Exclusion Standard data for smartoptions.io

Resource Scan

Scan Details

Site Domain smartoptions.io
Base Domain smartoptions.io
Scan Status Ok
Last Scan2024-06-28T20:17:46+00:00
Next Scan 2024-07-28T20:17:46+00:00

Last Scan

Scanned2024-06-28T20:17:46+00:00
URL https://smartoptions.io/robots.txt
Domain IPs 104.26.4.101, 104.26.5.101, 172.67.71.107, 2606:4700:20::681a:465, 2606:4700:20::681a:565, 2606:4700:20::ac43:476b
Response IP 104.26.5.101
Found Yes
Hash ceef1eac1ced174499bd81fe7ab485089b3b969adff88f0c42e7c81ad774ff0a
SimHash 6a201293f5b5

Groups

*

Rule Path
Disallow

googlebot
googlebot-image
googlebot-mobile
twitterbot
semrushbot
semrushbot-ct
bingbot
slurp
msnbot
yandex

Rule Path Comment
Allow /wp-admin/admin-ajax.php -
Disallow /wp-admin/ block access to admin section
Disallow /wp-login.php block access to admin section
Disallow /search/ block access to internal search result pages
Disallow *?s=* block access to internal search result pages
Disallow *?p=* block access to pages for which permalinks fails
Disallow *%26p%3D* block access to pages for which permalinks fails
Disallow *%26preview%3D* block access to preview pages
Disallow /tag/ block access to tag pages
Disallow /404-error/ block access to 404 page

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://smartoptions.io/sitemap_index.xml