beautyepic.com
robots.txt

Robots Exclusion Standard data for beautyepic.com

Resource Scan

Scan Details

Site Domain beautyepic.com
Base Domain beautyepic.com
Scan Status Ok
Last Scan2024-11-13T09:08:02+00:00
Next Scan 2024-11-20T09:08:02+00:00

Last Scan

Scanned2024-11-13T09:08:02+00:00
URL https://beautyepic.com/robots.txt
Domain IPs 104.21.49.178, 172.67.165.131, 2606:4700:3033::ac43:a583, 2606:4700:3034::6815:31b2
Response IP 172.67.165.131
Found Yes
Hash 3f6326b14fa21d3eb5bc45c3ae9d2c75e552a35fa8790311d578d9115f0f734b
SimHash 6754915b7fb5

Groups

*

Product Comment
* applies to all robots
Rule Path
Disallow /tag/
Disallow /latest-news-2/
Disallow /homepage-2/
Disallow /category/
Disallow /trackback/
Disallow */comments$
Disallow */trackback
Disallow */trackback$
Disallow /wp-comments
Disallow /wp-feed
Disallow /wp-trackback
Disallow */replytocom%3D

mj12bot

Rule Path
Disallow

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

boomtrain-content-bot*

Rule Path
Disallow
Allow /*

Other Records

Field Value
sitemap https://www.beautyepic.com/sitemap_index.xml