Log In Sign Up

frivlyz.com
robots.txt

Robots Exclusion Standard data for frivlyz.com

Resource Scan

Scan Details

Site Domain frivlyz.com
Base Domain frivlyz.com
Scan Status Ok
Last Scan2026-02-02T19:47:29+00:00
Next Scan 2026-02-09T19:47:29+00:00

Last Scan

Scanned2026-02-02T19:47:29+00:00
URL https://frivlyz.com/robots.txt
Redirect https://www.frivlyz.com/robots.txt
Redirect Domain www.frivlyz.com
Redirect Base frivlyz.com
Domain IPs 104.18.6.59, 104.18.7.59, 2606:4700::6812:63b, 2606:4700::6812:73b
Redirect IPs 104.18.6.59, 104.18.7.59, 2606:4700::6812:63b, 2606:4700::6812:73b
Response IP 104.18.6.59
Found Yes
Hash 652713be4ce2f15e6b280bbe3c59eecad9202847d7e53672575a405f28c235b0
SimHash 094cc813af11

Groups

ias_crawler
ias_wombles

Rule Path
Allow /

*

Rule Path
Disallow /?download=*
Disallow /?s=*

baiduspider*
sogou*

Rule Path
Disallow /

ahrefsbot
blexbot
ccbot
cliqzbot
dotbot
ia_archiver
mbcrawler
mj12bot
photon
scrapy
semrushbot
linguee

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.frivlyz.com/sitemaps/sitemap.xml
Back to top