Log In Sign Up

rolimons.com
robots.txt

Robots Exclusion Standard data for rolimons.com

Resource Scan

Scan Details

Site Domain rolimons.com
Base Domain rolimons.com
Scan Status Ok
Last Scan2024-06-25T20:44:50+00:00
Next Scan 2024-07-02T20:44:50+00:00

Last Scan

Scanned2024-06-25T20:44:50+00:00
URL https://rolimons.com/robots.txt
Redirect https://www.rolimons.com/robots.txt
Redirect Domain www.rolimons.com
Redirect Base rolimons.com
Domain IPs 104.26.8.58, 104.26.9.58, 172.67.74.181, 2606:4700:20::681a:83a, 2606:4700:20::681a:93a, 2606:4700:20::ac43:4ab5
Redirect IPs 104.26.8.58, 104.26.9.58, 172.67.74.181, 2606:4700:20::681a:83a, 2606:4700:20::681a:93a, 2606:4700:20::ac43:4ab5
Response IP 104.26.8.58
Found Yes
Hash 4a62416407da910183390849d3f399a7aeb86ff40d8dcd26307efea3ab9417df
SimHash 465196f0d2b1

Groups

*

Rule Path
Disallow
Disallow /shop

Other Records

Field Value
crawl-delay 2

adidxbot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Allow /

baiduspider-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

googlebot

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

gumgum

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

gumgum-bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

mediapartners-google

Rule Path
Allow /

mj12bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 20

msnbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Allow /

weborama

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

weborama-fetcher

Rule Path
Disallow /

yandex

Rule Path
Allow /
Disallow /uaid/
Disallow /history/

Other Records

Field Value
crawl-delay 10

bytespider

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.rolimons.com/sitemap.xml
Back to top