novica.com
robots.txt

Robots Exclusion Standard data for novica.com

Resource Scan

Scan Details

Site Domain novica.com
Base Domain novica.com
Scan Status Ok
Last Scan2024-10-09T18:17:35+00:00
Next Scan 2024-10-16T18:17:35+00:00

Last Scan

Scanned2024-10-09T18:17:35+00:00
URL https://novica.com/robots.txt
Redirect https://www.novica.com/robots.txt
Redirect Domain www.novica.com
Redirect Base novica.com
Domain IPs 152.199.38.85
Redirect IPs 152.199.38.85
Response IP 152.199.38.85
Found Yes
Hash 574fbfae3d88b056ee579a4a47e9468351e481ed78b60ed900fa81ca7b2eef10
SimHash 71091cf5e330

Groups

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.5

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /de/*
Disallow /cart/*
Disallow /*/cart/*
Disallow /checkout/*
Disallow /*/checkout/*
Disallow /custom-gift-sets/*
Disallow /*/custom-gift-sets/*
Disallow /ebay/*
Disallow /*/ebay/*
Disallow /lp/*
Disallow /*/lp/*
Disallow /mynovica/*
Disallow /*/mynovica/*
Disallow /me/*
Disallow /*/me/*
Disallow /shipguar/holidayshipping*
Disallow /*/shipguar/holidayshipping*
Disallow /*/pg-sortby-*/*
Disallow /*/s/*
Disallow /fetch/*
Disallow /*/fetch/*
Disallow /*?action=session.createusersession*
Disallow /*%26action%3Dsession.createusersession*

Other Records

Field Value
sitemap https://www.novica.com/sitemaps/sitemap-index.xml