Log In Sign Up

justfab.nl
robots.txt

Robots Exclusion Standard data for justfab.nl

Resource Scan

Scan Details

Site Domain justfab.nl
Base Domain justfab.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-12T01:00:47+00:00
Next Scan 2024-06-26T01:00:47+00:00

Last Successful Scan

Scanned2023-11-05T17:25:48+00:00
URL https://justfab.nl/robots.txt
Redirect https://www.justfab.nl/robots.txt
Redirect Domain www.justfab.nl
Redirect Base justfab.nl
Domain IPs 104.18.42.14, 172.64.145.242
Redirect IPs 104.18.42.14, 172.64.145.242
Response IP 104.18.42.14
Found Yes
Hash 6bb5a98ba3ee028894b57536b5f6461e2e87a502e595b6a34a46189a3a5d84ed
SimHash 7c40c568cb2a

Groups

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.8

*

Rule Path
Disallow /zapatos-tacon.htm?id_color_default=154
Disallow /zapatos-tacon.htm?id_color_default=120
Disallow /zapatos-tacon.htm?id_color_default=85
Disallow /cunas.htm?id_color_default=14
Disallow /zapatos-tacon.htm?id_color_default=80
Disallow /botas.htm?id_color_default=14en
Disallow /botas.htm?id_color_default=94
Disallow /botines.htm?id_color_default=135
Disallow /botas.htm?id_color_default=14
Disallow /sandalias.htm?id_color_default=113
Disallow /bolsos.htm?id_color_default=113
Disallow /botines.htm?id_color_default=14
Disallow /zapatos-tacon.htm?id_color_default=75
Disallow /cunas.htm?id_color_default=113
Disallow /*?id_color_default=*
Disallow /index.cfm?action=home.boutique
Disallow /index.cfm?action=boutique.home
Disallow /zapatos-mujer.htm?utm_source=tradedoubler&utm_medium=cpl&utm_campaign=affiliation&tduid=846b7aca442b3845978bd149f0164463
Disallow /zapatos-mujer.htm?utm_source=tradedoubler&utm_medium=cpl&utm_campaign=affiliation&tduid=603ef6c55927d73d9802072da4ba0933
Disallow /zapatos-mujer.htm?utm_source=tradedoubler&utm_medium=cpl&utm_campaign=affiliation&tduid=33791d83868b3a7429996cf0322611e0
Disallow /zapatos-mujer.htm?utm_source=tradedoubler&utm_medium=cpl&utm_campaign=affiliation&tduid=81f280fd5e1284adc1c5288cb589885e
Disallow /zapatos-mujer.htm?utm_source=tradedoubler&utm_medium=cpl&utm_campaign=affiliation&tduid=93215c00b248dd72b82da46523ce3c5d
Disallow /index.cfm?action=home.product_public_lp*
Disallow /index.cfm?action=reviews.view_product_review_summary_details*
Disallow /chaussures-femme.htm?utm_source=tradedoubler&utm_medium=cpl&utm_campaign=affiliation&tduid=b1ff6b5caac6f888c2d314e0484c8cd1
Disallow /chaussures-femme.htm?utm_source=tradedoubler&utm_medium=cpl&utm_campaign=affiliation&tduid=8faff3698782177522faaad3fc24cf76
Disallow /compensees.htm?id_color_default=80
Disallow /peep-toes.htm?id_color_default=113
Disallow /sacs.htm?id_color_default=14
Disallow /sandales.htm?id_color_default=210
Disallow /sandales.htm?id_color_default=89
Disallow /dms*/
Disallow /*?altoken=*
Disallow /*?alkey=*
Disallow /*?*&altoken=
Disallow /*?*&alkey=
Allow /dmg/F2564E/*
Allow /dmg/36BD6B/*
Allow /dmg/ABD186/*
Allow /dmg/1A079F/*
Allow /dmg/8D818E/*
Allow /dmg/83A445/*
Allow /dmg/83A445/*
Allow /dmg/BF8B4E/*
Allow /dmg/74F2AB/*
Allow /dmg/79807F/*
Back to top

Warnings

  • 3 invalid lines.
Back to top