us.fashionnetwork.com
robots.txt

Robots Exclusion Standard data for us.fashionnetwork.com

Resource Scan

Scan Details

Site Domain us.fashionnetwork.com
Base Domain fashionnetwork.com
Scan Status Ok
Last Scan2024-06-24T22:18:38+00:00
Next Scan 2024-07-08T22:18:38+00:00

Last Scan

Scanned2024-06-24T22:18:38+00:00
URL https://us.fashionnetwork.com/robots.txt
Domain IPs 104.22.6.170
Response IP 104.22.6.170
Found Yes
Hash 1d326ddded4f88d7679e96b07baa690b55561877f561e42c284e943a53ee1dc4
SimHash 07188d1de003

Groups

amazonbot

Rule Path
Disallow /

amazon-kendra-web-crawler-*

Rule Path
Disallow /

applebot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

steeler

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

*

Rule Path
Disallow /*affiche_imprime_annonce.php*
Disallow /*popup%3D*/
Disallow /*redirect.php*
Disallow /*send_friend_news.php*
Disallow /*version_pdf.php*
Disallow /annuaire/
Disallow /account/
Disallow /agenda/*
Disallow /ajax/
Disallow /contact/
Disallow /clickheat/click.php
Disallow /dashboard/
Disallow /documentation/
Disallow /download/
Disallow /fashion_build/js/l-js-optimized*
Disallow /fashion_build/js/l-js-optimized*
Disallow /flux/
Disallow /fluxNews/
Disallow /include/
Disallow /keysearch.jsp*
Disallow /login/*
Disallow /membres/removefromnewsletter/*
Disallow /mobile.php*
Disallow /network/
Disallow /news/inscr_friend.php
Disallow /news/downloadPdf/*
Disallow /newsletters/*
Disallow /nl.php*
Disallow /nl/*
Disallow /parrainer/
Disallow /popup/
Disallow /redirect.php*
Disallow /redirect_news_international.php*
Disallow /register/
Disallow /rss/feed/
Disallow /search/
Disallow /societes/
Disallow /texte/*
Disallow /traduction/
Disallow /transaction/
Disallow /news/articles.php*
Disallow /news/print.php*
Disallow /news/news.php*
Disallow /news/news_list.php*
Disallow /news/rss.php
Disallow /news/statistiques.php
Disallow /news/search.php*
Disallow /news/video_list.php*
Disallow /news/video_view.php*
Disallow /photos/catwalks.php
Disallow /photos/photo.php*
Disallow /photos/photo_list.php*
Disallow /photos/photo_arbo.php*
Disallow /tendance/tendance.php*
Disallow /tendance/tendance_list.php*
Disallow /contact/validForm/
Disallow /fragments/*

Comments

  • To be purged into Google's cache
  • Ajax URIs
  • Fragments