Log In Sign Up

cned.fr
robots.txt

Robots Exclusion Standard data for cned.fr

Resource Scan

Scan Details

Site Domain cned.fr
Base Domain cned.fr
Scan Status Ok
Last Scan2024-05-24T23:31:38+00:00
Next Scan 2024-06-23T23:31:38+00:00

Last Scan

Scanned2024-05-24T23:31:38+00:00
URL https://cned.fr/robots.txt
Redirect https://www.cned.fr/robots.txt
Redirect Domain www.cned.fr
Redirect Base cned.fr
Redirect IPs 51.11.239.227
Response IP 51.11.239.227
Found Yes
Hash d6e18862240b7420a0920c3c02e4ab40def8a3d3602b63ed61297d9620a74456
SimHash 2c941d19c544

Groups

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-mobile

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-sfkr

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexmedia

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexcatalog

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

yandexblogs

Rule Path
Disallow /

yandexnews

Rule Path
Disallow /

pagechecker

Rule Path
Disallow /

googlebot

Rule Path
Disallow /*.doc
Disallow /*.docx
Disallow /*.xls
Disallow /*.xlsx
Disallow /*.ppt
Disallow /*.pptx
Disallow /*.svg

*

Rule Path
Disallow /node/
Disallow /index.php/
Back to top

Other Records

Field Value
sitemap https://www.cned.fr/sitemap.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
Back to top