public.fr
robots.txt

Robots Exclusion Standard data for public.fr

Resource Scan

Scan Details

Site Domain public.fr
Base Domain public.fr
Scan Status Ok
Last Scan2024-11-09T20:22:03+00:00
Next Scan 2024-11-16T20:22:03+00:00

Last Scan

Scanned2024-11-09T20:22:03+00:00
URL https://public.fr/robots.txt
Redirect https://www.public.fr/robots.txt
Redirect Domain www.public.fr
Redirect Base public.fr
Domain IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Redirect IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Response IP 151.101.129.91
Found Yes
Hash ea81970629642e3c700244b4a1b4d8825d36537c4e9fc1147818200bef3a0f8e
SimHash 3981de30e803

Groups

*
*

Rule Path
Disallow /*/wp-*.php
Disallow /*/trackback/
Disallow /*/xmlrpc.php
Disallow /*/*/amp
Disallow /*/*?amp
Disallow /*?wptheme=
Disallow /*?comments=
Disallow /*?replytocom
Disallow /*?*preview_id=*
Disallow /serviceworker.js*
Disallow /*/page/*?s=
Disallow /?rest_route=
Disallow /*?s=
Disallow /search/*/
Disallow /wp-json/
Disallow /wp-includes/
Disallow /wp-content/mu-plugins/
Disallow /wp-content/cache/
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Allow /wp-includes/js
Disallow /wp-content/themes/
Allow /wp-content/themes/*/assets/
Allow /wp-content/themes/**/*.css
Allow /wp-content/themes/**/*.js
Allow /wp-content/themes/**/*.gif
Disallow /wp-content/plugins/
Allow /wp-content/plugins/**/*.css
Allow /wp-content/plugins/**/*.js
Allow /wp-content/plugins/**/*.gif

meltwater
gptbot
chatgpt-user
semrushbot
claudebot
claude-web
synthesio
copyrightcheck
ahrefsbot
deepcrawl

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.public.fr/sitemap_index.xml
sitemap https://www.public.fr/sitemap-news.xml

Comments

  • Disallow bots