detective-banque.fr
robots.txt

Robots Exclusion Standard data for detective-banque.fr

Resource Scan

Scan Details

Site Domain detective-banque.fr
Base Domain detective-banque.fr
Scan Status Ok
Last Scan2024-09-28T01:07:30+00:00
Next Scan 2024-10-05T01:07:30+00:00

Last Scan

Scanned2024-09-28T01:07:30+00:00
URL https://detective-banque.fr/robots.txt
Redirect https://www.detective-banque.fr/robots.txt
Redirect Domain www.detective-banque.fr
Redirect Base detective-banque.fr
Domain IPs 141.95.149.63
Redirect IPs 141.95.149.63
Response IP 141.95.149.63
Found Yes
Hash f28b6d1b7f0805e2022b13302b2de8ea4bb4cca9be1834c9e8d1ff2d051b7c5a
SimHash 30945c548a19

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /wp-login.php
Disallow /wp-admin
Disallow */trackback
Disallow */feed
Disallow */comments
Disallow /*.php
Disallow /*.inc
Disallow /*.gz
Disallow /*.swf
Disallow /*.wmv
Disallow /*.cgi
Disallow /*.xhtml
Disallow /wp-json/notificationx/v1/notice/?frontend=true
Allow /*.js$
Allow /*.css$
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

bard

Rule Path
Disallow /

sogou

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

cococrawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.detective-banque.fr/sitemap.xml

Comments

  • Blocking specific AI bots and common crawlers