Log In Sign Up

djfrenchy.be
robots.txt

Robots Exclusion Standard data for djfrenchy.be

Resource Scan

Scan Details

Site Domain djfrenchy.be
Base Domain djfrenchy.be
Scan Status Ok
Last Scan2024-11-11T07:42:38+00:00
Next Scan 2024-11-18T07:42:38+00:00

Last Scan

Scanned2024-11-11T07:42:38+00:00
URL https://djfrenchy.be/robots.txt
Domain IPs 46.105.204.2
Response IP 46.105.204.2
Found Yes
Hash 737321b1dd7c00139d7f6b2a0097d1b378a89523912ebc78dc8dcbdd14db7a1c
SimHash 714bd4c27eb0

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /search/
Disallow /wp-admin/
Disallow /*public_html/
Disallow /*index.php?

Other Records

Field Value
crawl-delay 30

semrushbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

feedburner

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://djfrenchy.com/sitemap.xml
Back to top

Comments

  • Robots.txt generated by Booter - Crwalers Manager at Fri, 16 Feb 2024 09:47:52 +0000
  • Do not modify this file, any contents will be overwritten
  • Bloquer certains bots malveillants
Back to top