Log In Sign Up

jeremy.hu
robots.txt

Robots Exclusion Standard data for jeremy.hu

Resource Scan

Scan Details

Site Domain jeremy.hu
Base Domain jeremy.hu
Scan Status Ok
Last Scan2025-07-20T15:13:15+00:00
Next Scan 2025-07-21T15:13:15+00:00

Last Scan

Scanned2025-07-20T15:13:15+00:00
URL https://jeremy.hu/robots.txt
Redirect https://herve.bzh/robots.txt
Redirect Domain herve.bzh
Redirect Base herve.bzh
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Redirect IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.96.1
Found Yes
Hash 5eb51be2aa7df6ab370b9016b07afca33a9c0996fe9903bbf2740da5441256e0
SimHash 60206b40a116

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow */feed

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://herve.bzh/sitemap.xml
sitemap https://herve.bzh/news-sitemap.xml
Back to top

Comments

  • Block AI scraping bots.
Back to top