benbecker.eu
robots.txt
Robots Exclusion Standard data for benbecker.eu
Resource Scan
Scan Details
| Site Domain | benbecker.eu |
| Base Domain | benbecker.eu |
| Scan Status | Ok |
| Last Scan | 2026-02-10T04:36:50+00:00 |
| Next Scan | 2026-02-24T04:36:50+00:00 |
Last Scan
| Scanned | 2026-02-10T04:36:50+00:00 |
| URL | https://benbecker.eu/robots.txt |
| Domain IPs | 2a12:5040:1:302:58b5:25ff:fe2b:4e35, 45.152.249.10 |
| Response IP | 45.152.249.10 |
| Found | Yes |
| Hash | bc2d2e437a221caa25dd0d481e89a6e83ec2d2d0b2afd29b4917acc00e0d1c83 |
| SimHash | 522a4b41e61a |
Groups
ahrefsbot
aliyun
aliyunsecbot
amazonbot
anthropic-ai
applebot
baiduspider
barkrowler
blexbot
bytespider
ccbot
chatgpt-user
claudebot
crawlbot
dataforseobot
dotbot
duckduckgo
exabot
facebookexternalhit
facebookbot
facebot
freshbot
gptbot
grapeshot
ias-va
meta-externalagent
mj12bot
newsai
petalbot
pinterestbot
proximic
scrapy
sebot-wa
semrushbot
sentibot
seositecheckup
serankingbacklinksbot
sogou
telegrambot
tiktokspider
trendictionbot
twitterbot
verity
yandexbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /tmp/ |
| Disallow | /cache/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-login.php |
| Disallow | /private/ |
| Disallow | /admin/ |
| Disallow | /includes/ |
| Disallow | /config/ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 300 |
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /tmp/ |
| Disallow | /cache/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-login.php |
| Disallow | /private/ |
| Disallow | /admin/ |
| Disallow | /includes/ |
| Disallow | /config/ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Comments