designtopics.nl
robots.txt
Robots Exclusion Standard data for designtopics.nl
Resource Scan
Scan Details
| Site Domain | designtopics.nl |
| Base Domain | designtopics.nl |
| Scan Status | Ok |
| Last Scan | 2026-02-19T22:46:21+00:00 |
| Next Scan | 2026-03-05T22:46:21+00:00 |
Last Scan
| Scanned | 2026-02-19T22:46:21+00:00 |
| URL | https://designtopics.nl/robots.txt |
| Domain IPs | 2a12:5040:1:303:be24:11ff:fe13:d44b, 45.152.250.88 |
| Response IP | 45.152.250.88 |
| Found | Yes |
| Hash | bc2d2e437a221caa25dd0d481e89a6e83ec2d2d0b2afd29b4917acc00e0d1c83 |
| SimHash | 522a4b41e61a |
Groups
ahrefsbot
aliyun
aliyunsecbot
amazonbot
anthropic-ai
applebot
baiduspider
barkrowler
blexbot
bytespider
ccbot
chatgpt-user
claudebot
crawlbot
dataforseobot
dotbot
duckduckgo
exabot
facebookexternalhit
facebookbot
facebot
freshbot
gptbot
grapeshot
ias-va
meta-externalagent
mj12bot
newsai
petalbot
pinterestbot
proximic
scrapy
sebot-wa
semrushbot
sentibot
seositecheckup
serankingbacklinksbot
sogou
telegrambot
tiktokspider
trendictionbot
twitterbot
verity
yandexbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /tmp/ |
| Disallow | /cache/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-login.php |
| Disallow | /private/ |
| Disallow | /admin/ |
| Disallow | /includes/ |
| Disallow | /config/ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 300 |
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /tmp/ |
| Disallow | /cache/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-login.php |
| Disallow | /private/ |
| Disallow | /admin/ |
| Disallow | /includes/ |
| Disallow | /config/ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Comments