dsibe.com
robots.txt
Robots Exclusion Standard data for dsibe.com
Resource Scan
Scan Details
| Site Domain | dsibe.com |
| Base Domain | dsibe.com |
| Scan Status | Ok |
| Last Scan | 2025-05-25T20:35:21+00:00 |
| Next Scan | 2025-06-24T20:35:21+00:00 |
Last Scan
| Scanned | 2025-05-25T20:35:21+00:00 |
| URL | https://www.dsibe.com/robots.txt |
| Domain IPs | 18.161.97.18, 18.161.97.4, 18.161.97.8, 18.161.97.97, 2600:9000:256f:3400:18:e7c7:b940:93a1, 2600:9000:256f:4200:18:e7c7:b940:93a1, 2600:9000:256f:8c00:18:e7c7:b940:93a1, 2600:9000:256f:9800:18:e7c7:b940:93a1, 2600:9000:256f:ce00:18:e7c7:b940:93a1, 2600:9000:256f:e000:18:e7c7:b940:93a1, 2600:9000:256f:ee00:18:e7c7:b940:93a1, 2600:9000:256f:f600:18:e7c7:b940:93a1 |
| Response IP | 52.85.49.17 |
| Found | Yes |
| Hash | 5498e4384274a2598d4d6a2b676892a92808e243d4c0fc00550d1213d426a482 |
| SimHash | 731e0101c1d4 |
Groups
ai2bot
ai2bot-dolma
amazonbot
applebot
applebot-extended
bytespider
ccbot
chatgpt-user
claude-web
claudebot
diffbot
facebookbot
friendlycrawler
gptbot
google-extended
googleother
googleother-image
googleother-video
icc-crawler
isscyberriskcrawler
imagesiftbot
kangaroo bot
meta-externalagent
meta-externalfetcher
mj12bot
oai-searchbot
perplexitybot
petalbot
scrapy
sidetrade indexer bot
timpibot
velenpublicwebcrawler
webzio-extended
youbot
anthropic-ai
cohere-ai
facebookexternalhit
iaskspider/2.0
img2dataset
omgili
omgilibot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /de/pdf |
Comments