cyberpunkdatabase.net
robots.txt
Robots Exclusion Standard data for cyberpunkdatabase.net
Resource Scan
Scan Details
| Site Domain | cyberpunkdatabase.net |
| Base Domain | cyberpunkdatabase.net |
| Scan Status | Ok |
| Last Scan | 2025-10-21T17:04:51+00:00 |
| Next Scan | 2025-11-04T17:04:51+00:00 |
Last Scan
| Scanned | 2025-10-21T17:04:51+00:00 |
| URL | https://cyberpunkdatabase.net/robots.txt |
| Domain IPs | 193.121.15.225, 2a02:a03f:8b33:3d00::cafe |
| Response IP | 193.121.15.225 |
| Found | Yes |
| Hash | 0d9373b7bc1586788ff3021f84b4732d4ca0d4cb9d533808a2c31557c777877b |
| SimHash | b0575730e490 |
Groups
semrushbot
semrushbot-ba
semrushbot-si
semrushbot
ahrefsbot
bytespider
adsbot-google
baiduspider
facebookexternalhit
facebookexternalhit
facebookcatalog
meta-externalagent
meta-externalads
meta-externalfetcher
meta-webindexer
applebot
petalbot
amazonbot
serpstatbot
blexbot
barkrowler
mj12bot
oai-searchbot
seznambot
dataforseobot
buck
tiktokspider
awariobot
criteobot
awakari
grammarly
seokicks
imagespider
thinkbot
bitsightbot
trendictionbot0.5.0
trendictionbot
odin
exabot
semanticscholarbot
terracotta
aliyunsecbot/aliyun
aliyunsecbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /dl/ |
| Disallow | /img/movies/ |
| Disallow | /img/games/ |
| Disallow | /img/music/ |
| Disallow | /img/books/ |
| Allow | / |
Warnings
- `disallowaitraining` is not a known field.
- `license` is not a known field.