clickbait.pl
robots.txt
Robots Exclusion Standard data for clickbait.pl
Resource Scan
Scan Details
| Site Domain | clickbait.pl |
| Base Domain | clickbait.pl |
| Scan Status | Ok |
| Last Scan | 2025-09-21T01:05:10+00:00 |
| Next Scan | 2025-10-21T01:05:10+00:00 |
Last Scan
| Scanned | 2025-09-21T01:05:10+00:00 |
| URL | https://clickbait.pl/robots.txt |
| Domain IPs | 104.21.69.191, 172.67.212.52, 2606:4700:3030::6815:45bf, 2606:4700:3035::ac43:d434 |
| Response IP | 104.21.69.191 |
| Found | Yes |
| Hash | c6bd35a23a1e5a2f88826961c287edab23e03216e7670576f85420a302159fe9 |
| SimHash | 711479cb8702 |
Groups
gptbot
ccbot
sentibot
sentibot
google-extended
facebookbot
omgili
omgilibot
archive.org_bot
ia_archiver
amazonbot
bingbot
amazonbot
applebot
applebot-extended
bytespider
chatgpt-user
claude-web
claudebot
diffbot
httrack
nutch
oai-searchbot
offline explorer
perplexitybot
scrapy
youbot
anthropic-ai
cohere-ai
| Rule | Path |
|---|---|
| Disallow | / |
Warnings
- 1 invalid line.