photowall.de
robots.txt
Robots Exclusion Standard data for photowall.de
Resource Scan
Scan Details
Site Domain | photowall.de |
Base Domain | photowall.de |
Scan Status | Ok |
Last Scan | 2024-09-08T01:51:35+00:00 |
Next Scan | 2024-10-08T01:51:35+00:00 |
Last Scan
Scanned | 2024-09-08T01:51:35+00:00 |
URL | https://photowall.de/robots.txt |
Domain IPs | 52.16.222.149, 52.208.80.204 |
Response IP | 52.16.222.149 |
Found | Yes |
Hash | bc5a024a147327ce33e33addd303ebba71a52b4d6d12ca5711924e0283835e06 |
SimHash | f0586fb36f87 |
Groups
chatgpt-user
ccbot
perplexitybot
anthropic-ai
claudebot
claude-web
bytespider
applebot
alexa
bingbot/2.0
imagesiftbot
omgili
omgilibot
friendlycrawler
awariorssbot
awariosmartbot
dataforseobot
diffbot
img2dataset
magpie-crawler
meltwater
peer39_crawler
piplbot
scoop.it
seekr
youbot
cohere-ai
Rule | Path |
---|---|
Disallow | / |
*
Rule | Path |
---|---|
Disallow | /*? |
Disallow | /checkout/ |
Disallow | /*/checkout/ |
Disallow | /admin$ |
Disallow | /admin/ |
Disallow | /api$ |
Disallow | /api/ |
Disallow | /*/api$ |
Disallow | /*/api/ |
Disallow | /klarna-checkout |
Disallow | /*/klarna-checkout |
Disallow | /cart$ |
Disallow | /cart/ |
Disallow | /*/cart$ |
Disallow | /*/cart/ |
Warnings
- `useragent` is not a known field.
Comments