photowall.se
robots.txt
Robots Exclusion Standard data for photowall.se
Resource Scan
Scan Details
Site Domain | photowall.se |
Base Domain | photowall.se |
Scan Status | Ok |
Last Scan | 2024-06-23T03:20:29+00:00 |
Next Scan | 2024-07-23T03:20:29+00:00 |
Last Scan
Scanned | 2024-06-23T03:20:29+00:00 |
URL | https://photowall.se/robots.txt |
Domain IPs | 54.194.9.22, 63.34.123.240 |
Response IP | 54.194.9.22 |
Found | Yes |
Hash | cb0114d023df613a2c93960c6a18c8beb1c751ac475677359ea01fcba4d552e0 |
SimHash | 74586d736e8d |
Groups
chatgpt-user
ccbot
perplexitybot
anthropic-ai
claudebot
claude-web
bytespider
applebot
alexa
bingbot/2.0
imagesiftbot
omgili
omgilibot
friendlycrawler
awariorssbot
awariosmartbot
dataforseobot
diffbot
img2dataset
magpie-crawler
meltwater
peer39_crawler
piplbot
scoop.it
seekr
youbot
cohere-ai
Rule | Path |
---|---|
Disallow | / |
*
Rule | Path |
---|---|
Disallow | /*? |
Disallow | /checkout/ |
Disallow | /*/checkout/ |
Disallow | /admin |
Disallow | /api |
Disallow | /*/api |
Disallow | /klarna-checkout |
Disallow | /*/klarna-checkout |
Disallow | /cart |
Disallow | /*/cart |
Warnings
- `useragent` is not a known field.
Comments