photowall.co.uk
robots.txt
Robots Exclusion Standard data for photowall.co.uk
Resource Scan
Scan Details
Site Domain | photowall.co.uk |
Base Domain | photowall.co.uk |
Scan Status | Ok |
Last Scan | 2024-06-22T06:38:29+00:00 |
Next Scan | 2024-07-22T06:38:29+00:00 |
Last Scan
Scanned | 2024-06-22T06:38:29+00:00 |
URL | https://photowall.co.uk/robots.txt |
Domain IPs | 34.250.41.33, 52.214.62.15 |
Response IP | 34.250.41.33 |
Found | Yes |
Hash | cb0114d023df613a2c93960c6a18c8beb1c751ac475677359ea01fcba4d552e0 |
SimHash | 74586d736e8d |
Groups
chatgpt-user
ccbot
perplexitybot
anthropic-ai
claudebot
claude-web
bytespider
applebot
alexa
bingbot/2.0
imagesiftbot
omgili
omgilibot
friendlycrawler
awariorssbot
awariosmartbot
dataforseobot
diffbot
img2dataset
magpie-crawler
meltwater
peer39_crawler
piplbot
scoop.it
seekr
youbot
cohere-ai
Rule | Path |
---|---|
Disallow | / |
*
Rule | Path |
---|---|
Disallow | /*? |
Disallow | /checkout/ |
Disallow | /*/checkout/ |
Disallow | /admin |
Disallow | /api |
Disallow | /*/api |
Disallow | /klarna-checkout |
Disallow | /*/klarna-checkout |
Disallow | /cart |
Disallow | /*/cart |
Warnings
- `useragent` is not a known field.
Comments