photowall.fr
robots.txt
Robots Exclusion Standard data for photowall.fr
Resource Scan
Scan Details
Site Domain | photowall.fr |
Base Domain | photowall.fr |
Scan Status | Ok |
Last Scan | 2024-06-07T08:37:32+00:00 |
Next Scan | 2024-07-07T08:37:32+00:00 |
Last Scan
Scanned | 2024-06-07T08:37:32+00:00 |
URL | https://photowall.fr/robots.txt |
Domain IPs | 52.211.219.59, 52.214.62.15 |
Response IP | 52.211.219.59 |
Found | Yes |
Hash | cb0114d023df613a2c93960c6a18c8beb1c751ac475677359ea01fcba4d552e0 |
SimHash | 74586d736e8d |
Groups
chatgpt-user
ccbot
perplexitybot
anthropic-ai
claudebot
claude-web
bytespider
applebot
alexa
bingbot/2.0
imagesiftbot
omgili
omgilibot
friendlycrawler
awariorssbot
awariosmartbot
dataforseobot
diffbot
img2dataset
magpie-crawler
meltwater
peer39_crawler
piplbot
scoop.it
seekr
youbot
cohere-ai
Rule | Path |
---|---|
Disallow | / |
*
Rule | Path |
---|---|
Disallow | /*? |
Disallow | /checkout/ |
Disallow | /*/checkout/ |
Disallow | /admin |
Disallow | /api |
Disallow | /*/api |
Disallow | /klarna-checkout |
Disallow | /*/klarna-checkout |
Disallow | /cart |
Disallow | /*/cart |
Warnings
- `useragent` is not a known field.
Comments