avaluer.com
robots.txt

Robots Exclusion Standard data for avaluer.com

Resource Scan

Scan Details

Site Domain avaluer.com
Base Domain avaluer.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-14T19:13:57+00:00
Next Scan 2025-10-28T19:13:57+00:00

Last Successful Scan

Scanned2025-09-29T18:45:02+00:00
URL https://avaluer.com/robots.txt
Domain IPs 104.21.90.248, 172.67.163.88, 2606:4700:3032::6815:5af8, 2606:4700:3037::ac43:a358
Response IP 104.21.90.248
Found Yes
Hash 6eeafc393c681d72c10a00c3aa42d68805772369383163e4663c41e34c37000b
SimHash 0b7cc5c2ec98

Groups

mediapartners-google

Rule Path
Disallow /users/

*

Rule Path
Disallow /itmloader.php

ia_archiver
scoutjet
dotbot
spbot
netseer
downscout
seznam
exabot
bender
discobot
valueclick
lucidmedia
snapbot
sosospider
weesee
ahrefsbot
megaindex.ru
cliqzbot
semrushbot
mj12bot
blexbot
zumbot
sogou web spider

Rule Path
Disallow /explore
Disallow /search
Disallow /category

Comments

  • always allow adsense
  • All robots Block
  • bot-specific