panotreffit.com
robots.txt

Robots Exclusion Standard data for panotreffit.com

Resource Scan

Scan Details

Site Domain panotreffit.com
Base Domain panotreffit.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-03-22T16:44:14+00:00
Next Scan 2024-06-20T16:44:14+00:00

Last Successful Scan

Scanned2023-08-02T22:17:14+00:00
URL https://panotreffit.com/robots.txt
Domain IPs 104.21.76.188, 172.67.199.112, 2606:4700:3031::6815:4cbc, 2606:4700:3034::ac43:c770
Response IP 172.67.199.112
Found Yes
Hash 8a1e9b0a37685fddc179b63bae71e129b2105e49214175d63d0797e5b392d4f8
SimHash 1075c906ed87

Groups

googlebot-image

Rule Path
Disallow /

*

Rule Path
Disallow /terms
Disallow /data-protection
Disallow /rules
Disallow /contact
Disallow /register
Disallow /search
Disallow /login

panscient.com

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

webprosbot

Rule Path
Disallow /

pandalytics

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

netsystemsresearch

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefsbot/3.1

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefsbot/7.0

Rule Path
Disallow /

ahrefsbot/2.0

Rule Path
Disallow /

ahrefs

Rule Path
Disallow /

ahrefs.com<http://ahrefs.com>

Rule Path
Disallow /

http://ahrefs.com/robot/

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

xenu’s

Rule Path
Disallow /

xenu’s link sleuth 1.1c

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

ttd-content

Rule Path
Disallow /

netestate ne crawler (+http://www.website-datenbank.de/)

Rule Path
Disallow /

picscout

Rule Path
Disallow /

tineye

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

archive.org bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

Comments

  • robotstxt.org