Log In Sign Up

fresh-galleries.com
robots.txt

Robots Exclusion Standard data for fresh-galleries.com

Resource Scan

Scan Details

Site Domain fresh-galleries.com
Base Domain fresh-galleries.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-11T04:16:01+00:00
Next Scan 2026-04-11T04:16:01+00:00

Last Successful Scan

Scanned2025-02-23T00:53:21+00:00
URL https://fresh-galleries.com/robots.txt
Domain IPs 104.21.24.15, 172.67.216.73, 2606:4700:3032::6815:180f, 2606:4700:3033::ac43:d849
Response IP 172.67.216.73
Found Yes
Hash 85a21c383507cdf05801a4eed540dbafeca9f79112b2b0c534d357bbf5840f56
SimHash 433fd5c0c019

Groups

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

proximic

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 7

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 7

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60
Back to top

Warnings

  • 2 invalid lines.
Back to top