unifolks.com
robots.txt

Robots Exclusion Standard data for unifolks.com

Resource Scan

Scan Details

Site Domain unifolks.com
Base Domain unifolks.com
Scan Status Ok
Last Scan2024-11-15T08:45:44+00:00
Next Scan 2024-11-22T08:45:44+00:00

Last Scan

Scanned2024-11-15T08:45:44+00:00
URL https://unifolks.com/robots.txt
Redirect http://www.unifolks.com/robots.txt
Redirect Domain www.unifolks.com
Redirect Base unifolks.com
Domain IPs 104.21.15.39, 172.67.161.81, 2606:4700:3034::6815:f27, 2606:4700:3037::ac43:a151
Redirect IPs 104.21.15.39, 172.67.161.81, 2606:4700:3034::6815:f27, 2606:4700:3037::ac43:a151
Response IP 172.67.161.81
Found Yes
Hash 23713b52b9dc8fd9fe06f8db6ef37d924b7be82907e958fd511bf23ed25029e6
SimHash 4a14dc80e3c2

Groups

*

Rule Path
Disallow /attached-files/*

ia_archiver-web.archive.org

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

grammarly

Rule Path
Disallow /

grammarly

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

amazonbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10