spunky.im
robots.txt

Robots Exclusion Standard data for spunky.im

Resource Scan

Scan Details

Site Domain spunky.im
Base Domain spunky.im
Scan Status Ok
Last Scan2025-07-01T09:47:44+00:00
Next Scan 2025-07-08T09:47:44+00:00

Last Scan

Scanned2025-07-01T09:47:44+00:00
URL https://spunky.im/robots.txt
Domain IPs 104.21.6.165, 172.67.135.8, 2606:4700:3033::ac43:8708, 2606:4700:3034::6815:6a5
Response IP 172.67.135.8
Found Yes
Hash 6b7afe0704a682b32d09f9bb793f803d862f18996d457d82cab7a080db3e3405
SimHash 10005e40ca82

Groups

*

Rule Path
Allow /
Allow /en/
Allow /ko/
Allow /ru/
Allow /pt/
Disallow /Base-Template/
Disallow /css/
Disallow /js/
Disallow /img/
Disallow /node_modules/
Disallow /.git/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://spunky.im/sitemap.xml

Comments

  • robots.txt for https://spunky.im
  • 站点地图
  • 爬虫限制
  • 禁止访问的目录