insaafpunjabi.com
robots.txt

Robots Exclusion Standard data for insaafpunjabi.com

Resource Scan

Scan Details

Site Domain insaafpunjabi.com
Base Domain insaafpunjabi.com
Scan Status Ok
Last Scan2025-12-11T23:17:30+00:00
Next Scan 2026-01-10T23:17:30+00:00

Last Scan

Scanned2025-12-11T23:17:30+00:00
URL https://insaafpunjabi.com/robots.txt
Domain IPs 104.26.8.72, 104.26.9.72, 172.67.70.122, 2606:4700:20::681a:848, 2606:4700:20::681a:948, 2606:4700:20::ac43:467a
Response IP 104.26.8.72
Found Yes
Hash a6e5c42a07dbc41ba3a7ac6c587ff9ff4a492c4ce25cbb7eb87f3e1a9ee4969b
SimHash 6d115f50e426

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /api/
Disallow /_next/
Disallow /user/
Disallow /checkout/

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap http://insaafpunjabi.com/sitemap.xml

Comments

  • Robots.txt for http://insaafpunjabi.com
  • Allow all crawlers
  • Allow crawling of all content
  • Disallow admin and private areas
  • Sitemap location
  • Crawl delay (optional)