punjabi.com
robots.txt

Robots Exclusion Standard data for punjabi.com

Resource Scan

Scan Details

Site Domain punjabi.com
Base Domain punjabi.com
Scan Status Ok
Last Scan2026-03-02T15:26:23+00:00
Next Scan 2026-03-09T15:26:23+00:00

Last Scan

Scanned2026-03-02T15:26:23+00:00
URL https://punjabi.com/robots.txt
Domain IPs 104.21.78.101, 172.67.220.54, 2606:4700:3030::ac43:dc36, 2606:4700:3031::6815:4e65
Response IP 172.67.220.54
Found Yes
Hash 39985c3c6453c0d3f7998a8abb7f230a5a00e005f1f94f21dd1b60b6d356a623
SimHash 691c4e12eec1

Groups

*

Rule Path
Allow /api/others/sitemap.xml
Disallow /api/
Disallow /admin/
Disallow /private/
Disallow /user-data/
Disallow /404
Disallow /legal/privacy
Disallow /legal/terms
Disallow /legal/copyright
Disallow /coming_soon
Disallow /videos
Allow /

Other Records

Field Value
sitemap https://admin.punjabi.com/api/others/sitemap.xml

Comments

  • Block all web crawlers from accessing sensitive areas
  • Allow crawlers to access all other parts of the site