Log In Sign Up

wazen.sa
robots.txt

Robots Exclusion Standard data for wazen.sa

Resource Scan

Scan Details

Site Domain wazen.sa
Base Domain wazen.sa
Scan Status Ok
Last Scan2025-10-28T21:59:28+00:00
Next Scan 2025-11-27T21:59:28+00:00

Last Scan

Scanned2025-10-28T21:59:28+00:00
URL https://wazen.sa/robots.txt
Domain IPs 104.26.2.27, 104.26.3.27, 172.67.74.82, 2606:4700:20::681a:21b, 2606:4700:20::681a:31b, 2606:4700:20::ac43:4a52
Response IP 104.26.3.27
Found Yes
Hash 3edf34b3abf290b8b20239e733cd78da63277650e38335b3545cbae8705ae2fe
SimHash 651c1815a7f2

Groups

*

Rule Path
Allow /
Disallow /private/
Disallow /api/
Disallow /*.json$
Disallow /*?*

Other Records

Field Value
crawl-delay 10

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /*.pdf$
Disallow /*.doc$
Disallow /*.docx$
Back to top

Other Records

Field Value
sitemap https://wazen.sa/sitemap.xml
Back to top

Comments

  • Specific rules for major bots
  • Prevent media files from being crawled (if needed)
  • Sitemaps
Back to top