Log In Sign Up

notificationapi.com
robots.txt

Robots Exclusion Standard data for notificationapi.com

Resource Scan

Scan Details

Site Domain notificationapi.com
Base Domain notificationapi.com
Scan Status Ok
Last Scan2025-10-07T01:14:37+00:00
Next Scan 2025-11-06T01:14:37+00:00

Last Scan

Scanned2025-10-07T01:14:37+00:00
URL https://notificationapi.com/robots.txt
Redirect https://www.notificationapi.com/robots.txt
Redirect Domain www.notificationapi.com
Redirect Base notificationapi.com
Domain IPs 3.165.102.117, 3.165.102.121, 3.165.102.41, 3.165.102.57
Redirect IPs 3.165.102.117, 3.165.102.121, 3.165.102.41, 3.165.102.57
Response IP 3.165.102.121
Found Yes
Hash a8e2678cd8e2cfa4a01016a79815cf82d05608f39d90e7b91bb53bca71f84190
SimHash 6470dd336bec

Groups

*

Rule Path
Allow /
Disallow /search*
Disallow /_*
Disallow /admin*
Disallow /*.json$
Disallow /*.xml$
Disallow /docs/*/docs/
Disallow /docs/components/*.astro
Disallow /**/NotificationAPI$
Disallow /docs/*/NotificationAPI$

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

facebookexternalhit

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.notificationapi.com/sitemap-index.xml
Back to top

Comments

  • Robots.txt for www.notificationapi.com
  • Updated for better SEO and crawler management
  • Allow all crawlers access to public content
  • Prevent indexing of search results or internal pages
  • Block malformed docs URLs and component files
  • Respect crawl delay to avoid overwhelming the server
  • Specific rules for major search engines
  • Block aggressive crawlers that don't respect rate limits
  • Allow AhrefsBot for SEO monitoring but with crawl delay
  • Sitemap location
  • Note: The app subdomain (app.notificationapi.com) has its own robots.txt
  • that blocks all crawlers since it contains private application content
Back to top