Log In Sign Up

neptuneworld.in
robots.txt

Robots Exclusion Standard data for neptuneworld.in

Resource Scan

Scan Details

Site Domain neptuneworld.in
Base Domain neptuneworld.in
Scan Status Ok
Last Scan2026-03-01T07:32:34+00:00
Next Scan 2026-03-08T07:32:34+00:00

Last Scan

Scanned2026-03-01T07:32:34+00:00
URL https://neptuneworld.in/robots.txt
Domain IPs 104.21.93.19, 172.67.202.182, 2606:4700:3033::ac43:cab6, 2606:4700:3034::6815:5d13
Response IP 172.67.202.182
Found Yes
Hash f96c37d06a2cbb70e5265be52445ca3f04d86afe01b725cef4083f47d433ac24
SimHash 001f5bb56781

Groups

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

slurp

Product Comment
slurp Yahoo
Rule Path
Disallow

duckduckbot

Rule Path
Disallow

yandex

Rule Path
Disallow

baiduspider

Rule Path
Disallow

applebot

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

bidswitchbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

ttd-content

Rule Path
Disallow /

cortex-xpanse

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

curl

Rule Path
Disallow /

wget

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

*

Rule Path
Disallow /admin/
Disallow /api/
Back to top

Other Records

Field Value
sitemap https://neptuneworld.in/sitemap.xml
Back to top

Comments

  • robots.txt for neptuneworld.in
  • ------------------------
  • âœ
  • ------------------------
  • ------------------------
  • 🚫 Block SEO Crawlers
  • ------------------------
  • ------------------------
  • 🚫 Block Ad-Tech Crawlers (Not useful if you only use Google Ads)
  • ------------------------
  • ------------------------
  • 🚫 Block Aggressive Scrapers
  • ------------------------
  • ------------------------
  • Block admin and other urls
  • ------------------------
  • ------------------------
  • Sitemap (optional, if you have one)
  • ------------------------
Back to top

Warnings

  • 1 invalid line.
Back to top