Log In Sign Up

pikartlights.com
robots.txt

Robots Exclusion Standard data for pikartlights.com

Resource Scan

Scan Details

Site Domain pikartlights.com
Base Domain pikartlights.com
Scan Status Ok
Last Scan2025-11-02T18:50:36+00:00
Next Scan 2025-12-02T18:50:36+00:00

Last Scan

Scanned2025-11-02T18:50:36+00:00
URL https://pikartlights.com/robots.txt
Domain IPs 104.21.5.122, 172.67.133.101, 2606:4700:3033::6815:57a, 2606:4700:3034::ac43:8565
Response IP 104.21.5.122
Found Yes
Hash cb093974eec5c50f1fd9a901809ae9f1fdbae593e7eb6c9e19956f7eaf74e425
SimHash 4b0ccc104490

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://pikartlights.com/sitemap.xml
Back to top

Comments

  • Good bots: Allow sitemap access
  • Bad bots - block completely
  • All other bots - slow them down
Back to top