Log In Sign Up

bihar.pscnotes.com
robots.txt

Robots Exclusion Standard data for bihar.pscnotes.com

Resource Scan

Scan Details

Site Domain bihar.pscnotes.com
Base Domain pscnotes.com
Scan Status Ok
Last Scan2025-10-22T01:52:18+00:00
Next Scan 2025-11-21T01:52:18+00:00

Last Scan

Scanned2025-10-22T01:52:18+00:00
URL https://bihar.pscnotes.com/robots.txt
Domain IPs 148.113.6.36
Response IP 148.113.6.36
Found Yes
Hash 23cc1a3fd85d400a5d67ff99c71d9e31c13fbe9068b376c447f88450c51e2340
SimHash 413cdc52d6b2

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yandex

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

curl

Rule Path
Disallow /

wget

Rule Path
Disallow /

java

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Disallow /edd/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /trackback/
Disallow /wp-login/
Disallow /author/*
Disallow /profile/*
Disallow */trackback
Disallow /*.inc$
Disallow */feed/
Disallow /*.php$

Other Records

Field Value
crawl-delay 10

ia_archiver

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow
Allow /*
Back to top

Other Records

Field Value
sitemap https://bihar.pscnotes.com/sitemap_index.xml
Back to top

Comments

  • ========================================
  • âœ
  • ========================================
  • ========================================
  • 🔒 BLOCKLISTED BOTS (Known Abusers)
  • ========================================
  • ========================================
  • 🚫 GENERAL BOT RESTRICTIONS (for all)
  • ========================================
Back to top

Warnings

  • 1 invalid line.
Back to top