Log In Sign Up

pankajbatra.com
robots.txt

Robots Exclusion Standard data for pankajbatra.com

Resource Scan

Scan Details

Site Domain pankajbatra.com
Base Domain pankajbatra.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-07T23:47:28+00:00
Next Scan 2025-11-05T23:47:28+00:00

Last Successful Scan

Scanned2025-04-10T23:13:00+00:00
URL https://pankajbatra.com/robots.txt
Domain IPs 104.21.4.138, 172.67.132.34, 2606:4700:3032::ac43:8422, 2606:4700:3036::6815:48a
Response IP 172.67.132.34
Found Yes
Hash 0165171d481a4edfb9bfc89ab6e28a1b735cc5e0062dcbdf124c78e52bca0515
SimHash 6301485072d7

Groups

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Disallow

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /trackback/
Allow /wp-content/uploads/

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap http://www.pankajbatra.com/sitemap.xml.gz
Back to top

Comments

  • Google Image
  • Google AdSense
  • global
  • Disallow: /feed/
  • Disallow: /comments/
  • Disallow: /category/*/*
  • Disallow: */trackback/
  • Disallow: */feed/
  • Disallow: */comments/
  • Disallow: /*?
  • User-Agent: *
  • Allow: /
  • BEGIN XML-SITEMAP-PLUGIN
  • END XML-SITEMAP-PLUGIN
Back to top