99pandit.com
robots.txt

Robots Exclusion Standard data for 99pandit.com

Resource Scan

Scan Details

Site Domain 99pandit.com
Base Domain 99pandit.com
Scan Status Ok
Last Scan2025-10-17T17:09:21+00:00
Next Scan 2025-10-24T17:09:21+00:00

Last Scan

Scanned2025-10-17T17:09:21+00:00
URL https://99pandit.com/robots.txt
Domain IPs 101.53.148.136
Response IP 101.53.148.136
Found Yes
Hash 2a14731a5d83312fe8f9c657db7c67bf03a27da30d40efc6036eeec876cb5ff7
SimHash 10ee9c608c30

Groups

crawl

Rule Path
Disallow /

*

Rule Path
Disallow

unknown robot identified by *bot

Rule Path
Disallow /

unknown robot identified by bot*

Rule Path
Disallow /

robot

Rule Path
Disallow /

wget

Rule Path
Disallow /

nbot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

dalvik

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

feed

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

empty user agent string

Rule Path
Disallow /

others

Rule Path
Disallow /
Disallow /wp-admin/
Disallow /GETTABLE/*
Disallow /blog/category/*
Disallow /blog/author/*
Disallow /comments/*
Disallow /blog/tag/*
Disallow /tag/*
Disallow /support/tag/*
Disallow /category/*
Disallow /login/
Disallow /login/*
Disallow /wp-content/grade/
Disallow /wp-content/grade/*
Disallow /newhome
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://99pandit.com/sitemap_index.xml
sitemap https://99pandit.com/page-sitemap.xml
sitemap https://99pandit.com/sitemap.xml