Log In Sign Up

sarkarihiring.com
robots.txt

Robots Exclusion Standard data for sarkarihiring.com

Resource Scan

Scan Details

Site Domain sarkarihiring.com
Base Domain sarkarihiring.com
Scan Status Ok
Last Scan2025-12-24T02:41:30+00:00
Next Scan 2025-12-31T02:41:30+00:00

Last Scan

Scanned2025-12-24T02:41:30+00:00
URL https://sarkarihiring.com/robots.txt
Domain IPs 104.21.78.60, 172.67.217.57, 2606:4700:3034::6815:4e3c, 2606:4700:3037::ac43:d939
Response IP 104.21.78.60
Found Yes
Hash 87cfcca22409dadce473f9a124ec47198714b494abf41e66658565a34af9b485
SimHash ea24941387b3

Groups

*

Rule Path Comment
Disallow /wp-admin/ -
Allow /wp-admin/admin-ajax.php -
Disallow /*?s= Blocks internal search results
Disallow /tag/ Blocks low-value tags
Disallow /author/ Blocks author archives
Disallow /trackback/ Blocks trackback spam

facebookexternalhit

Rule Path
Allow /

facebot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://sarkarihiring.com/sitemap_index.xml
Back to top

Comments

  • --- Core Admin Rules ---
  • --- Crawl Waste Control (Block low-value, allow discovery) ---
  • NOTE: Pagination (/page/) and PDFs are ALLOWED to be crawled here.
  • Indexing (preventing PDF ranking) is controlled by your Cloudflare Transform Rule.
  • --- Social Bots (for link previews) ---
  • --- Sitemap ---
Back to top