Log In Sign Up

pakobserver.net
robots.txt

Robots Exclusion Standard data for pakobserver.net

Resource Scan

Scan Details

Site Domain pakobserver.net
Base Domain pakobserver.net
Scan Status Ok
Last Scan2024-11-15T16:35:21+00:00
Next Scan 2024-11-22T16:35:21+00:00

Last Scan

Scanned2024-11-15T16:35:21+00:00
URL https://pakobserver.net/robots.txt
Domain IPs 104.26.14.6, 104.26.15.6, 172.67.71.80, 2606:4700:20::681a:e06, 2606:4700:20::681a:f06, 2606:4700:20::ac43:4750
Response IP 104.26.14.6
Found Yes
Hash 96ccad382d7afceb23d5963f8031d5d0d6fd0e3ba71c207fd6a5351587a9965e
SimHash ff54d921f9a8

Groups

*

Rule Path
Disallow /wp-json/
Disallow /?s=*
Disallow /?search
Disallow /search/*
Disallow */trackback
Disallow */comments
Disallow /*.pdf$
Disallow /tag
Allow /wp-admin/admin-ajax.php

nuclei
wikido
riddler
petalbot
zoominfobot
node/simplecrawler
cazoodlebot
dotbot/1.0
barkrowler
blexbot
magpie-crawler
ias_crawler
semrushbot
dataforseobot
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
splitsignalbot
semrushbot-ocob
gptbot
imagesiftbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://pakobserver.net/sitemap_index.xml
Back to top

Comments

  • Ban bots that don't benefit us.
  • --------------------------------
Back to top