Log In Sign Up

publinet.pro
robots.txt

Robots Exclusion Standard data for publinet.pro

Resource Scan

Scan Details

Site Domain publinet.pro
Base Domain publinet.pro
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer redirected incorrectly.
Last Scan2024-11-14T23:11:09+00:00
Next Scan 2025-01-13T23:11:09+00:00

Last Successful Scan

Scanned2024-08-24T23:08:35+00:00
URL https://publinet.pro/robots.txt
Redirect https://bollylinks.info/robots.txt
Redirect Domain bollylinks.info
Redirect Base bollylinks.info
Domain IPs 104.21.23.105, 172.67.210.127, 2606:4700:3031::ac43:d27f, 2606:4700:3035::6815:1769
Redirect IPs 104.21.52.8, 172.67.192.199, 2606:4700:3033::6815:3408, 2606:4700:3034::ac43:c0c7
Response IP 172.67.192.199
Found Yes
Hash b7efa79bfbd4b8ae96ad41d4d71bfef1d6dc4abc7eb0af708fc502150455df46
SimHash 44bf0e82ecd4

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /xmlrpc.php

Other Records

Field Value
crawl-delay 100

dotbot

Rule Path
Disallow /

giftghostbot

Rule Path
Disallow /

seznam

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

dataprovider6.101

Rule Path
Disallow /

dataprovidersiteexplorer

Rule Path
Disallow /

dazoobot1.0

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

domainstatsbot1.0

Rule Path
Disallow /

dotbot1.1

Rule Path
Disallow /

dubaiindex

Rule Path
Disallow /

ecommercebot

Rule Path
Disallow /

expertsearchspider

Rule Path
Disallow /

feedbin

Rule Path
Disallow /

fetch2.0a

Rule Path
Disallow /

ffbot1.0

Rule Path
Disallow /

focusbot1.1

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

huaweisymantecspider1.0

Rule Path
Disallow /

jobdiggerspider

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

lipperheylinkexplorer

Rule Path
Disallow /

lssrocketcrawler1.0

Rule Path
Disallow /

lyt.srv1.5

Rule Path
Disallow /

miadev0.0.1

Rule Path
Disallow /

najdi.si3.1

Rule Path
Disallow /

bountiibot

Rule Path
Disallow /

experibot_v1

Rule Path
Disallow /

bixocrawler

Rule Path
Disallow /

bixocrawler testcrawler

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

crowsnest0.5

Rule Path
Disallow /

cukbot

Rule Path
Disallow /

dataprovider6.92

Rule Path
Disallow /

dblbot1.0

Rule Path
Disallow /

diffbot0.1

Rule Path
Disallow /

digg deeperv1

Rule Path
Disallow /

discobot1.0

Rule Path
Disallow /

discobot1.1

Rule Path
Disallow /

discobot2.0

Rule Path
Disallow /

discoverybot2.0

Rule Path
Disallow /

dlvr.it1.0

Rule Path
Disallow /

drupact0.7

Rule Path
Disallow /

ezooms1.0

Rule Path
Disallow /

fastbot crawler beta 2.0

Rule Path
Disallow /

fastbot crawler beta 4.0

Rule Path
Disallow /

feedly social

Rule Path
Disallow /

feedly1.0

Rule Path
Disallow /

feedlybot1.0

Rule Path
Disallow /

feedspot

Rule Path
Disallow /

feedspotbot1.0

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

classbot

Rule Path
Disallow /

cispa vulnerability notification

Rule Path
Disallow /

cirrusexplorer1.1

Rule Path
Disallow /

checksemnutch-1.10

Rule Path
Disallow /

catchbot5.0

Rule Path
Disallow /

catchbot3.0

Rule Path
Disallow /

catchbot2.0

Rule Path
Disallow /

catchbot1.0

Rule Path
Disallow /

camontspider1.0

Rule Path
Disallow /

buzzbot1.0

Rule Path
Disallow /

buzzbot

Rule Path
Disallow /

businessseek.biz_spider

Rule Path
Disallow /

bubing

Rule Path
Disallow /

nail

Rule Path
Disallow /

fyberspider1.3

Rule Path
Disallow /

findlinks1.1.6-beta5

Rule Path
Disallow /

g2reader-bot1.0

Rule Path
Disallow /

findlinks1.1.6-beta6

Rule Path
Disallow /

findlinks2.0

Rule Path
Disallow /

findlinks2.0.1

Rule Path
Disallow /

findlinks2.0.2

Rule Path
Disallow /

findlinks2.0.4

Rule Path
Disallow /

findlinks2.0.5

Rule Path
Disallow /

findlinks2.0.9

Rule Path
Disallow /

findlinks2.1

Rule Path
Disallow /

findlinks2.1.5

Rule Path
Disallow /

findlinks2.1.3

Rule Path
Disallow /

findlinks2.2

Rule Path
Disallow /

findlinks2.5

Rule Path
Disallow /

findlinks2.6

Rule Path
Disallow /

findlinks1.0

Rule Path
Disallow /

findlinks1.1.3-beta8

Rule Path
Disallow /

findlinks1.1.3-beta9

Rule Path
Disallow /

findlinks1.1.4-beta7

Rule Path
Disallow /

findlinks1.1.6-beta1

Rule Path
Disallow /

findlinks1.1.6-beta1 yacy

Rule Path
Disallow /

findlinks1.1.6-beta2

Rule Path
Disallow /

findlinks1.1.6-beta3

Rule Path
Disallow /

findlinks1.1.6-beta4

Rule Path
Disallow /

bixo

Rule Path
Disallow /

bixolabs1.0

Rule Path
Disallow /

crawlera1.10.2

Rule Path
Disallow /

bingbot

Rule Path
Allow /

yandex

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

googlebot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://bollylinks.info/news-sitemap.xml
sitemap https://bollylinks.info/sitemap.xml
sitemap https://bollylinks.info/sitemap.rss
Back to top

Warnings

  • 6 invalid lines.
  • `clean-param` is not a known field.
Back to top