/.well-known/

Log In Sign Up

onlyindianporn2.com
robots.txt

Robots Exclusion Standard data for onlyindianporn2.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	onlyindianporn2.com
Base Domain	onlyindianporn2.com
Scan Status	Ok
Last Scan	2024-09-14T07:31:51+00:00
Next Scan	2024-10-14T07:31:51+00:00

Last Scan

Scanned	2024-09-14T07:31:51+00:00
URL	https://onlyindianporn2.com/robots.txt
Domain IPs	104.21.95.28, 172.67.169.77, 2606:4700:3034::ac43:a94d, 2606:4700:3037::6815:5f1c
Response IP	172.67.169.77
Found	Yes
Hash	1da557f2b0055d05bf358a2f62ae11cd39d8c6a255dd99d482abad90847624fc
SimHash	501ce0c0ecb8

Groups

*

Rule

Path

Disallow

/*.php*

blexbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

vagabondo

Rule

Path

Disallow

/

seokicks-robot

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

archive.org_bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

heritrix

Rule

Path

Disallow

/

netestate ne crawler

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

queryseekerspider

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

baiduspider

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

30

msnbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

bingbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

Back to top

Warnings

2 invalid lines.

Back to top