/.well-known/

Log In Sign Up

onlyindianpornx.com
robots.txt

Robots Exclusion Standard data for onlyindianpornx.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	onlyindianpornx.com
Base Domain	onlyindianpornx.com
Scan Status	Ok
Last Scan	2024-10-03T00:11:42+00:00
Next Scan	2024-11-02T00:11:42+00:00

Last Scan

Scanned	2024-10-03T00:11:42+00:00
URL	https://www.onlyindianpornx.com/robots.txt
Redirect	https://onlyindianpornx.com/robots.txt
Redirect Domain	onlyindianpornx.com
Redirect Base	onlyindianpornx.com
Domain IPs	104.21.9.214, 172.67.161.86, 2606:4700:3030::6815:9d6, 2606:4700:3031::ac43:a156
Redirect IPs	104.21.9.214, 172.67.161.86, 2606:4700:3030::6815:9d6, 2606:4700:3031::ac43:a156
Response IP	104.21.9.214
Found	Yes
Hash	1da557f2b0055d05bf358a2f62ae11cd39d8c6a255dd99d482abad90847624fc
SimHash	501ce0c0ecb8

Groups

*

Rule

Path

Disallow

/*.php*

blexbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

vagabondo

Rule

Path

Disallow

/

seokicks-robot

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

archive.org_bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

heritrix

Rule

Path

Disallow

/

netestate ne crawler

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

queryseekerspider

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

baiduspider

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

30

msnbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

bingbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

Back to top

Warnings

2 invalid lines.

Back to top