/.well-known/

Log In Sign Up

xxxmillion.com
robots.txt

Robots Exclusion Standard data for xxxmillion.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	xxxmillion.com
Base Domain	xxxmillion.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Server returned a client error.
Last Scan	2024-09-27T19:55:11+00:00
Next Scan	2024-10-11T19:55:11+00:00

Last Successful Scan

Scanned	2024-08-20T03:28:41+00:00
URL	https://xxxmillion.com/robots.txt
Domain IPs	104.21.24.68, 172.67.217.89, 2606:4700:3031::6815:1844, 2606:4700:3037::ac43:d959
Response IP	104.21.24.68
Found	Yes
Hash	7342f4ef7b3a5b2c775a51755ea4fc6e07c84a8348a42a8cde126f99a911a0b6
SimHash	501e43404773

Groups

gptbot

Rule

Path

Disallow

/

yandexbot

Rule

Path

Disallow

/

bingbot

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

mojeekbot

Rule

Path

Disallow

/

bytespider

Rule

Path

Disallow

/

ccbot

Rule

Path

Disallow

/

dataforseobot

Rule

Path

Disallow

/

seznambot

Rule

Path

Disallow

/

cfnetwork

Rule

Path

Disallow

/

applebot

Rule

Path

Disallow

/

petalbot

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

googleother

Rule

Path

Disallow

/

googleother-image

Rule

Path

Disallow

/

googleother-video

Rule

Path

Disallow

/

*

Rule

Path

Disallow

/m/

Disallow

/mobile/

Allow

/

Back to top