findingtheuniverse.com
robots.txt

Robots Exclusion Standard data for findingtheuniverse.com

Resource Scan

Scan Details

Site Domain findingtheuniverse.com
Base Domain findingtheuniverse.com
Scan Status Ok
Last Scan2025-12-21T13:41:38+00:00
Next Scan 2025-12-28T13:41:38+00:00

Last Scan

Scanned2025-12-21T13:41:38+00:00
URL https://findingtheuniverse.com/robots.txt
Domain IPs 104.21.58.176, 172.67.162.102, 2606:4700:3032::6815:3ab0, 2606:4700:3037::ac43:a266
Response IP 172.67.162.102
Found Yes
Hash 529021782a8187483839c3622654fceec0c2836912cb4927081ef4c051c6b50f
SimHash 61040103c9a2

Groups

mj12bot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

*

Rule Path
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.findingtheuniverse.com/sitemap_index.xml