astrolookup.com
robots.txt
Robots Exclusion Standard data for astrolookup.com
Resource Scan
Scan Details
| Site Domain | astrolookup.com |
| Base Domain | astrolookup.com |
| Scan Status | Ok |
| Last Scan | 4/21/2025, 5:34:15 PM |
| Next Scan | 4/28/2025, 5:34:15 PM |
Last Scan
| Scanned | 4/21/2025, 5:34:15 PM |
| URL | https://astrolookup.com/robots.txt |
| Redirect | https://www.astrolookup.com/robots.txt |
| Redirect Domain | www.astrolookup.com |
| Redirect Base | astrolookup.com |
| Domain IPs | 104.21.73.102, 172.67.189.164, 2606:4700:3032::ac43:bda4, 2606:4700:3037::6815:4966 |
| Redirect IPs | 104.21.73.102, 172.67.189.164, 2606:4700:3032::ac43:bda4, 2606:4700:3037::6815:4966 |
| Response IP | 172.67.189.164 |
| Found | Yes |
| Hash | 15c6f004890e082f1cf70efc45d1dd9d4f122d1dcb9075b0527e6b3013b9b32d |
| SimHash | 6b765fb0c6d1 |
Groups
googlebot
googlebot-mobile
googlebot-image
googlebot-news
googlebot-video
googlebot/2.1
googlebot-mobile/2.1
google favicon
google page speed insights
chrome-lighthouse
googleweblight
chrome/w.x.y.z
google chrome lighthouse
storebot-google
crios
adsbot-google-mobile
adsbot-google
mediapartners-google
mediapartners-google/2.1
adsbot-google-mobile-apps
apis-google
storebot-google
google-inspectiontool
googleother
google-extended
| Rule | Path |
|---|---|
| Disallow | /ads/ |
| Disallow | /jscript/ |
| Disallow | /include/ |
| Disallow | /notforg/ |
| Disallow | /push/ |
| Disallow | /*?cx |
| Disallow | /?q=* |
| Disallow | /*?q |
| Disallow | /*?term |
| Disallow | /feed.php |
| Disallow | /feeds.php |
| Disallow | /rss.php |
msnbot
bingbot/2.0
bingbot
bingbot
slurp
teoma
yahoo-mmcrawler
yahoo-blogs/v3.9
duckduckbot
exabot
facebot
facebookexternalhit/1.0
facebookexternalhit/1.1
facebookexternalhit
meta-externalagent/1.1
twitterbot
aol
msnbot-media
ask jeeves/teoma
applebot
| Rule | Path |
|---|---|
| Disallow | /ads/ |
| Disallow | /jscript/ |
| Disallow | /include/ |
| Disallow | /notforg/ |
| Disallow | /push/ |
| Disallow | /*?cx |
| Disallow | /*?term |
| Disallow | /feed.php |
| Disallow | /feeds.php |
| Disallow | /rss.php |
| Disallow | /?qs=* |
| Disallow | /*?qs |
| Disallow | /*.php$ |
| Disallow | /?q=* |
| Disallow | /*?q |
| Disallow | /*? |
| Disallow | /*?$ |
| Disallow | /*?* |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.astrolookup.com/Sitemap.txt |
| sitemap | https://www.astrolookup.com/sitemap.xml |