aicontentlabs.com
robots.txt

Robots Exclusion Standard data for aicontentlabs.com

Resource Scan

Scan Details

Site Domain aicontentlabs.com
Base Domain aicontentlabs.com
Scan Status Ok
Last Scan2024-10-19T15:21:01+00:00
Next Scan 2024-11-18T15:21:01+00:00

Last Scan

Scanned2024-10-19T15:21:01+00:00
URL https://aicontentlabs.com/robots.txt
Redirect https://www.aicontentlabs.com/robots.txt
Redirect Domain www.aicontentlabs.com
Redirect Base aicontentlabs.com
Domain IPs 76.76.21.21
Redirect IPs 76.76.21.164, 76.76.21.93
Response IP 76.76.21.61
Found Yes
Hash 72a661240659b257b763bdafb1d6784b6da989e13ed9a786d38afb539240b8b9
SimHash 591e98427c90

Groups

*

Rule Path
Allow /
Disallow /api/*
Disallow /_next/*
Disallow /api/
Disallow /_next/

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.aicontentlabs.com/sitemap.xml
sitemap https://www.aicontentlabs.com/prompts/sitemap.xml
sitemap https://www.aicontentlabs.com/flows/sitemap.xml