thecollective.in
robots.txt

Robots Exclusion Standard data for thecollective.in

Resource Scan

Scan Details

Site Domain thecollective.in
Base Domain thecollective.in
Scan Status Ok
Last Scan2024-05-31T09:59:54+00:00
Next Scan 2024-06-30T09:59:54+00:00

Last Scan

Scanned2024-05-31T09:59:54+00:00
URL https://thecollective.in/robots.txt
Redirect https://www.thecollective.in:443/robots.txt
Redirect Domain www.thecollective.in
Redirect Base thecollective.in
Domain IPs 13.200.122.69, 13.232.147.176, 52.66.34.19
Redirect IPs 184.87.193.88, 184.87.193.97, 2600:1413:b000:14::b857:c158, 2600:1413:b000:14::b857:c161
Response IP 72.247.127.224
Found Yes
Hash 356b047db61a5feed830b5c8217da1784af6bb791222192343561db4f719c96c
SimHash 6545d7f0df50

Groups

googlebot
mediapartners-google
adsbot-google
adsbot-google-mobile
googlebot-image
googlebot-mobile
ahrefsbot
ahrefssiteaudit
semrushbot
seoprofiler
urlprofiler
moz
bingbot
msnbot
msnbot-media
adidxbot
twitterbot

Rule Path
Allow *.js
Allow *.css
Allow /

*

Rule Path
Disallow /

swiftbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.thecollective.in/sitemap.xml
sitemap https://blog.thecollective.in/sitemap.xml