crepu.net
robots.txt

Robots Exclusion Standard data for crepu.net

Resource Scan

Scan Details

Site Domain crepu.net
Base Domain crepu.net
Scan Status Ok
Last Scan2024-06-13T05:01:50+00:00
Next Scan 2024-06-20T05:01:50+00:00

Last Scan

Scanned2024-06-13T05:01:50+00:00
URL https://crepu.net/robots.txt
Domain IPs 3.165.102.119, 3.165.102.54, 3.165.102.61, 3.165.102.98
Response IP 3.165.102.54
Found Yes
Hash 3ec70629f7aa09449b42669ed966fcfb76127585f60c08ed73627442a50827b7
SimHash 1834d200e173

Groups

twitterbot

Rule Path
Disallow /
Allow /post/*
Allow /user/*

facebookexternalhit

Rule Path
Disallow /
Allow /post/*
Allow /user/*

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

*

Rule Path
Allow /
Allow /plus
Disallow /post/*
Disallow /board/*
Disallow /user/*
Disallow /search/*
Disallow /invite/*