gaycupidstories.com
robots.txt

Robots Exclusion Standard data for gaycupidstories.com

Resource Scan

Scan Details

Site Domain gaycupidstories.com
Base Domain gaycupidstories.com
Scan Status Ok
Last Scan2024-09-23T06:31:35+00:00
Next Scan 2024-10-23T06:31:35+00:00

Last Scan

Scanned2024-09-23T06:31:35+00:00
URL https://gaycupidstories.com/robots.txt
Redirect https://www.gaycupidstories.com/robots.txt
Redirect Domain www.gaycupidstories.com
Redirect Base gaycupidstories.com
Domain IPs 104.21.4.245, 172.67.187.63, 2606:4700:3033::6815:4f5, 2606:4700:3034::ac43:bb3f
Redirect IPs 104.21.4.245, 172.67.187.63, 2606:4700:3033::6815:4f5, 2606:4700:3034::ac43:bb3f
Response IP 172.67.187.63
Found Yes
Hash f66d97de8ff22bb5962c9d2913f22b9c28ac02d198d3799833669aae290ba478
SimHash f854d790035a

Groups

*

Rule Path
Disallow /story/index/
Disallow /story/letter/
Disallow /story/fetchComments
Disallow /story/showSecret
Disallow /story/editSecret
Disallow /story/edit
Disallow /story/publish
Disallow /story/deleteRevisions
Disallow /story/search
Disallow /series/index
Disallow /series/letter
Disallow /user/listLikedStories/
Disallow /comments/

baiduspider
yisouspider
petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

Warnings

  • 1 invalid line.