community.travel
robots.txt

Robots Exclusion Standard data for community.travel

Resource Scan

Scan Details

Site Domain community.travel
Base Domain community.travel
Scan Status Ok
Last Scan2024-09-24T15:05:38+00:00
Next Scan 2024-10-24T15:05:38+00:00

Last Scan

Scanned2024-09-24T15:05:38+00:00
URL https://community.travel/robots.txt
Domain IPs 3.164.182.14, 3.164.182.38, 3.164.182.58, 3.164.182.97
Response IP 13.226.2.52
Found Yes
Hash e25961935df09da3258ce53616f52995a93b8c6ed34152c6f7465e50495ffa68
SimHash 094eb6636821

Groups

*

Rule Path
Disallow /cms/
Disallow /push/
Disallow /service/
Disallow /client/dynforms/
Disallow /client/ajax/

converacrawler

Rule Path
Disallow /

tmcrawler

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

freewebmonitoring

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

huaweisymantec

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

Comments

  • Web monitoring service. Who needs?
  • Check student papers for fraud, not needed
  • Identity Thef searcher
  • Security searcher