forum.egosoft.com
robots.txt

Robots Exclusion Standard data for forum.egosoft.com

Resource Scan

Scan Details

Site Domain forum.egosoft.com
Base Domain egosoft.com
Scan Status Ok
Last Scan2025-11-29T08:24:15+00:00
Next Scan 2025-12-29T08:24:15+00:00

Last Scan

Scanned2025-11-29T08:24:15+00:00
URL https://forum.egosoft.com/robots.txt
Domain IPs 92.205.227.239
Response IP 92.205.227.239
Found Yes
Hash ae8ba482c0555813ace8eedca484638bb84303ce35fcfee2859c2d0c21a9633a
SimHash d038dc0a6b40

Groups

*

Rule Path
Disallow /posting.php
Disallow /admin
Disallow /privmsg.php
Disallow /search.php
Disallow /login.php
Disallow /memberlist.php
Disallow /images
Disallow /includes
Disallow /profile.php
Disallow /cache
Disallow /files
Disallow /ucp.php

Other Records

Field Value
crawl-delay 2

crazywebcrawler

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sightupbot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

java/

Rule Path
Disallow /