mogul.sg
robots.txt

Robots Exclusion Standard data for mogul.sg

Resource Scan

Scan Details

Site Domain mogul.sg
Base Domain mogul.sg
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-05T03:04:49+00:00
Next Scan 2026-03-05T03:04:49+00:00

Last Successful Scan

Scanned2024-07-20T12:19:08+00:00
URL https://mogul.sg/robots.txt
Redirect https://www.mogul.sg/robots.txt
Redirect Domain www.mogul.sg
Redirect Base mogul.sg
Domain IPs 104.21.63.233, 172.67.173.22, 2606:4700:3030::6815:3fe9, 2606:4700:3034::ac43:ad16
Redirect IPs 104.21.63.233, 172.67.173.22, 2606:4700:3030::6815:3fe9, 2606:4700:3034::ac43:ad16
Response IP 104.21.63.233
Found Yes
Hash b295e57f350e5a61224330e0a9e800c37bd74f7c26f19a9564568ff898e77eb7
SimHash 4a33da64c6f2

Groups

googlebot

Rule Path
Disallow /account
Disallow /login

Other Records

Field Value
crawl-delay 120

bingbot

Rule Path
Disallow /account
Disallow /login

Other Records

Field Value
crawl-delay 4

slurp

Rule Path
Disallow /account
Disallow /login

Other Records

Field Value
crawl-delay 4

trovitbot

Rule Path
Disallow /

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

*

Rule Path
Disallow /api

Other Records

Field Value
sitemap https://www.mogul.sg/sitemap.xml

Warnings

  • `host` is not a known field.