thefly.com
robots.txt

Robots Exclusion Standard data for thefly.com

Resource Scan

Scan Details

Site Domain thefly.com
Base Domain thefly.com
Scan Status Ok
Last Scan2025-05-17T01:46:41+00:00
Next Scan 2025-05-24T01:46:41+00:00

Last Scan

Scanned2025-05-17T01:46:41+00:00
URL https://thefly.com/robots.txt
Domain IPs 34.111.240.49
Response IP 34.111.240.49
Found Yes
Hash db048876070ef553020c84458878b418b24b0d29e452c8cf6c9cf789c832de0c
SimHash de4d55184e91

Groups

laboraybot
semrushbot
barkrowler
petalbot
mj12bot
dotbot
blexbot
neevabot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow

*

Rule Path
Disallow /font/
Disallow /includes/
Disallow /modules/
Disallow /sounds/
Disallow /plugins/
Disallow /tests/
Disallow /tools/
Disallow /updates/
Disallow /update.php

Other Records

Field Value
sitemap https://thefly.com/sitemapIndex.php

Warnings

  • `//disallow` is not a known field.