thehauterfly.com
robots.txt
Robots Exclusion Standard data for thehauterfly.com
Resource Scan
Scan Details
Site Domain | thehauterfly.com |
Base Domain | thehauterfly.com |
Scan Status | Ok |
Last Scan | 2024-05-23T09:24:26+00:00 |
Next Scan | 2024-05-30T09:24:26+00:00 |
Last Scan
Scanned | 2024-05-23T09:24:26+00:00 |
URL | https://thehauterfly.com/robots.txt |
Redirect | https://hauterrfly.com:443/robots.txt |
Redirect Domain | hauterrfly.com |
Redirect Base | hauterrfly.com |
Domain IPs | 13.126.52.15, 15.206.46.68, 3.111.255.19 |
Redirect IPs | 13.225.103.115, 13.225.103.126, 13.225.103.19, 13.225.103.6 |
Response IP | 13.35.18.128 |
Found | Yes |
Hash | a8dfedb6acefb2e7e77fab601235d5567acddfd4b631265cdcb770bbde5912d5 |
SimHash | 48246249a233 |
Groups
*
Rule | Path |
---|---|
Allow | / |
Disallow | */feed$ |
Disallow | */?s=* |
Disallow | /wp-admin/ |
Disallow | /wp-content/plugins/ |
Disallow | /events/ |
Disallow | /wp-content/sitemaps/pagination-sitemap.xml |
Disallow | /page/ |
Disallow | */2018/* |
Disallow | */2019/* |
Disallow | */2015/* |
Disallow | */2017/* |
Disallow | */2016/* |
Disallow | */?q=%2F* |
Disallow | */webp-express/* |
Disallow | */embed/?embed=true |
Disallow | */web-stories/page/* |
Disallow | */search/* |
Disallow | */embed/* |
Disallow | */page/* |
Disallow | */Kinjal |
Disallow | */undefined/* |
Disallow | */about%3Ablank* |
Disallow | */podcast/* |
Disallow | /xmlrpc.php |
Disallow | /*sex* |
Disallow | /tag/sex/ |
Disallow | /home/ |
Other Records
Field | Value |
---|---|
sitemap | https://hauterrfly.com/post-sitemap1.xml |
sitemap | https://hauterrfly.com/post-sitemap2.xml |
sitemap | https://hauterrfly.com/post-sitemap3.xml |
sitemap | https://hauterrfly.com/post-sitemap4.xml |
sitemap | https://hauterrfly.com/post-sitemap5.xml |
sitemap | https://hauterrfly.com/page-sitemap.xml |
sitemap | https://hauterrfly.com/category-sitemap.xml |
sitemap | https://hauterrfly.com/feed/podcast/the-tits-bits |