techfuturae.com
robots.txt
Robots Exclusion Standard data for techfuturae.com
Resource Scan
Scan Details
| Site Domain | techfuturae.com |
| Base Domain | techfuturae.com |
| Scan Status | Ok |
| Last Scan | 2024-11-11T02:50:52+00:00 |
| Next Scan | 2024-11-18T02:50:52+00:00 |
Last Scan
| Scanned | 2024-11-11T02:50:52+00:00 |
| URL | https://techfuturae.com/robots.txt |
| Redirect | https://www.techfuturae.com/robots.txt |
| Redirect Domain | www.techfuturae.com |
| Redirect Base | techfuturae.com |
| Domain IPs | 104.19.154.92 |
| Redirect IPs | 104.16.150.108, 104.16.151.108, 2606:4700::6810:966c, 2606:4700::6810:976c |
| Response IP | 104.16.150.108 |
| Found | Yes |
| Hash | 0776f258e8c71b85993f245931e0f2ece6d5b176c486fe3b2b8e1e16cd23b1f0 |
| SimHash | 1a61dc12c800 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /?s= |
| Disallow | /search/ |
| Disallow | /wp-admin |
| Disallow | /*/feed/ |
| Disallow | /wp-login.php |
| Disallow | /wp-admin/ |
| Disallow | /go/ |
| Disallow | /trackback/ |
| Disallow | /wp-register.php |
| Allow | /wp-admin/admin-ajax.php |
screaming frog seo spider
spbot
exabot
gigabot
superbot
superhttp
netspider
deepcrawl
website\ extractor
website\ quester
webstripper
oncrawl
claritybot
webwhacker
surfbot
dotbot
rytebot
rogerbot
semrushbot
semrushbot-sa
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
semrushbot-seoab
semrush
mj12bot
xenu
searchmetricsbot
sistrix crawler
seokicks-robot
ia_archiver
sitebulb
archive.org_bot
ia_archiver-web.archive.org
ravencrawler
blexbot
seo-powersuite-bot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.techfuturae.com/sitemap_index.xml |
| sitemap | https://www.techfuturae.com/post-sitemap.xml |
| sitemap | https://www.techfuturae.com/page-sitemap.xml |