revenuegeeks.com
robots.txt
Robots Exclusion Standard data for revenuegeeks.com
Resource Scan
Scan Details
| Site Domain | revenuegeeks.com |
| Base Domain | revenuegeeks.com |
| Scan Status | Ok |
| Last Scan | 2024-11-05T21:09:28+00:00 |
| Next Scan | 2024-12-05T21:09:28+00:00 |
Last Scan
| Scanned | 2024-11-05T21:09:28+00:00 |
| URL | https://revenuegeeks.com/robots.txt |
| Domain IPs | 104.16.150.108, 104.16.151.108, 2606:4700::6810:966c, 2606:4700::6810:976c |
| Response IP | 104.16.150.108 |
| Found | Yes |
| Hash | 56d99b7aef8452a70e5ca182029de89cc137fdd7a1164f97f899059685229cd3 |
| SimHash | 525554565850 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /?s=* |
| Disallow | /search/* |
| Disallow | /api/ |
| Disallow | /wp-admin |
| Disallow | /wp-login.php |
| Disallow | /wp-admin/ |
| Disallow | /go/ |
| Disallow | /*//1000$ |
| Disallow | /trackback/ |
| Disallow | /*/1000$ |
| Disallow | /wp-register.php |
| Allow | /wp-admin/admin-ajax.php |
screaming frog seo spider
spbot
exabot
gigabot
superbot
superhttp
netspider
bytespider
website\ extractor
website\ quester
webstripper
webwhacker
surfbot
oncrawl
deepcrawl
rytebot
dotbot
claritybot
rogerbot
sitebulb
semrushbot
semrushbot-sa
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
semrushbot-seoab
semrush
mj12bot
xenu
searchmetricsbot
sistrix crawler
seokicks-robot
ia_archiver
archive.org_bot
ia_archiver-web.archive.org
ravencrawler
blexbot
seo-powersuite-bot
ltx71
nutch
httrack
curl
wget
python-urllib
ahrefsbot
surferseo
mozbot
serpstatbot
screpybot
siteanalyzerbot
seolyzer
lipperhey
serprobot
zyte
octoparse
parsehub
scrapy
diffbot
botify
checkbot
dareboost
seomator
seo-browser
webceo
woorank
buzzsumo
stormcrawler
screaming frog
openlinkprofiler
seobilitybot
seobility
linkpadbot
screamingfrog
opensiteexplorer
backlinkcrawler
linkdiagnosis
linkdex
linkody
linkresearchtools
monitorbacklinks
rankactive
rankranger
rankwatch
seomonitor
seoprofiler
sheerseo
sitecheckerbot
webmeup
yacybot
archivebot
loadtimebot
spinn3r
lipperhey-kaus-australis
pagepeeker
webcapture
ftrf: friendly robot coming from feeder.co
metauri
prerender
queryseekerspider
twingly recon
velenpublicwebcrawler
obot
seoscanners.net
aspiegelbot
seekport crawler
genieo
touche
ezooms
findxbot
wotbox
everyonesocialbot
citygrid
grapeshotcrawler
nerdybot
toplistbot
urlappendbot
vegi bot
trendictionbot
brandverity
mediatoolkitbot
plukkie
surebot
tineye-bot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://revenuegeeks.com/sitemap_index.xml |
| sitemap | https://revenuegeeks.com/post-sitemap.xml |
| sitemap | https://revenuegeeks.com/page-sitemap.xml |
Warnings
- 1 invalid line.