rewardia.com
robots.txt

Robots Exclusion Standard data for rewardia.com

Resource Scan

Scan Details

Site Domain rewardia.com
Base Domain rewardia.com
Scan Status Ok
Last Scan2024-11-12T08:03:18+00:00
Next Scan 2024-11-19T08:03:18+00:00

Last Scan

Scanned2024-11-12T08:03:18+00:00
URL https://rewardia.com/robots.txt
Redirect https://www.rewardia.com/robots.txt
Redirect Domain www.rewardia.com
Redirect Base rewardia.com
Domain IPs 69.167.186.40
Redirect IPs 69.167.186.40
Response IP 69.167.186.40
Found Yes
Hash 3d103b59a1eeeffab134c702af1fd68762981c72c07d49bd6e18a8a5b340151f
SimHash 9cbe7ada42e7

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /fun/
Disallow /_
Disallow /GameRd.aspx
Disallow /*?pk=rr
Disallow /*?pk=xt
Disallow /*?pk=fb
Disallow /*?pk=xx
Disallow /*?pk=so
Disallow /*?pk=ss
Disallow /*?pk=st
Disallow /*?pk=go
Disallow /*?pk=cp
Disallow /*?pk=ds
Disallow /*?pk=pc
Disallow /*?pk=pu
Disallow /*?pk=ec
Disallow /*?pk=qc
Disallow /*?pk=qp
Disallow /*?pk=rc
Disallow /*?pk=bg
Disallow /*?pk=cg
Disallow /*?pk=rp
Disallow /*?pk=sp
Disallow /*?pk=sv
Disallow /*?pk=sc
Disallow /*?pk=qv
Disallow /*?pk=ev
Disallow /*?pk=rv
Disallow /*?pk=sp
Disallow /*?pk=sg
Disallow /*?pk=rd
Disallow /*?pk=pl
Disallow /*?pk=pf
Disallow /*.xslt
Disallow /*.xml
Disallow /*?*Sort=*

yandex

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

cherry picker

Rule Path
Disallow /

crescent internet toolpak

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

teleport

Rule Path
Disallow /

telesoft

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

vspider

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webemailextractor

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

israbot

Rule Path
Disallow

orthogaffe

Rule Path
Disallow

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

internetseer.com

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

grub

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

htdig

Rule Path
Disallow /

httrack

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

netprospector

Rule Path
Disallow /

perman surfer

Rule Path
Disallow /

psbot

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

yahooseeker

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

Warnings

  • 2 invalid lines.