Log In Sign Up

priceme.com.my
robots.txt

Robots Exclusion Standard data for priceme.com.my

Resource Scan

Scan Details

Site Domain priceme.com.my
Base Domain priceme.com.my
Scan Status Ok
Last Scan2024-11-15T11:52:44+00:00
Next Scan 2024-11-22T11:52:44+00:00

Last Scan

Scanned2024-11-15T11:52:44+00:00
URL https://priceme.com.my/robots.txt
Redirect https://my.priceme.com/robots.txt
Redirect Domain my.priceme.com
Redirect Base priceme.com
Domain IPs 104.21.56.244, 172.67.138.233, 2606:4700:3032::6815:38f4, 2606:4700:3032::ac43:8ae9
Redirect IPs 104.26.10.248, 104.26.11.248, 172.67.68.123, 2606:4700:20::681a:af8, 2606:4700:20::681a:bf8, 2606:4700:20::ac43:447b
Response IP 104.26.11.248
Found Yes
Hash 2ee725e20576280ad42fbb8891b357b55e91ae63da063b19c7ad2fe104c8db9e
SimHash e01c508bce95

Groups

*

Rule Path
Disallow /*/*%7D%7D$
Disallow /*.axd$
Disallow /shops-nearby
Disallow /mycart*
Disallow /privacypolicy
Disallow /Feedback?*
Disallow /blog/wp-login.php*
Disallow /RetailerReview/
Disallow /.axd$
Disallow /JudgementLogin
Disallow /JudgementLogin*
Disallow /GetCatalogfav
Disallow /GetProductfav
Disallow /GetProductList
Disallow /consumer/feed
Disallow /consumer/wp-admin/
Disallow /money/termdepositspostback.aspx*
Disallow /money/savingsaccountspostback.aspx*
Disallow /Compare.aspx*

blexbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seznambot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

teemer

Rule Path
Disallow /

netseer

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

eurobot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

wget

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

onet.pl

Rule Path
Disallow /

yandex

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

cyberpatrol

Rule Path
Disallow /

houxoucrawler

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

dumbbot

Rule Path
Disallow /

generic

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

nexabot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

owr_crawler

Rule Path
Disallow /

rabaz

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

scoutabout

Rule Path
Disallow /

semanticdiscovery

Rule Path
Disallow /

cherry picker

Rule Path
Disallow /

crescent internet toolpak

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

teleport

Rule Path
Disallow /

telesoft

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

vspider

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webemailextractor

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

israbot

Rule Path
Disallow

orthogaffe

Rule Path
Disallow

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

internetseer.com

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

grub

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

htdig

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

netprospector

Rule Path
Disallow /

perman surfer

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

yahooseeker

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://my.priceme.com/sitemap/SiteMap.xml
Back to top

Comments

  • file for https://my.priceme.com
  • disallow rules for AI crawlers (ChatGPT, Claude etc.)
Back to top