Log In Sign Up

priceme.com
robots.txt

Robots Exclusion Standard data for priceme.com

Resource Scan

Scan Details

Site Domain priceme.com
Base Domain priceme.com
Scan Status Ok
Last Scan2024-11-09T11:53:36+00:00
Next Scan 2024-11-16T11:53:36+00:00

Last Scan

Scanned2024-11-09T11:53:36+00:00
URL https://priceme.com/robots.txt
Redirect https://sg.priceme.com/robots.txt
Redirect Domain sg.priceme.com
Redirect Base priceme.com
Domain IPs 178.63.45.174
Redirect IPs 104.26.10.248, 104.26.11.248, 172.67.68.123, 2606:4700:20::681a:af8, 2606:4700:20::681a:bf8, 2606:4700:20::ac43:447b
Response IP 172.67.68.123
Found Yes
Hash 3e7da491a29d37e57c515a0a7024c1c6eeacb7577aa175c1c523c31637aca74a
SimHash e01c509ace9d

Groups

*

Rule Path
Disallow /*/*%7D%7D$
Disallow /*.axd$
Disallow /shops-nearby
Disallow /mycart*
Disallow /privacypolicy
Disallow /Feedback?*
Disallow /blog/wp-login.php*
Disallow /RetailerReview/
Disallow /.axd$
Disallow /JudgementLogin
Disallow /JudgementLogin*
Disallow /GetCatalogfav
Disallow /GetProductfav
Disallow /GetProductList
Disallow /consumer/feed
Disallow /consumer/wp-admin/
Disallow /money/termdepositspostback.aspx*
Disallow /money/savingsaccountspostback.aspx*
Disallow /Compare.aspx*

blexbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seznambot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

teemer

Rule Path
Disallow /

netseer

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

eurobot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

wget

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

onet.pl

Rule Path
Disallow /

yandex

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

cyberpatrol

Rule Path
Disallow /

houxoucrawler

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

dumbbot

Rule Path
Disallow /

generic

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

nexabot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

owr_crawler

Rule Path
Disallow /

rabaz

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

scoutabout

Rule Path
Disallow /

semanticdiscovery

Rule Path
Disallow /

cherry picker

Rule Path
Disallow /

crescent internet toolpak

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

teleport

Rule Path
Disallow /

telesoft

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

vspider

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webemailextractor

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

israbot

Rule Path
Disallow

orthogaffe

Rule Path
Disallow

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

internetseer.com

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

grub

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

htdig

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

netprospector

Rule Path
Disallow /

perman surfer

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

yahooseeker

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://sg.priceme.com/sitemap/SiteMap.xml
Back to top

Comments

  • file for https://sg.priceme.com
  • disallow rules for AI crawlers (ChatGPT, Claude etc.)
Back to top