pinprestige.com
robots.txt

Robots Exclusion Standard data for pinprestige.com

Resource Scan

Scan Details

Site Domain pinprestige.com
Base Domain pinprestige.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-15T16:40:23+00:00
Next Scan 2024-06-14T16:40:23+00:00

Last Successful Scan

Scanned2024-01-24T16:00:15+00:00
URL https://pinprestige.com/robots.txt
Redirect https://www.pinprestige.com/robots.txt
Redirect Domain www.pinprestige.com
Redirect Base pinprestige.com
Domain IPs 104.21.57.126, 172.67.163.225, 2606:4700:3032::6815:397e, 2606:4700:3034::ac43:a3e1
Redirect IPs 18.155.68.114, 18.155.68.16, 18.155.68.76, 18.155.68.89, 2600:9000:23d2:1600:12:ba6e:34c0:93a1, 2600:9000:23d2:2200:12:ba6e:34c0:93a1, 2600:9000:23d2:400:12:ba6e:34c0:93a1, 2600:9000:23d2:6e00:12:ba6e:34c0:93a1, 2600:9000:23d2:9400:12:ba6e:34c0:93a1, 2600:9000:23d2:a00:12:ba6e:34c0:93a1, 2600:9000:23d2:b200:12:ba6e:34c0:93a1, 2600:9000:23d2:ce00:12:ba6e:34c0:93a1
Response IP 18.66.97.4
Found Yes
Hash b2b600ac72308c658131f2eed3f191e099e20cab5dc80d7a1bc1a47acf22e11e
SimHash 100b6070ce31

Groups

*

Rule Path
Allow /
Disallow /*/wp-admin
Disallow /*/wp-login.php
Disallow *?s=*
Disallow *?search=*
Disallow */newsletter/*
Disallow /wp-content/plugins/
Disallow */?post*
Disallow /*?amp=*

yak/1.0

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

buck

Rule Path
Disallow /

addthis

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

webmeupbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

trendiction

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

trendiction.com

Rule Path
Disallow /

trendiction.de

Rule Path
Disallow /

mediamathbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

urx-api/*

Rule Path
Disallow /

urxbot/*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.pinprestige.com/sg/news-sitemap.xml
sitemap https://www.pinprestige.com/sg/sitemap_index.xml
sitemap https://www.pinprestige.com/my/news-sitemap.xml
sitemap https://www.pinprestige.com/my/sitemap_index.xml