wupy.xyz
robots.txt

Robots Exclusion Standard data for wupy.xyz

Resource Scan

Scan Details

Site Domain wupy.xyz
Base Domain wupy.xyz
Scan Status Ok
Last Scan2024-04-22T13:50:10+00:00
Next Scan 2024-05-22T13:50:10+00:00

Last Scan

Scanned2024-04-22T13:50:10+00:00
URL http://wupy.xyz/robots.txt
Domain IPs 184.73.186.224
Response IP 184.73.186.224
Found Yes
Hash c21299ee8fa6da2addbb4a4abac123efdf4541948d594c09cbfc9f8f9a1dbe69
SimHash 4e234370f3b7

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow */format%3Athumbnail
Disallow /download/
Disallow /*/download?
Disallow /couchmode/
Disallow /groups/*/forum/*
Disallow /musicstore/preview
Disallow /musicstore/download
Disallow /forgot_password
Disallow /help/forgot_login
Disallow /help/verify_email
Disallow /help/zendesk_sso
Disallow /help/sso
Disallow /user/verify
Disallow /search
Disallow /_*
Allow /_rv/viewer
Allow /_next/viewer
Allow /_rv/gtm
Allow /_next/gtm
Allow /_global_notifications
Allow /_rv/jwt
Allow /_next/jwt
Allow /_rv/title
Allow /_next/title

Other Records

Field Value
crawl-delay 2

seznambot

Rule Path
Disallow */format%3Athumbnail
Disallow /download/
Disallow /*/download?
Disallow /couchmode/
Disallow /musicstore/preview
Disallow /musicstore/download
Disallow /forgot_password
Disallow /help/forgot_login
Disallow /help/verify_email
Disallow /help/zendesk_sso
Disallow /help/sso
Disallow /api/
Disallow /user/verify
Disallow /search/
Disallow /search?
Disallow /search%5C?
Disallow /_*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Comments

  • You know, for robots
  • robots@vimeo.com

Warnings

  • `request-rate` is not a known field.