progesmarket.com
robots.txt

Robots Exclusion Standard data for progesmarket.com

Resource Scan

Scan Details

Site Domain progesmarket.com
Base Domain progesmarket.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-12-02T00:23:08+00:00
Next Scan 2026-01-01T00:23:08+00:00

Last Successful Scan

Scanned2025-11-01T19:56:13+00:00
URL https://progesmarket.com/robots.txt
Domain IPs 147.93.92.225, 2a02:4780:27:2157:0:37f9:965d:10
Response IP 147.93.92.225
Found Yes
Hash 776b69819fd2a9a0005c4d722ead327b2ec0f2835acc79e6ff62d2fa73ce9957
SimHash 79849824cad2

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /login
Disallow /register
Disallow /password/*
Disallow /email/verify/*
Disallow /two-factor-challenge
Disallow /user/profile-information
Disallow /user/two-factor-authentication
Disallow /user/confirmed-password-status
Disallow /user/confirmed-two-factor-authentication
Disallow /user/profile-photos
Disallow /user/other-browser-sessions
Disallow /user/delete-profile
Disallow /user/logout-other-browser-sessions
Disallow /user/two-factor-qr-code
Disallow /user/two-factor-secret-key
Disallow /user/two-factor-recovery-codes
Disallow /user/api-tokens
Disallow /user/delete-api-token
Disallow /user/create-api-token
Disallow /user/update-api-token
Disallow /user/update-password
Disallow /user/update-profile-information
Disallow /user/update-profile-photo
Disallow /user/delete-profile-photo
Disallow /user/update-two-factor-authentication
Disallow /user/update-password-via-token
Disallow /user/confirm-password
Disallow /user/confirm-two-factor-authentication
Disallow /user/generate-two-factor-recovery-codes
Disallow /user/update-api-token-permissions
Disallow /user/update-api-token-scopes
Disallow /user/update-api-token-abilities
Disallow /user/update-api-token-abilities-via-token
Disallow /user/update-api-token-permissions-via-token
Disallow /user/update-api-token-scopes-via-token
Disallow /user/update-api-token-abilities-via-token
Disallow /user/update-api-token-permissions-via-token
Disallow /user/update-api-token-scopes-via-token

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.progesmarket.com/sitemap.xml

Comments

  • Sitemap
  • Crawl-delay