us.beruby.com
robots.txt
Robots Exclusion Standard data for us.beruby.com
Resource Scan
Scan Details
| Site Domain | us.beruby.com |
| Base Domain | beruby.com |
| Scan Status | Ok |
| Last Scan | 2024-11-06T18:24:18+00:00 |
| Next Scan | 2024-12-06T18:24:18+00:00 |
Last Scan
| Scanned | 2024-11-06T18:24:18+00:00 |
| URL | https://us.beruby.com/robots.txt |
| Domain IPs | 35.241.34.48 |
| Response IP | 35.241.34.48 |
| Found | Yes |
| Hash | 5c2502540f4d679abbdc628cd39a55ab7ebb208d006e3f6996ddaff489988221 |
| SimHash | 6489ab374d7e |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /account |
| Disallow | /admin |
| Disallow | /users |
| Disallow | /reviews |
| Disallow | /portal/password_forgotten |
| Disallow | /portal/welcome |
| Disallow | /portal/landing/* |
| Disallow | /portal/landing_01 |
| Disallow | /portal/landing_02 |
| Disallow | /portal/landing_03 |
| Disallow | /portal/landing_04 |
| Disallow | /portal/landing_05 |
| Disallow | /portal/landing_06 |
| Disallow | /portal/search |
| Disallow | /portal/search_engine |
| Disallow | /users/balance |
| Disallow | /users/transactions |
| Disallow | /users/claims |
| Disallow | /users/my_network |
| Disallow | /users/webmaster |
| Disallow | /users/profile |
| Disallow | /users/edit |
| Disallow | /users/communication |
| Disallow | /user_requests/list/beruby |
| Disallow | /payment_request/new |
| Disallow | /nuevo-comentario-sobre-articulo |
| Disallow | /novo-comentario-sobre-loja |
| Disallow | /nuovo-commento-su-articolo |
| Disallow | /web |
| Disallow | /api |
| Disallow | /perfil |
| Disallow | /open_profile |
| Disallow | /profile |
| Disallow | /users/my_open_profile |
| Disallow | /profilo |
| Allow | /portal/landing/1000 |
| Allow | /web/what-is-beruby |
| Allow | /web/terms-and-conditions |
| Allow | /web/privacy-policy |
| Allow | /web/cookies_more_info |
| Allow | /web/beruby-cashback |
| Allow | /web/press |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Comments