gumtree.co.za
robots.txt
Robots Exclusion Standard data for gumtree.co.za
Resource Scan
Scan Details
| Site Domain | gumtree.co.za |
| Base Domain | gumtree.co.za |
| Scan Status | Ok |
| Last Scan | 2024-09-27T22:34:52+00:00 |
| Next Scan | 2024-10-04T22:34:52+00:00 |
Last Scan
| Scanned | 2024-09-27T22:34:52+00:00 |
| URL | https://gumtree.co.za/robots.txt |
| Redirect | https://www.gumtree.co.za/robots.txt |
| Redirect Domain | www.gumtree.co.za |
| Redirect Base | gumtree.co.za |
| Domain IPs | 13.107.213.56, 13.107.246.56 |
| Redirect IPs | 13.107.246.59, 2620:1ec:bdf::59 |
| Response IP | 13.107.246.59 |
| Found | Yes |
| Hash | c068da30258bebf6c1588d9b1cd59f885dfc07ab729fc089ef6a9c1c754f6a98 |
| SimHash | 4c045ed06cd0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | *?*order=* |
| Disallow | *?*galleryView=* |
| Disallow | *?*sort=* |
| Disallow | *?*q=* |
| Disallow | *?*sr=* |
| Disallow | /f-pbfvb/* |
| Disallow | *?*be=* |
| Disallow | *?*fe=* |
| Disallow | *sr~1$ |
| Disallow | *sr~2$ |
| Disallow | *sr~8$ |
| Disallow | /s-*/page-*/v1*q0p* |
| Disallow | /306629115/* |
| Disallow | /bolt-2dot0-frontend$ |
| Disallow | /7162/* |
| Disallow | *?*AdType=* |
| Disallow | *?*categoryId=* |
| Disallow | *?*isSearchForm=* |
| Disallow | *?*km=* |
| Disallow | *?*locationId=* |
| Disallow | *?*pr=* |
| Disallow | *?*redirect=* |
| Disallow | *?*redirectToParent* |
| Disallow | *?*skipAttrFilter=* |
| Disallow | *?status=* |
| Disallow | /forgotPassword.html?* |
| Disallow | /register.html?* |
| Disallow | /login.html?* |
| Disallow | /post.html?* |
| Disallow | *?*priceType=* |
| Disallow | /rui-api/* |
| Disallow | *.ugzy |
bingbot
| Rule | Path |
|---|---|
| Disallow | *?*order=* |
| Disallow | *?*galleryView=* |
| Disallow | *?*sort=* |
| Disallow | *?*q=* |
| Disallow | *?*sr=* |
| Disallow | /f-pbfvb/* |
| Disallow | *?*be=* |
| Disallow | *?*fe=* |
| Disallow | *sr~1$ |
| Disallow | *sr~2$ |
| Disallow | *sr~8$ |
| Disallow | /s-*/page-*/v1*q0p* |
| Disallow | /306629115/* |
| Disallow | /bolt-2dot0-frontend$ |
| Disallow | /7162/* |
| Disallow | *?*AdType=* |
| Disallow | *?*categoryId=* |
| Disallow | *?*isSearchForm=* |
| Disallow | *?*km=* |
| Disallow | *?*locationId=* |
| Disallow | *?*pr=* |
| Disallow | *?*redirect=* |
| Disallow | *?*redirectToParent* |
| Disallow | *?*skipAttrFilter=* |
| Disallow | *?status=* |
| Disallow | /forgotPassword.html?* |
| Disallow | /register.html?* |
| Disallow | /login.html?* |
| Disallow | /post.html?* |
| Disallow | *?*priceType=* |
| Disallow | /rui-api/* |
| Disallow | *.ugzy |
| Disallow | */undefined$ |
adsbot-google
| Rule | Path |
|---|---|
| Disallow | *?*order=* |
| Disallow | *?*galleryView=* |
| Disallow | *?*sort=* |
| Disallow | *?*q=* |
| Disallow | *?*sr=* |
| Disallow | /f-pbfvb/* |
| Disallow | *?*be=* |
| Disallow | *?*fe=* |
| Disallow | *sr~1$ |
| Disallow | *sr~2$ |
| Disallow | *sr~8$ |
| Disallow | /s-*/page-*/v1*q0p* |
| Disallow | /306629115/* |
| Disallow | /bolt-2dot0-frontend$ |
| Disallow | /7162/* |
| Disallow | *?*AdType=* |
| Disallow | *?*categoryId=* |
| Disallow | *?*isSearchForm=* |
| Disallow | *?*km=* |
| Disallow | *?*locationId=* |
| Disallow | *?*pr=* |
| Disallow | *?*redirect=* |
| Disallow | *?*redirectToParent* |
| Disallow | *?*skipAttrFilter=* |
| Disallow | *?status=* |
| Disallow | /forgotPassword.html?* |
| Disallow | /register.html?* |
| Disallow | /login.html?* |
| Disallow | /post.html?* |
| Disallow | *?*priceType=* |
| Disallow | /rui-api/* |
| Disallow | *.ugzy |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | *?*order=* |
| Disallow | *?*galleryView=* |
| Disallow | *?*sort=* |
| Disallow | *?*q=* |
| Disallow | *?*sr=* |
| Disallow | /f-pbfvb/* |
| Disallow | *?*be=* |
| Disallow | *?*fe=* |
| Disallow | *sr~1$ |
| Disallow | *sr~2$ |
| Disallow | *sr~8$ |
| Disallow | /s-*/page-*/v1*q0p* |
| Disallow | /306629115/* |
| Disallow | /bolt-2dot0-frontend$ |
| Disallow | /7162/* |
| Disallow | *?*AdType=* |
| Disallow | *?*categoryId=* |
| Disallow | *?*isSearchForm=* |
| Disallow | *?*km=* |
| Disallow | *?*locationId=* |
| Disallow | *?*pr=* |
| Disallow | *?*redirect=* |
| Disallow | *?*redirectToParent* |
| Disallow | *?*skipAttrFilter=* |
| Disallow | *?status=* |
| Disallow | /forgotPassword.html?* |
| Disallow | /register.html?* |
| Disallow | /login.html?* |
| Disallow | /post.html?* |
| Disallow | *?*priceType=* |
| Disallow | /rui-api/* |
| Disallow | *.ugzy |
googlebot
| Rule | Path |
|---|---|
| Disallow | *?*order=* |
| Disallow | *?*galleryView=* |
| Disallow | *?*sort=* |
| Disallow | *?*q=* |
| Disallow | *?*sr=* |
| Disallow | /f-pbfvb/* |
| Disallow | *?*be=* |
| Disallow | *?*fe=* |
| Disallow | *sr~1$ |
| Disallow | *sr~2$ |
| Disallow | *sr~8$ |
| Disallow | /s-*/page-*/v1*q0p* |
| Disallow | /306629115/* |
| Disallow | /bolt-2dot0-frontend$ |
| Disallow | /7162/* |
| Disallow | *?*AdType=* |
| Disallow | *?*categoryId=* |
| Disallow | *?*isSearchForm=* |
| Disallow | *?*km=* |
| Disallow | *?*locationId=* |
| Disallow | *?*pr=* |
| Disallow | *?*redirect=* |
| Disallow | *?*redirectToParent* |
| Disallow | *?*skipAttrFilter=* |
| Disallow | *?status=* |
| Disallow | /forgotPassword.html?* |
| Disallow | /register.html?* |
| Disallow | /login.html?* |
| Disallow | /post.html?* |
| Disallow | *?*priceType=* |
| Disallow | /rui-api/* |
| Disallow | *.ugzy |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.gumtree.co.za/sitemaps/za/vip/vip_index.xml |
| sitemap | https://www.gumtree.co.za/sitemaps/za/loccat/loccat_index.xml |
| sitemap | https://www.gumtree.co.za/sitemaps/za/loccatattr/loccatattr_index.xml |
Comments