johannesburg-gt-za.global-free-classified-ads.com
robots.txt
Robots Exclusion Standard data for johannesburg-gt-za.global-free-classified-ads.com
Resource Scan
Scan Details
| Site Domain | johannesburg-gt-za.global-free-classified-ads.com |
| Base Domain | global-free-classified-ads.com |
| Scan Status | Ok |
| Last Scan | 2024-09-22T14:38:22+00:00 |
| Next Scan | 2024-10-22T14:38:22+00:00 |
Last Scan
| Scanned | 2024-09-22T14:38:22+00:00 |
| URL | https://johannesburg-gt-za.global-free-classified-ads.com/robots.txt |
| Domain IPs | 104.26.10.146, 104.26.11.146, 172.67.69.25, 2606:4700:20::681a:a92, 2606:4700:20::681a:b92, 2606:4700:20::ac43:4519 |
| Response IP | 104.26.10.146 |
| Found | Yes |
| Hash | c3858d3efc17320f40bddba1001944c04e4f666fafa065113152b31ed20c94d7 |
| SimHash | 7d4ebbe8c604 |
Groups
emailcollector
emailsiphon
emailwolf
python-urllib
webzip
website quester
webster pro
wget
tocrawl/urldispatcher
amazonbot
semrushbot
semrushbot-sa
megaindex.ru
claudebot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Allow | /*?*a=19 |
| Allow | /*?*a=11 |
| Disallow | /show_help.php |
| Disallow | /*?*a=tag |
| Disallow | /*?a=ap |
| Disallow | /*?*a=13 |
| Disallow | /*?*a=18 |
| Disallow | /*?*a=20 |
| Disallow | /*?*a=4 |
| Disallow | /*?*a=1 |
| Disallow | /index.php?a=cart |
| Disallow | /*?*addon=sharing |
| Disallow | /other/ |
| Disallow | /cdn-cgi/ |
| Disallow | /help/print_sec_img.php |
| Disallow | /AJAX.php |
| Disallow | /votes/ |
| Disallow | /*?*hash= |
| Disallow | /get_image.php |
| Disallow | /seller- |
| Disallow | /.well-known/assetlinks.json |
| Disallow | /.well-known/apple-app-site-association |
| Disallow | /register.php?b= |
| Disallow | /ads.txt |
| Disallow | /site_off.htm |
| Disallow | /wp-* |
Comments