petergen.com
robots.txt
Robots Exclusion Standard data for petergen.com
Resource Scan
Scan Details
| Site Domain | petergen.com |
| Base Domain | petergen.com |
| Scan Status | Ok |
| Last Scan | 2025-09-26T04:49:03+00:00 |
| Next Scan | 2025-10-26T04:49:03+00:00 |
Last Scan
| Scanned | 2025-09-26T04:49:03+00:00 |
| URL | https://petergen.com/robots.txt |
| Domain IPs | 45.130.41.159 |
| Response IP | 45.130.41.159 |
| Found | Yes |
| Hash | 2204148851ccbf6c079c7c51405f94aa51e69be71dd2ccd10fccc009152306e0 |
| SimHash | f5f0d9841915 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /paymentok.php |
| Disallow | /download.php |
| Disallow | /showpages.php |
| Disallow | /domknigispb.php |
| Disallow | /getfile.php |
| Disallow | /productpage.php |
| Disallow | /obnul.php |
| Disallow | /register.php |
| Disallow | /tsylki.php |
| Disallow | /tsylki1.php |
| Disallow | /module2.php |
| Disallow | /module3.php |
| Disallow | /module4.php |
| Disallow | /module5.php |
| Disallow | /module6.php |
| Disallow | /module7.php |
| Disallow | /pnn/liteinf.php |
| Disallow | /pnn/bibliostat.php |
| Disallow | /pnn/bibliografia.php |
| Disallow | /pnn/testing.php |
| Disallow | /pnn/freedb.php |
| Disallow | /pnn/showrazdel.php |
| Disallow | /pnn/otvet.php |
| Disallow | /pnn/otvetuni.php |
| Disallow | /shop/buyfile.php |
| Disallow | /shop/consult.php |
| Disallow | /user/index.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Warnings
- `clean-param` is not a known field.