pegipegi.com
robots.txt
Robots Exclusion Standard data for pegipegi.com
Resource Scan
Scan Details
| Site Domain | pegipegi.com |
| Base Domain | pegipegi.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a server error. |
| Last Scan | 2024-07-05T03:56:23+00:00 |
| Next Scan | 2024-10-03T03:56:23+00:00 |
Last Successful Scan
| Scanned | 2023-12-02T03:54:39+00:00 |
| URL | https://pegipegi.com/robots.txt |
| Redirect | https://www.pegipegi.com/robots.txt |
| Redirect Domain | www.pegipegi.com |
| Redirect Base | pegipegi.com |
| Domain IPs | 13.213.169.245, 18.136.192.17 |
| Redirect IPs | 13.250.129.222, 3.1.104.174 |
| Response IP | 3.1.104.174 |
| Found | Yes |
| Hash | cec4fd07cb23be29064739c35ee6e2ffcf58bdb531933e14fc83f96d08cef83e |
| SimHash | e4014385c32a |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /cgi-bin/ |
| Disallow | /pembayaran/kredivo/finish.html |
| Disallow | /mobile/pembayaran/kredivo/finish.html |
| Disallow | /rsc/common/js/base/TripadvisorShow.js |
| Disallow | /rsc/common/mobile/js/TripadvisorShowSp.js |
| Disallow | /HotelUniqueReview/v1/hotel/tripadvisor/review/list |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.pegipegi.com/siteindex-hotel.xml |
| sitemap | https://www.pegipegi.com/flight/xml/airlines-sitemap.xml |
| sitemap | https://www.pegipegi.com/flight/xml/airlinesmonth-sitemap.xml |
| sitemap | https://www.pegipegi.com/flight/xml/area-sitemap.xml |
| sitemap | https://www.pegipegi.com/flight/xml/areadate-sitemap.xml |
| sitemap | https://www.pegipegi.com/flight/xml/areatoarea-sitemap.xml |
| sitemap | https://www.pegipegi.com/flight/xml/areatoareamonth-sitemap-one.xml |
| sitemap | https://www.pegipegi.com/flight/xml/areatoareamonth-sitemap-two.xml |
| sitemap | https://www.pegipegi.com/flight/xml/areatoareamonth-sitemap-three.xml |
| sitemap | https://www.pegipegi.com/flight/xml/promo-scheduledate-sitemap.xml |
| sitemap | https://www.pegipegi.com/kereta-api/xml/sitemap_train_area.xml |
| sitemap | https://www.pegipegi.com/kereta-api/xml/sitemap_train_areatoarea.xml |
| sitemap | https://www.pegipegi.com/travel/sitemap.xml |
| sitemap | https://www.pegipegi.com/siteindex-bus.xml |