masonsmart.com
robots.txt
Robots Exclusion Standard data for masonsmart.com
Resource Scan
Scan Details
| Site Domain | masonsmart.com |
| Base Domain | masonsmart.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a server error. |
| Last Scan | 2024-08-21T06:19:48+00:00 |
| Next Scan | 2024-11-19T06:19:48+00:00 |
Last Successful Scan
| Scanned | 2023-04-30T03:17:29+00:00 |
| URL | http://masonsmart.com/robots.txt |
| Redirect | http://www.masonsmart.com/robots.txt |
| Redirect Domain | www.masonsmart.com |
| Redirect Base | masonsmart.com |
| Domain IPs | 13.250.129.152, 2406:da18:9d0:143e:8e74:1b1a:98b9:2813, 2406:da18:9d0:143f:29e7:ae24:cfea:e9bb, 54.151.156.30 |
| Redirect IPs | 104.21.78.233, 172.67.138.105, 2606:4700:3032::ac43:8a69, 2606:4700:3037::6815:4ee9 |
| Response IP | 104.21.78.233 |
| Found | Yes |
| Hash | e866d8391020d79abda8c0b00bfaad26d0c6aa8b6c61453b0ddb749e399540a8 |
| SimHash | 695bda815095 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /bin/ |
| Disallow | /dyn/ |
| Disallow | /objects/ |
| Disallow | /help/ |
| Disallow | /no-such-url.html |
| Disallow | /fb/channel.html |
| Disallow | /*.pdf$ |
googlebot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /dyn/ |
| Disallow | /help/ |
| Disallow | /no-such-url.html |
| Disallow | /fb/channel.html |
| Disallow | /*.pdf$ |
googlebot-mobile
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /bin/ |
| Disallow | /dyn/ |
| Disallow | /help/ |
| Disallow | /no-such-url.html |
| Disallow | /fb/channel.html |
| Disallow | /*.pdf$ |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /bin/ |
| Disallow | /dyn/ |
| Disallow | /help/ |
| Disallow | /no-such-url.html |
| Disallow | /fb/channel.html |
| Disallow | /*.pdf$ |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /bin/ |
| Disallow | /dyn/ |
| Disallow | /help/ |
| Disallow | /fb/channel.html |
| Disallow | /no-such-url.html |
| Disallow | /*.pdf$ |
bingbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /bin/ |
| Disallow | /dyn/ |
| Disallow | /objects/ |
| Disallow | /help/ |
| Disallow | /no-such-url.html |
| Disallow | /fb/channel.html |
| Disallow | /*.pdf$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /bin/ |
| Disallow | /dyn/ |
| Disallow | /objects/ |
| Disallow | /help/ |
| Disallow | /no-such-url.html |
| Disallow | /fb/channel.html |
| Disallow | /*.pdf$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.masonsmart.com/gTcLmm0w.xml |
Comments