newsmth.net
robots.txt
Robots Exclusion Standard data for newsmth.net
Resource Scan
Scan Details
| Site Domain | newsmth.net |
| Base Domain | newsmth.net |
| Scan Status | Ok |
| Last Scan | 2024-10-04T00:36:36+00:00 |
| Next Scan | 2024-10-11T00:36:36+00:00 |
Last Scan
| Scanned | 2024-10-04T00:36:36+00:00 |
| URL | https://newsmth.net/robots.txt |
| Redirect | https://www.newsmth.net/robots.txt |
| Redirect Domain | www.newsmth.net |
| Redirect Base | newsmth.net |
| Domain IPs | 120.92.211.176 |
| Redirect IPs | 120.92.211.176 |
| Response IP | 120.92.211.176 |
| Found | Yes |
| Hash | 5532c63787f9d08b7011e63d18c9142ae7eca2b982594073f9707163df7b7008 |
| SimHash | 2a54d220c485 |
Groups
msnbot
| Rule | Path |
|---|---|
| Allow | /index.html |
| Allow | /indexpages/ |
| Allow | /frames.html |
| Allow | /mainpage.html |
| Allow | /bbsguestleft.html |
| Allow | /bbs0an.php |
| Allow | /bbsanc.php |
| Allow | /bbstcon.php |
| Allow | /sitemap.html |
| Allow | /sitemap.xml |
| Allow | /pc/ |
| Allow | /nForum/mainpage |
| Allow | /nForum/elite |
| Allow | /nForum/article |
| Allow | /nForum/section |
| Disallow | / |
sogou
| Rule | Path |
|---|---|
| Allow | /index.html |
| Allow | /indexpages/ |
| Allow | /frames.html |
| Allow | /mainpage.html |
| Allow | /bbsguestleft.html |
| Allow | /bbs0an.php |
| Allow | /bbsanc.php |
| Allow | /bbstcon.php |
| Allow | /sitemap.html |
| Allow | /sitemap.xml |
| Allow | /pc/ |
| Allow | /nForum/mainpage |
| Allow | /nForum/elite |
| Allow | /nForum/article |
| Allow | /nForum/board |
| Allow | /nForum/section |
| Allow | /rss/ |
| Disallow | / |
slurp
| Rule | Path |
|---|---|
| Allow | /index.html |
| Allow | /indexpages/ |
| Allow | /frames.html |
| Allow | /mainpage.html |
| Allow | /bbsguestleft.html |
| Allow | /bbs0an.php |
| Allow | /bbsanc.php |
| Allow | /bbstcon.php |
| Allow | /sitemap.html |
| Allow | /sitemap.xml |
| Allow | /pc/ |
| Allow | /nForum/mainpage |
| Allow | /nForum/elite |
| Allow | /nForum/article |
| Allow | /nForum/section |
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
googlebot
| Rule | Path |
|---|---|
| Allow | /index.html |
| Allow | /indexpages/ |
| Allow | /frames.html |
| Allow | /mainpage.html |
| Allow | /bbsguestleft.html |
| Allow | /bbs0an.php |
| Allow | /bbsanc.php |
| Allow | /bbstcon.php |
| Allow | /sitemap.html |
| Allow | /sitemap.xml |
| Allow | /pc/ |
| Allow | /nForum/mainpage |
| Allow | /nForum/elite |
| Allow | /nForum/article |
| Allow | /nForum/board |
| Allow | /nForum/section |
| Allow | /nForum/googlead |
| Allow | /nForum/ |
| Allow | /rss/ |
| Disallow | / |
mediapartners-google
| Rule | Path |
|---|---|
| Allow | /index.html |
| Allow | /indexpages/ |
| Allow | /frames.html |
| Allow | /mainpage.html |
| Allow | /mainpage.php |
| Allow | /frames.php |
| Allow | /bbsleft.php |
| Allow | /bbsguestleft.html |
| Allow | /bbs0an.php |
| Allow | /bbsanc.php |
| Allow | /bbstcon.php |
| Allow | /sitemap.html |
| Allow | /sitemap.xml |
| Allow | /pc/ |
| Allow | /nForum/mainpage |
| Allow | /nForum/elite |
| Allow | /nForum/article |
| Allow | /nForum/board |
| Allow | /nForum/section |
| Allow | /rss/ |
| Disallow | / |
baiduspider
| Rule | Path |
|---|---|
| Allow | /index.html |
| Allow | /indexpages/ |
| Allow | /frames.html |
| Allow | /mainpage.html |
| Allow | /bbsguestleft.html |
| Allow | /bbs0an.php |
| Allow | /bbsanc.php |
| Allow | /bbstcon.php |
| Allow | /sitemap.html |
| Allow | /sitemap.xml |
| Allow | /pc/ |
| Allow | /nForum/mainpage |
| Allow | /nForum/elite |
| Allow | /nForum/article |
| Allow | /nForum/board |
| Allow | /nForum/section |
| Allow | /rss/ |
| Allow | /$ |
| Disallow | / |
jikespider
| Rule | Path |
|---|---|
| Allow | /index.html |
| Allow | /indexpages/ |
| Allow | /frames.html |
| Allow | /mainpage.html |
| Allow | /bbsguestleft.html |
| Allow | /bbs0an.php |
| Allow | /bbsanc.php |
| Allow | /bbstcon.php |
| Allow | /sitemap.html |
| Allow | /sitemap.xml |
| Allow | /pc/ |
| Allow | /nForum/mainpage |
| Allow | /nForum/elite |
| Allow | /nForum/article |
| Allow | /nForum/board |
| Allow | /nForum/section |
| Allow | /rss/ |
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.newsmth.net/sitemap.xml |
| sitemap | https://www.newsmth.net/sitemap.xml |