mondaq.com
robots.txt
Robots Exclusion Standard data for mondaq.com
Resource Scan
Scan Details
| Site Domain | mondaq.com |
| Base Domain | mondaq.com |
| Scan Status | Ok |
| Last Scan | 2024-07-03T11:24:08+00:00 |
| Next Scan | 2024-08-02T11:24:08+00:00 |
Last Scan
| Scanned | 2024-07-03T11:24:08+00:00 |
| URL | https://mondaq.com/robots.txt |
| Domain IPs | 34.252.192.116, 54.72.232.206, 54.72.54.5 |
| Response IP | 54.72.232.206 |
| Found | Yes |
| Hash | 0096fe187dbe271371a0fdf6c3791c12c39f661ab46be8ff07ab0ebe67eb5df7 |
| SimHash | 9a9d5541cad0 |
Groups
mozilla/5.0 (compatible; archive.org_bot/heritrix-1.15.1-x +http://pandora.nla.gov.au/crawl.html),
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; becomebot/2.2.1; msie 6.0 compatible; +http://www.become.com/webmasters.html)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; becomebot/2.0.3; +http://www.become.com/webmasters.html)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; becomebot/2.0.1; +http://www.become.com/webmasters.html)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; becomebot/1.23; +http://www.become.com/webmasters.html)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0 (compatible; becomebot/1.70; msie 6.0 compatible; +http://www.become.com/webmasters.html)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /account/* |
| Disallow | /Account/* |
| Disallow | /admin/ |
| Disallow | /Admin/ |
| Disallow | /advicecentre/redirection |
| Disallow | /Advicecentre/Redirection |
| Disallow | /article_feeder.asp |
| Disallow | /Article_feeder.asp |
| Disallow | /article/email |
| Disallow | /Article/Email |
| Disallow | /article/loadSimilarArticles |
| Disallow | /Article/LoadSimilarArticles |
| Disallow | /article/loadTranslate |
| Disallow | /Article/LoadTranslate |
| Disallow | /article/question |
| Disallow | /Article/Question |
| Disallow | /author/contact |
| Disallow | /Author/Contact |
| Disallow | /author/email |
| Disallow | /Author/Email |
| Disallow | /company/email |
| Disallow | /Company/Email |
| Disallow | /company/email |
| Disallow | /Company/Email |
| Disallow | /company/lexauthor/ |
| Disallow | /Company/Lexauthor/ |
| Disallow | /contact/ |
| Disallow | /Contact/ |
| Disallow | /content/ |
| Disallow | /Content/ |
| Disallow | /editorial/ |
| Disallow | /Editorial/ |
| Disallow | /home/redirect/* |
| Disallow | /Home/Redirect/* |
| Disallow | /home/translate |
| Disallow | /Home/Translate |
| Disallow | /home/updateSession |
| Disallow | /Home/UpdateSession |
| Disallow | /redirection.asp |
| Disallow | /Redirection.asp |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.mondaq.com/sitemapindex.xml |