testdeploy.oilprice.com
robots.txt
Robots Exclusion Standard data for testdeploy.oilprice.com
Resource Scan
Scan Details
| Site Domain | testdeploy.oilprice.com |
| Base Domain | oilprice.com |
| Scan Status | Ok |
| Last Scan | 2024-05-26T10:22:38+00:00 |
| Next Scan | 2024-06-02T10:22:38+00:00 |
Last Scan
| Scanned | 2024-05-26T10:22:38+00:00 |
| URL | https://testdeploy.oilprice.com/robots.txt |
| Domain IPs | 3.133.189.247, 3.143.176.78, 3.18.168.115 |
| Response IP | 3.143.176.78 |
| Found | Yes |
| Hash | 8463121be3714d8657b2a2c44ea35d0616010962eb801c3c772488a93e6f47a3 |
| SimHash | 0b33c55066d2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /search |
| Disallow | /goto/ |
| Disallow | /commodity-price-charts?1* |
| Disallow | /_private/ |
| Disallow | /_vti_bin/ |
| Disallow | /_vti_cnf/ |
| Disallow | /_vti_log/ |
| Disallow | /_vti_pvt/ |
| Disallow | /_vti_txt/ |
| Disallow | /newsletters/ |
| Disallow | /feed/3 |
| Disallow | /newsfeeds/redirect/ |
| Disallow | /salestool |
| Disallow | /salestool/ |
| Disallow | /articles/sponsored |
| Disallow | /articles/sponsored/ |
| Disallow | /auth/* |
| Disallow | /account/* |
| Disallow | /administrator/ |
| Disallow | /cache/ |
| Disallow | /components/ |
| Disallow | /component/ |
| Disallow | /includes/ |
| Disallow | /installation/ |
| Disallow | /language/ |
| Disallow | /libraries/ |
| Disallow | /media/ |
| Disallow | /modules/ |
| Disallow | /plugins/ |
| Disallow | /templates/ |
| Disallow | /tmp/ |
| Disallow | /xmlrpc/ |
| Disallow | /ajax/csrf |
| Disallow | /ajax/add_view/* |
| Disallow | /ajax/return_country |
| Disallow | /widgets/* |
| Disallow | /freewidgets/* |
| Disallow | /opc/* |
| Disallow | /ajax/get_banner/* |
| Disallow | /ajax/get_user_blends |
| Disallow | /ajax/add_pushcrew |
| Disallow | /ajax/* |
| Disallow | /ajax/get_chart_prices/ |
| Disallow | /commodity-widgets.php |
| Disallow | /iframes/barcharts |
| Disallow | /market-intelligence/signup_success/6 |
| Disallow | /newwidgets.php?3 |
| Disallow | /specialreports/8_Mega_Trends.html |
| Disallow | /widgets/energyproduction.html |
| Disallow | /*data.json$ |
| Disallow | /*data.json?t=* |
| Disallow | /*last.json$ |
| Disallow | /*last.json?t=* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://oilprice.com/sitemap_index.xml |
| sitemap | https://oilprice.com/sitemap.xml |
| sitemap | https://oilprice.com/googlenews.xml |
Comments