autodeal.com.ph
robots.txt

Robots Exclusion Standard data for autodeal.com.ph

Resource Scan

Scan Details

Site Domain autodeal.com.ph
Base Domain autodeal.com.ph
Scan Status Ok
Last Scan2024-06-01T10:49:34+00:00
Next Scan 2024-06-08T10:49:34+00:00

Last Scan

Scanned2024-06-01T10:49:34+00:00
URL https://autodeal.com.ph/robots.txt
Redirect https://www.autodeal.com.ph/robots.txt
Redirect Domain www.autodeal.com.ph
Redirect Base autodeal.com.ph
Domain IPs 13.213.133.125, 18.139.150.154
Redirect IPs 13.213.133.125, 18.139.150.154
Response IP 18.139.150.154
Found Yes
Hash f7fc14db6578e095831e836b98543038e3d0209aaa3c2f3aa809c0e7205a75a5
SimHash a4133b824f79

Groups

spider

Rule Path
Disallow /

bot-

Rule Path
Disallow /

bot/

Rule Path
Disallow /

linkchecker

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

java

Rule Path
Disallow /

nicebot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

powermarks

Rule Path
Disallow /

missigua_locator

Rule Path
Disallow /

web downloader

Rule Path
Disallow /

lanshanbot

Rule Path
Disallow /

custo

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

httrack off-line browser

Rule Path
Disallow /

nutchcvs

Rule Path
Disallow /

t-h-u-n-d-e-r-s-t-o-n-e

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

htmlparser

Rule Path
Disallow /

crawl

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

larbin

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

npbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /app_dev.php/
Disallow /subscribe
Disallow /bpi-auto-loans
Disallow /pgw
Disallow */test-drive-request%26
Disallow /bpi-loan-calculator
Disallow /lead-reengagement/
Disallow /geolocate/
Disallow /ducati-case-study
Disallow /dealers/cars
Disallow /used-cars/price-drop-alert
Disallow /ajax/show-cookie-use-notice
Disallow /promoter-pdf-report
Disallow /comparison-bid/
Disallow /car-loan-application/widget/view-car-preview
Disallow /fb-conversion/
Disallow /save-promo/
Disallow */submit-review
Disallow /is-promo-saved/
Disallow /comparison-tool/car-model-variants

Other Records

Field Value
sitemap https://www.autodeal.com.ph/sitemap/secure_sitemap_index.xml

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • Disallow: /comparison-bid/suggest-car
  • Disallow: /comparison-bid/record-click