Log In Sign Up

repairpal.com
robots.txt

Robots Exclusion Standard data for repairpal.com

Resource Scan

Scan Details

Site Domain repairpal.com
Base Domain repairpal.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-20T00:52:41+00:00
Next Scan 2024-08-18T00:52:41+00:00

Last Successful Scan

Scanned2023-07-01T23:08:43+00:00
URL https://repairpal.com/robots.txt
Domain IPs 34.120.82.36
Response IP 34.120.82.36
Found Yes
Hash 176605f3819ed146e24e756e8217e7a27cb62217ce1dd8dde009fcff8a369d5a
SimHash b98d53dac4b0

Groups

*

Rule Path
Disallow /styleguide/*
Disallow /search/*
Disallow /search?*
Disallow /questions/answered*
Disallow /questions/unanswered*
Disallow /questions/*/answered*
Disallow /questions/*/unanswered*
Disallow /questions/*/*/answered*
Disallow /questions/*/*/unanswered*
Disallow /*/questions/answered*
Disallow /*/questions/unanswered*
Disallow /problems/newest*
Disallow /problems/repairpal%2Bverified*
Disallow /problems/*/newest*
Disallow /problems/*/repairpal%2Bverified*
Disallow /problems/*/*/newest*
Disallow /problems/*/*/repairpal%2Bverified*
Disallow /*/problems/newest*
Disallow /*/problems/repairpal%2Bverified*
Disallow /data-requests
Disallow /privacy-policy
Disallow /privacy_policy
Disallow /terms-of-service
Disallow /terms_of_service
Disallow /profiles/
Disallow /account/
Disallow /account_service_locations/create
Disallow /car_type_questions/new
Disallow /car_type_questions/create
Disallow /car_type_questions/*/*/new
Disallow /car_type_reviews/
Disallow /common_problems/new
Disallow /common_problems/navigate
Disallow /like/
Disallow /follow/
Disallow /feedback/
Disallow /nearby-shops-and-dealers/
Disallow /*-repair-nearby
Disallow /rate_shop/
Disallow /service_location_appointments/
Disallow /claim_shop/
Disallow /email_this/
Disallow /review_shop/rate/
Disallow /*/print_deal*
Disallow /car_type_question_answer/rate/
Disallow /car_type_question_answers/
Disallow /flag/new/
Disallow /shop_portal/
Disallow /tooltips/
Disallow /twilio/
Disallow /profiles/
Disallow /directory/record_view
Disallow /directory?*
Disallow /*.rss$
Disallow /estimator/assign_car*
Disallow /estimator/results*
Disallow /widget/6/$
Disallow /widget/8/$
Disallow /*/ajax_load_service_types_list
Disallow /*/ajax_service_location_review_button
Disallow /*/problems/*type%3Dnewest*
Disallow /*/problems/*type%3Dpopular*
Disallow /*/problems/*type%3Drepairpal*
Disallow /*/problems*category%3D*
Disallow /*car_type_question_search_form*
Disallow /carmax$
Disallow /usaa$
Disallow /twg$
Disallow /carchex$
Disallow /forevercar$
Disallow /rsf/*
Disallow /ib/*
Disallow /*/none/*
Disallow /diagnostic_trouble_codes/show/obd*
Disallow /r/*
Disallow /quotes
Disallow /2005353/*
Disallow /rails/active_storage/*
Disallow /analytics/*
Disallow /mixpanel/*
Disallow /*?*sort_by=
Disallow /*?*filter_by=
Disallow /*?*checking_cookies=
Disallow /*?*specialty_filter=
Disallow /*?*car_type_id=
Disallow /*?*utf8=
Disallow /*?*car_year=
Disallow /*?*car_slug=
Disallow /*?*car_model_id=
Disallow /*?*car_brand_id=
Disallow /*?*RepairPal%2BVerified=
Disallow /*?*Newest=
Disallow /*?*Most%2BReported=
Disallow /*?*utm_source=
Disallow /*?*utm_medium=
Disallow /*?*utm_campaign=
Disallow /*?*utm_content=
Disallow /*?*scope=
Disallow /*?*car_system=
Disallow /*?*pop_appt_request=
Disallow /*?*car_model=
Disallow /*?*type=
Disallow /*?*sort=
Disallow /*?*page=
Disallow /*?*id=
Disallow /*?*answered=
Disallow /*?*car_type%5Bcar_brand_id%5D=
Disallow /*?*car_type%5Bcar_model_id%5D=
Disallow /*?*car_type%5Byear%5D=
Disallow /*?*new_on_load=
Disallow /*?*tab=
Disallow /*?*TB_iframe=
Disallow /*?*category_id=
Disallow /*?*category=
Disallow /*?*page_link=
Disallow /*?*rwd=
Disallow /*?*popular=
Disallow /*?*all_questions=
Disallow /*?*rating=
Disallow /*?*symptom=
Disallow /*?*modal=
Disallow /*?*accordion=
Disallow /*?*sortBy=
Disallow /*?*filterBy=
Disallow /*?*sg=*
Allow /car_chooser/years_for_model.json?car_model_id=*

adsbot-google

Rule Path
Allow /*sg%3D*
Allow /*?*utm_source=
Allow /*?*utm_medium=
Allow /*?*utm_campaign=
Allow /*?*utm_content=

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

mauibot (crawler.feedback+dc@gmail.com)

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ahc/1.0

Rule Path
Disallow /

ahc/2.0

Rule Path
Disallow /

ahc/2.1

Rule Path
Disallow /

axios 0.21.1

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://repairpal.com/sitemap_car_recalls_index.xml.gz
sitemap https://repairpal.com/sitemap_car_reviews_index.xml.gz
sitemap https://repairpal.com/sitemap_cars_index.xml.gz
sitemap https://repairpal.com/sitemap_city_make_index.xml.gz
sitemap https://repairpal.com/sitemap_city_index.xml.gz
sitemap https://repairpal.com/sitemap_comparisons_index.xml.gz
sitemap https://repairpal.com/sitemap_content_index.xml.gz
sitemap https://repairpal.com/sitemap_directory_index.xml.gz
sitemap https://repairpal.com/sitemap_estimates_index.xml.gz
sitemap https://repairpal.com/sitemap_general_index.xml.gz
sitemap https://repairpal.com/sitemap_obd_index.xml.gz
sitemap https://repairpal.com/sitemap_problem_reports_index.xml.gz
sitemap https://repairpal.com/sitemap_problems_index.xml.gz
sitemap https://repairpal.com/sitemap_q_and_a_index.xml.gz
sitemap https://repairpal.com/sitemap_questions_index.xml.gz
sitemap https://repairpal.com/sitemap_recalls_index.xml.gz
sitemap https://repairpal.com/sitemap_reliability_index.xml.gz
sitemap https://repairpal.com/sitemap_service_locations_index.xml.gz
sitemap https://repairpal.com/sitemap_symptoms_index.xml.gz
Back to top

Comments

  • disallow get params
  • sem ads have sg param.
  • we don't want googlebot or others crawling those.
  • sem ads have sg param
  • we only want AdsBot to crawl these, and ...
  • we want AdsBot to only crawl these.
  • blocking bad bots
Back to top