Log In Sign Up

yallamotor.com
robots.txt

Robots Exclusion Standard data for yallamotor.com

Resource Scan

Scan Details

Site Domain yallamotor.com
Base Domain yallamotor.com
Scan Status Ok
Last Scan2024-10-27T17:42:21+00:00
Next Scan 2024-11-03T17:42:21+00:00

Last Scan

Scanned2024-10-27T17:42:21+00:00
URL https://yallamotor.com/robots.txt
Redirect https://www.yallamotor.com/robots.txt
Redirect Domain www.yallamotor.com
Redirect Base yallamotor.com
Domain IPs 172.66.40.62, 172.66.43.194, 2606:4700:3108::ac42:283e, 2606:4700:3108::ac42:2bc2
Redirect IPs 172.66.40.62, 172.66.43.194, 2606:4700:3108::ac42:283e, 2606:4700:3108::ac42:2bc2
Response IP 172.66.43.194
Found Yes
Hash a6d44f8663697c075317eff27b0fc61cf8889533c2c723b180032e31b6683767
SimHash 860d09a1c6b8

Groups

grapeshot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /user/*
Disallow /sales/*
Disallow /conversations/*
Allow /new-cars/compare
Allow /new-cars/compare/nissan-x-trail-2024-34477-vs-toyota-rav4-2024-34492
Allow /new-cars/compare/land-rover-defender-2024-34924-vs-mercedes-benz-g-class-2024-35238
Allow /new-cars/compare/bmw-x5-2024-35295-vs-genesis-gv80-2024-34518
Allow /new-cars/compare/nissan-kicks-2024-35106-vs-kia-seltos-2024-38111
Allow /new-cars/compare/ford-bronco-raptor-2024-38711-vs-jeep-wrangler-unlimited-2024-38895
Allow /new-cars/compare/jetour-t2-2024-38839-vs-tank-300-2024-38694
Allow /new-cars/compare/nissan-patrol-2024-34966-vs-toyota-land-cruiser-2024-34071
Allow /new-cars/compare/toyota-supra-2024-34700-vs-nissan-z-2024-34096
Allow /new-cars/compare/dodge-challenger-2024-35859-vs-ford-mustang-2024-36053
Allow /new-cars/compare/suzuki-swift-2024-35523-vs-kia-picanto-2024-35525
Allow /new-cars/compare/mini-convertible-2024-36638-vs-abarth-595-2024-37960
Allow /new-cars/compare/volkswagen-golf-gti-2024-34280-vs-honda-civic-type-r-2024-38703
Allow /new-cars/compare/bmw-m5-sedan-2024-37006-vs-porsche-panamera-2024-35652
Allow /new-cars/compare/mazda-6-2024-35425-vs-nissan-altima-2024-35096
Allow /new-cars/compare/toyota-corolla-2024-34628-vs-honda-civic-2024-34158
Allow /new-cars/compare/porsche-macan-2024-35355-vs-audi-q5-2024-36015
Allow /new-cars/compare/mazda-cx-5-2024-35450-vs-toyota-corolla-cross-2024-38288
Disallow /new-cars/compare/*vs*
Disallow /new-cars/search?*
Disallow /*?page=
Disallow /*?sort=
Allow /used-cars/search
Allow /new-cars/search
Allow /used-cars/certified-pre-owned/search
Disallow /used-cars/19*
Disallow /used-cars/20*
Disallow /ar/used-cars/19*
Disallow /ar/used-cars/20*
Disallow /used-cars/yr_*
Disallow /ar/used-cars/yr_*
Disallow /used-cars/*/20*
Disallow /used-cars/*/19*
Disallow /ar/used-cars/*/20*
Disallow /ar/used-cars/*/19*
Disallow /used-cars/*/km_*
Disallow /ar/used-cars/*/km_*
Disallow /news/*-%5Ba-zA-Z%5D*$
Allow /news
Allow /news/uae
Allow /news/ksa
Allow /news/egypt
Allow /news/qatar
Allow /news/oman
Allow /news/kuwait
Allow /news/bahrain
Allow /news/*-%5B0-9%5D*$
Back to top

Other Records

Field Value
sitemap https://www.yallamotor.com/sitemap.xml
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • Compare Pages
  • Allow the main compare page
  • Explicitly allow specific compare pages
  • Block any other compare pages that match *vs*
  • Search Pages
  • Block paginated and sorted search result pages
  • Allow important search
  • Block all URLs that contain years at the end, such as /used-cars/1937 (1900 to 1999, 2000 to 2099)
  • Block all URLs that start with "yr_" in the used cars section (e.g., /used-cars/yr_2015_2018, /ar/used-cars/yr_earlier_2024)
  • Block any used cars URLs that include a year (e.g., /used-cars/audi/2023, /ar/used-cars/toyota/caprice/2015)
  • Block any URLs that contain kilometer range information (e.g., /km_50000_more)
  • News Pages
  • For user searched blogs Explicitly disallowed the specific slug
  • Explicitly allowed the specific slug
  • Explicitly allowed all articles by content team
Back to top