Log In Sign Up

motorclutch.com
robots.txt

Robots Exclusion Standard data for motorclutch.com

Resource Scan

Scan Details

Site Domain motorclutch.com
Base Domain motorclutch.com
Scan Status Ok
Last Scan2026-02-11T02:54:52+00:00
Next Scan 2026-02-18T02:54:52+00:00

Last Scan

Scanned2026-02-11T02:54:52+00:00
URL https://motorclutch.com/robots.txt
Domain IPs 103.171.45.137
Response IP 103.171.45.137
Found Yes
Hash 43d7088c8c60840cd1766bab2b789b06a8c21369448fc1799193711cf1f1d195
SimHash 0e985a038163

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

googlebot

Rule Path
Allow /

google-extended

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yandex

Rule Path
Allow /

ccbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

facebookexternalhit/1.1

Rule Path
Allow /

facebot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

pinterestbot

Rule Path
Allow /

redditbot

Rule Path
Allow /

discordbot

Rule Path
Allow /

slackbot

Rule Path
Allow /

telegrambot

Rule Path
Allow /

skypeuripreview

Rule Path
Allow /

mastodon

Rule Path
Allow /

activitypub

Rule Path
Allow /

bluesky

Rule Path
Allow /

atproto-crawler

Rule Path
Allow /

misskey

Rule Path
Allow /

lemmybot

Rule Path
Allow /

pleroma

Rule Path
Allow /

flipboardproxy

Rule Path
Allow /

flipboard

Rule Path
Allow /

feedly

Rule Path
Allow /

inoreader

Rule Path
Allow /

pocketparser

Rule Path
Allow /

viber

Rule Path
Allow /

linebot

Rule Path
Allow /

kakaobot

Rule Path
Allow /

gptbot

Rule Path
Allow /

google-extended

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

ccbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://motorclutch.com/sitemap_index.xml
Back to top

Comments

  • This file is automatically added by Rank Math SEO plugin to help a website index better
  • More info: https://rankmath.com/?utm_source=Plugin&utm_medium=Robots&utm_campaign=WP
  • --------------- SEARCH ENGINES ---------------
  • --------------- SOCIAL MEDIA + PREVIEW BOTS ---------------
  • Facebook (Facebook, Instagram, Threads, Messenger)
  • WhatsApp
  • Twitter / X
  • LinkedIn
  • Pinterest
  • Reddit
  • Discord
  • Slack
  • Telegram
  • Skype
  • --------------- FEDIVERSE / DECENTRALIZED SOCIAL ---------------
  • Mastodon (ActivityPub preview bot)
  • Bluesky (ATProto crawler)
  • Misskey (Fediverse)
  • Lemmy
  • Pleroma
  • --------------- NEWS AGGREGATORS / FEED SERVICES ---------------
  • Flipboard
  • Feedly
  • Inoreader
  • Pocket (GetPocket)
  • --------------- MESSAGING / EMBED SERVICES ---------------
  • Viber
  • Line
  • KakaoTalk
  • --------------- AI CRAWLERS ---------------
Back to top