Log In Sign Up

findcardetailing.co.uk
robots.txt

Robots Exclusion Standard data for findcardetailing.co.uk

Resource Scan

Scan Details

Site Domain findcardetailing.co.uk
Base Domain findcardetailing.co.uk
Scan Status Ok
Last Scan2025-05-29T21:00:58+00:00
Next Scan 2025-06-05T21:00:58+00:00

Last Scan

Scanned2025-05-29T21:00:58+00:00
URL https://findcardetailing.co.uk/robots.txt
Domain IPs 2a02:4780:16:83a8:8f61:6852:e165:3e82, 77.37.66.119
Response IP 77.37.115.159
Found Yes
Hash d56edf9a6f2498ac522434e9a20d5e37753f15320619553f05feaa40c2b120d8
SimHash 20b4d380851a

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-json/
Disallow /?rest_route=
Disallow /search/
Disallow /?s=

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Disallow /wp-json/
Disallow /?rest_route=
Disallow /search/
Disallow /?s=

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /wp-content/uploads/

applebot

Rule Path
Allow /

yandex

Rule Path
Allow /

yandeximages

Rule Path
Allow /wp-content/uploads/

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

qwantify

Rule Path
Allow /

facebook

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

facebookscraper

Rule Path
Allow /

facebot

Rule Path
Allow /

instagrambot

Rule Path
Allow /

whatsapp bot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

pinterest bot

Rule Path
Allow /

discordbot

Rule Path
Allow /

*

Rule Path
Allow /*.webp$

*

Rule Path
Allow /*.jpg$

*

Rule Path
Allow /*.png$

*

Rule Path
Allow /*.html$

*

Rule Path
Allow /*.php$
Back to top

Other Records

Field Value
sitemap http://findcardetailing.co.uk/post-sitemap.xml
sitemap http://findcardetailing.co.uk/page-sitemap.xml
Back to top

Comments

  • Wordpress Default
  • Prevent Crawling of WordPress JSON API Endpoints
  • Block Search URLs /search/ and /?s=
  • Block Ahrefs Crawler
  • Block Semrush Crawler
  • Block Moz Crawler
  • Block Majestic Crawler
  • Block archive.org bots
  • Block Chatgpt
  • Rankmath Sitemap Link
  • Prevent Crawling of WordPress JSON API Endpoints
  • Block Search URLs /search/ and /?s=
  • Allow Google Bot
  • Allow Google Images Bot
  • Allow Google Media Partners Bot
  • Allow Google AdsBot Bot
  • Allow Google Mobile Bot
  • Allow Bing Bot
  • Allow MSN Bot
  • Allow MSNBot Media Bot
  • Allow Apple Bot
  • Allow Yandex Bot
  • Allow Yandex Images Bot
  • Allow Yahoo Search (Slurp bot)
  • Allow DuckDuckGo Bot
  • Allow Qwant Bot
  • Allow Facebook Bot
  • Allow Instagram Bot
  • Allow Whatsapp Bot
  • Allow Twitter Bot
  • Allow Linkedin Bot
  • Allow Pinterest Bot
  • Allow Discord Bot
  • Allow Webp Images
  • Allow Jpg Images
  • Allow Png Images
  • Allow Html Files
  • Allow Php Files
Back to top