transcom.uk
robots.txt

Robots Exclusion Standard data for transcom.uk

Resource Scan

Scan Details

Site Domain transcom.uk
Base Domain transcom.uk
Scan Status Ok
Last Scan2026-01-03T23:16:00+00:00
Next Scan 2026-02-02T23:16:00+00:00

Last Scan

Scanned2026-01-03T23:16:00+00:00
URL https://transcom.uk/robots.txt
Domain IPs 194.26.222.166
Response IP 194.26.222.166
Found Yes
Hash 065edf6b452756bcf85e898db986502c0aa305900538e0224e1bf33bea834519
SimHash 683a7e436685

Groups

*

Rule Path
Allow /
Allow /index.php
Allow /dashboard.php?register
Allow /dashboard.php?login
Allow /droplist/
Allow /auction/
Allow /blog/
Allow /sitemap.php
Disallow /dashboard.php?*
Disallow /*?action=*
Disallow /*?return=*
Disallow /admin/
Disallow /api/
Disallow /cgi-bin/
Disallow /tmp/
Disallow /private/
Disallow /includes/
Disallow /*?user=*
Disallow /*?session=*
Disallow /*?sid=*
Disallow /*?token=*
Disallow /*login*
Disallow /*logout*
Disallow /*account*
Disallow /auction/*?domain=*
Disallow /auction/*?search=*
Disallow /auction/*?filter=*
Disallow /auction/*?sort=*
Allow /assets/
Allow /css/
Allow /js/
Allow /images/
Allow /*.css
Allow /*.js
Allow /*.png
Allow /*.jpg
Allow /*.jpeg
Allow /*.gif
Allow /*.svg
Allow /*.webp

googlebot

Rule Path
Allow /
Disallow /dashboard.php?*
Disallow /admin/

bingbot

Rule Path
Allow /
Disallow /dashboard.php?*
Disallow /admin/

slurp

Rule Path
Allow /
Disallow /dashboard.php?*
Disallow /admin/

Other Records

Field Value
sitemap https://transcom.uk/sitemap.xml

Comments

  • robots.txt for transcom.uk
  • Transcom Group - UK Domain Dropcatch & Live Auction Platform
  • Allow all legitimate search engine bots
  • Allow public pages
  • Disallow private/authenticated areas
  • Disallow user-specific and session pages
  • Disallow specific auction parameters (keep clean URLs in search)
  • Allow static assets
  • Sitemap location
  • Specific bot rules
  • Block aggressive scrapers (optional - uncomment if experiencing issues)
  • User-agent: AhrefsBot
  • Crawl-delay: 10
  • User-agent: SemrushBot
  • Crawl-delay: 10
  • User-agent: MJ12bot
  • Disallow: /
  • User-agent: DotBot
  • Disallow: /
  • User-agent: BLEXBot
  • Disallow: /
  • Crawl-delay for general bots (optional)
  • Crawl-delay: 2