Log In Sign Up

antalis.co.uk
robots.txt

Robots Exclusion Standard data for antalis.co.uk

Resource Scan

Scan Details

Site Domain antalis.co.uk
Base Domain antalis.co.uk
Scan Status Ok
Last Scan2024-06-10T02:16:13+00:00
Next Scan 2024-07-10T02:16:13+00:00

Last Scan

Scanned2024-06-10T02:16:13+00:00
URL https://www.antalis.co.uk/robots.txt
Domain IPs 104.69.175.238, 2600:1413:1:596::3ee0, 2600:1413:1:5a1::3ee0
Response IP 104.69.175.238
Found Yes
Hash 14f9129de0be3a199506be41d5733672b076b7a868f5d908c543415d2d2cddc0
SimHash 6f2c1ab18388

Groups

*

Rule Path
Disallow /sso/
Disallow /eshop/*/ws/html/secure/
Disallow /eshop/ws/html/secure/
Disallow /*productListing?sectorId=*
Disallow /*productDetails?id=*
Disallow /*productListing?Id=*
Disallow /*recordState%3D*
Disallow /*resultPage*
Disallow /*.pdf
Disallow /*%26N%3D
Disallow /*?N=
Disallow /*%26showMoreIds%3D
Disallow /*?showMoreIds=
Disallow /*%26keyWord%3D
Disallow /*?keyWord=
Disallow /*?ns=
Disallow /*%26ns%3D
Disallow /*?p=
Disallow /*%26p%3D

Other Records

Field Value
crawl-delay 10

etaospider

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

arquivo

Rule Path
Disallow /

atomseobot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

crawler_eb_germany

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

deepnoc

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

dnbcrawler-analytics

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

duckduckgo

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

infotigerbot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

matchorybot

Rule Path
Disallow /

marketingtracer

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

olbicobot

Rule Path
Disallow /

opteobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

roboter

Rule Path
Disallow /

seekport

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seostar

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sidetrade

Rule Path
Disallow /

startmebot

Rule Path
Disallow /

statuscake_pagespeed_indev

Rule Path
Disallow /

stormcrawler

Rule Path
Disallow /

tigerbot

Rule Path
Disallow /

tinytestbot

Rule Path
Disallow /

tmmbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

wiederfreibot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexmetrika

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.antalis.co.uk/home.sitemap.xml
sitemap https://www.antalis.co.uk/eshop/ws/sitemap.xml
Back to top

Comments

  • Delay by default
  • Exclude internal
  • Exclude internal content
  • Exclude URL parameters
  • unwanted bot
  • sitemaps
Back to top