store.marseille.aeroport.fr
robots.txt

Robots Exclusion Standard data for store.marseille.aeroport.fr

Resource Scan

Scan Details

Site Domain store.marseille.aeroport.fr
Base Domain marseille.aeroport.fr
Scan Status Ok
Last Scan2024-11-11T09:35:14+00:00
Next Scan 2024-11-25T09:35:14+00:00

Last Scan

Scanned2024-11-11T09:35:14+00:00
URL https://store.marseille.aeroport.fr/robots.txt
Domain IPs 2600:9000:2024:1200:d:8e3e:380:93a1, 2600:9000:2024:3400:d:8e3e:380:93a1, 2600:9000:2024:3c00:d:8e3e:380:93a1, 2600:9000:2024:5000:d:8e3e:380:93a1, 2600:9000:2024:8c00:d:8e3e:380:93a1, 2600:9000:2024:9400:d:8e3e:380:93a1, 2600:9000:2024:9c00:d:8e3e:380:93a1, 2600:9000:2024:c400:d:8e3e:380:93a1, 65.9.112.118, 65.9.112.34, 65.9.112.87, 65.9.112.89
Response IP 18.165.122.76
Found Yes
Hash 77b9656fa5b78c8d5891985ffe02ad9e173b9c005f50702e5c1016143f5c65c5
SimHash 6a0d501781b2

Groups

*

Rule Path
Disallow /admin
Disallow /fr_FR/login
Disallow /fr_FR/cart
Disallow /fr_FR/account
Disallow /digital-chart
Disallow /*?id=*
Disallow /*%26id%3D*
Disallow *.sql
Disallow *.sql.gz

googlebot

Rule Path
Disallow /*?tx_indexedsearch

zyborg

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

boitho.com-dc

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

discobot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

kalooga

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

ocelli

Rule Path
Disallow /

penthesilea

Rule Path
Disallow /

phpdig

Rule Path
Disallow /

searchdaimon.com-dc

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

snapbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

woozweb-monitoring

Rule Path
Disallow /

woriobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /

Other Records

Field Value
sitemap https://store.marseille.aeroport.fr/sitemap.xml

Comments

  • Only allow URLs generated with RealURL
  • Google specific settings
  • Add sitemap