randozone.com
robots.txt

Robots Exclusion Standard data for randozone.com

Resource Scan

Scan Details

Site Domain randozone.com
Base Domain randozone.com
Scan Status Ok
Last Scan2024-06-28T02:40:28+00:00
Next Scan 2024-07-05T02:40:28+00:00

Last Scan

Scanned2024-06-28T02:40:28+00:00
URL https://randozone.com/robots.txt
Redirect https://www.randozone.com/robots.txt
Redirect Domain www.randozone.com
Redirect Base randozone.com
Domain IPs 194.165.49.111
Redirect IPs 194.165.49.111
Response IP 194.165.49.111
Found Yes
Hash 1ac05ce8576541f7bc50f2da602c2af257dca11236928c0979f636cc994e3fd0
SimHash 5a9cd151a01b

Groups

*

Rule Path
Disallow /forum/
Disallow /client/
Disallow /contrib/
Disallow /catalogue.php
Disallow /catalogue/
Disallow /search/
Disallow /*action%3Dcalendar
Disallow /*action%3Dprofile
Disallow /*action%3Dgallery
Disallow /*action%3Ddownloads
Disallow /redirect.php
Disallow /redirect-article.php
Disallow /*PHPSESSID%3D
Disallow /*redirect.php
Disallow /map/?cp=*
Disallow /map/?lat=*
Disallow /search-g.php?q=*
Disallow /search-fiche-perti.php
Disallow /search-iti.php
Disallow /*search.php
Disallow /pict_mont/up.php
Disallow /wall-download.php

Other Records

Field Value
crawl-delay 5

npbot
grapeshotcrawler
httrack
sitebot
sitesnagger
sitesucker
stackoverflow
riddler
ru_bot
sogou web spider
sosospider
spankbot
tineye
turnitinbot
yandex
baiduspider
blexbot
megaindex.ru/2.0
sputnikbot/2.3
seekport crawler
the knowledge ai
cliqzbot
mj12bot
chatgpt-user
gptbot
ccbot

Rule Path
Disallow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

criteobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

proximic

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.randozone.com/sitemap.xml
sitemap https://www.randozone.com/sitemap2.txt

Comments

  • User-agent: Screaming Frog SEO Spider