multigift.com
robots.txt
Robots Exclusion Standard data for multigift.com
Resource Scan
Scan Details
| Site Domain | multigift.com |
| Base Domain | multigift.com |
| Scan Status | Ok |
| Last Scan | 2024-10-07T15:58:35+00:00 |
| Next Scan | 2024-11-06T15:58:35+00:00 |
Last Scan
| Scanned | 2024-10-07T15:58:35+00:00 |
| URL | https://multigift.com/robots.txt |
| Domain IPs | 185.233.175.190, 2a0c:eb00:0:f7:185:233:175:190 |
| Response IP | 185.233.175.190 |
| Found | Yes |
| Hash | 75e789766373b651205e7c04566af161ba271937219ce8dd62afcbf648bef6de |
| SimHash | 1f1662fc65a5 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /app/ |
| Disallow | /store_closed.html |
| Disallow | /DevAdmin.php |
| Disallow | *?dispatch=products.search |
| Disallow | *?features_hash |
| Disallow | *?get_variation_list |
| Disallow | *?is_ajax |
| Disallow | *?selected_section |
| Disallow | *?sl |
| Disallow | *?sort_by |
| Disallow | *?sort_order |
| Disallow | /multigift.html |
| Disallow | /nl/multigift.html |
| Disallow | /en/multigift.html |
| Disallow | /fr/multigift.html |
| Disallow | /de/multigift.html |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
ahrefsbot
ahrefsbot/5.2
ahrefsbot/6.1
alphaseobot
amazonbot
baiduspider
blexbot
blexbot/1.0
cazoodlebot
ccbot
ccbot/2.0
coccocbot
coccocbot-web
coccocbot-image
dataforseobot
docomo
dotbot
dotbot/1.0
dotbot/1.1
dotbot/1.2
gigabot
indeedbot
infoobot
infoobot/0.1
mail.ru_bot/2.0
mj12bot
mj12bot/v1.3.3
mojeekbot
mojeekbot/0.6
mslbot
nimbostratus-bot
orbbot
orbbot/1.1
petalbot
pinterestbot
pinterestbot/1.0
pricebot
purebot
purebot/1.1
sbider
sbider/nutch-1
scrapy
scrapy/2.3.0
seekportbot
seekport crawler
semrushbot
semrushbot/3~bl
semrushbot/7~bl
seokicks
seokicks-robot
serpstatbot
serpstatbot/1.0
seznambot
seznambot/3.2
siteexplorer
smtbot
sosospider
sosospider+
streamline3bot
turnitinbot
turnitinbot/2
twengabot
twengabot/1
twengabot/1.1
twengabot-2.0
vagabondo
vagabondo/4.0
velenpublicwebcrawler
yahoo
yandex
yandexbot
yandexbot/3.0
zoominfo
zoominfobot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://multigift.com/sitemap.xml |