nordictattoosupplies.com
robots.txt

Robots Exclusion Standard data for nordictattoosupplies.com

Resource Scan

Scan Details

Site Domain nordictattoosupplies.com
Base Domain nordictattoosupplies.com
Scan Status Ok
Last Scan2024-09-25T19:19:20+00:00
Next Scan 2024-10-25T19:19:20+00:00

Last Scan

Scanned2024-09-25T19:19:20+00:00
URL https://nordictattoosupplies.com/robots.txt
Domain IPs 94.237.39.25
Response IP 94.237.39.25
Found Yes
Hash 5e2abdd0ffcf9fa2d7baafb669fd743768c0f9f73483fddc518ae4b4f8b73974
SimHash 0e3c928aeebc

Groups

*

Rule Path
Disallow /epages/Site.admin/
Disallow /epages/*ViewAction%3DFacetedSearchProducts
Disallow /epages/*ViewAction%3DMobileSF-DetailFacetedSearchProducts
Disallow /epages/*ViewAction%3DViewFaceted
Disallow /epages/*ViewAction%3DDetailSearchProducts
Disallow /epages/*ViewAction%3DMobileSF-DetailSearchProducts
Disallow /epages/*ViewAction%3DViewDetailSearchProducts
Disallow /epages/*ViewAction%3DMobileSF-ViewDetailSearchProducts
Disallow /epages/*ViewAction%3DViewContentSearchMask
Disallow /epages/*ViewAction%3DContentSearch
Disallow /epages/*ViewAction%3DUnityViewAddToWishlistScript
Disallow /epages/*ViewAction%3DUnityViewFacebookBusinessPixelCode
Disallow /epages/*ViewAction%3DUnityViewRatenkaufBannerScript
Disallow /epages/*ViewAction%3DUnityViewRegisterSessionScript
Disallow /epages/*ViewAction%3DUnitySF-ViewGoogleAdWordsScript
Disallow /epages/*ViewAction%3DUnityViewGoogleSmartShoppingScript
Disallow /epages/*ViewAction%3DUnityViewMetaCatalogScript
Disallow /epages/*ViewAction%3DUnityViewPayPalPPCPScript
Disallow /epages/*ViewAction%3DViewPayPalPPCPScript

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

amazonbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

serpstatbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Comments

  • robots.txt for epages
  • applies to all robots
  • disallow TBO, PBO
  • prevent mass indexing of search pages