catalogue.pesi.com.au
robots.txt

Robots Exclusion Standard data for catalogue.pesi.com.au

Resource Scan

Scan Details

Site Domain catalogue.pesi.com.au
Base Domain pesi.com.au
Scan Status Ok
Last Scan2024-10-25T02:13:42+00:00
Next Scan 2024-11-24T02:13:42+00:00

Last Scan

Scanned2024-10-25T02:13:42+00:00
URL https://catalogue.pesi.com.au/robots.txt
Domain IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash 0a6db2e1e83521385e5b12886a3cf0c03ace3b94d96f46e1076fcd3ca51f71e2
SimHash 733ff2354aa0

Groups

*

Rule Path
Allow /
Disallow /*?classroomId=
Disallow /*?tokenType=
Disallow /*?ProductID=
Disallow /*?SpeakerId=
Disallow /*?internalad=
Disallow /*?discountId=
Disallow /*?isFromSalePage=
Disallow /*?search=
Disallow /*?q=
Disallow /*?returnurl=
Disallow /*?x=
Disallow /*%26classroomId%3D
Disallow /*%26tokenType%3D
Disallow /*%26ProductID%3D
Disallow /*%26SpeakerId%3D
Disallow /*%26internalad%3D
Disallow /*%26discountId%3D
Disallow /*%26isFromSalePage%3D
Disallow /*%26search%3D
Disallow /*%26q%3D
Disallow /*%26returnurl%3D
Disallow /*%26x%3D
Disallow /*_affilate-
Disallow /*_affilate_
Disallow /*_automatedemail-
Disallow /*_dedicatedemail-
Disallow /*_dedicatedemail_
Disallow /*_email-
Disallow /*_emailsq-
Disallow /*_featuredad-
Disallow /*_mailer-
Disallow /*_mailer_
Disallow /*_redirect-
Disallow /*_redirect_
Disallow /*_redirectlink-
Disallow /*_redirectpos-
Disallow /*_socialmedia-
Disallow /*_socialmedia_
Disallow /classroom/addtocart
Disallow /classroom/classroomproductsdescription
Disallow /Classroom/ClassroomProductsDescription
Disallow /discount
Disallow /home/updatebannershowncount
Disallow /items/getcertcreditdetail
Disallow /items/getproductsinfo
Disallow /items/getproductspeaker
Disallow /Items/GetProductSpeaker
Disallow /items/productreviews
Disallow /productgroupcampaigns
Disallow /speaker/speakerdetail
Disallow /21600225
Disallow /*?__hstc=
Disallow /*?__hssc=
Disallow /*?__hsfp=
Disallow /*?_hsenc=
Disallow /*?_hsmi=
Disallow /*%26__hstc%3D
Disallow /*%26__hssc%3D
Disallow /*%26__hsfp%3D
Disallow /*%26_hsenc%3D
Disallow /*%26_hsmi%3D
Disallow /*?response_type=
Disallow /*?scope=
Disallow /*?response_mode=
Disallow /*?nonce=
Disallow /*%26response_type%3D
Disallow /*%26response_mode%3D
Disallow /*%26scope%3D
Disallow /*%26nonce%3D

adbeat

Rule Path
Disallow /

admantx

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bomborabot

Rule Path
Disallow /

bvbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

clickagy

Rule Path
Disallow /

cocolyzebot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

domcopbot

Rule Path
Disallow /

embed.ly

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

gulperbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

internet-structure-research-project-bot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

nimbostratus

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

onalyticabot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

pubmatic crawler bot

Rule Path
Disallow /

riddlerbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sbl-bot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

seekport

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

semantic-visions

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sqlmap

Rule Path
Disallow /

traackr

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

voluumdsp

Rule Path
Disallow /

wc-test-dev-bot

Rule Path
Disallow /

whatcms

Rule Path
Disallow /

webtechbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://catalogue.pesi.com.au/sitemap.xml

Comments

  • I hope you brought the tacos
  • Disallow P23 Parameters
  • Disallow P23 Paths
  • Disallow Google Ad Manager Paths
  • Disallow HubSpot Parameters
  • Disallow SSO Parameters
  • Disallow Agents