Log In Sign Up

meridol.co.za
robots.txt

Robots Exclusion Standard data for meridol.co.za

Resource Scan

Scan Details

Site Domain meridol.co.za
Base Domain meridol.co.za
Scan Status Ok
Last Scan2024-10-28T23:05:10+00:00
Next Scan 2024-11-11T23:05:10+00:00

Last Scan

Scanned2024-10-28T23:05:10+00:00
URL https://www.meridol.co.za/robots.txt
Domain IPs 2600:1413:b000:1b::17d7:70d, 2600:1413:b000:1b::17d7:712, 96.17.180.162, 96.17.180.185
Response IP 23.52.171.201
Found Yes
Hash a188d74a8d2ad13647212cb0c45bd2fa9e656c3e9252cf8ef15b10ea81647db4
SimHash 6535a650e7b4

Groups

swiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow */search
Disallow */snippet/
Disallow */content/
Disallow */cp-sites/
Allow /cp-sites/oral-care/meridol-redesign
Back to top

Other Records

Field Value
sitemap https://www.meridol.co.za/sitemap.xml
Back to top

Comments

  • internal search
  • Images
  • sitemaps
Back to top