kontenpedia.com
robots.txt

Robots Exclusion Standard data for kontenpedia.com

Resource Scan

Scan Details

Site Domain kontenpedia.com
Base Domain kontenpedia.com
Scan Status Ok
Last Scan6/5/2025, 9:37:29 AM
Next Scan 7/5/2025, 9:37:29 AM

Last Scan

Scanned6/5/2025, 9:37:29 AM
URL https://kontenpedia.com/robots.txt
Redirect https://www.kontenpedia.com/robots.txt
Redirect Domain www.kontenpedia.com
Redirect Base kontenpedia.com
Domain IPs 104.21.30.190, 172.67.173.138, 2606:4700:3035::ac43:ad8a, 2606:4700:3037::6815:1ebe
Redirect IPs 104.21.30.190, 172.67.173.138, 2606:4700:3035::ac43:ad8a, 2606:4700:3037::6815:1ebe
Response IP 104.21.30.190
Found Yes
Hash d81a21d4dd91f53e578e6a505a5120d64e2e123b55622b4c58824b87e80d12d2
SimHash 4844dc31d7f7

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot-mobile

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot-image

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

msnbot-media

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

twitterbot

Rule Path
Allow /

facebot

Rule Path
Allow /

alexabot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.kontenpedia.com/petasite.xml