napiarfolyam.hu
robots.txt

Robots Exclusion Standard data for napiarfolyam.hu

Resource Scan

Scan Details

Site Domain napiarfolyam.hu
Base Domain napiarfolyam.hu
Scan Status Ok
Last Scan2024-09-23T09:57:30+00:00
Next Scan 2024-09-30T09:57:30+00:00

Last Scan

Scanned2024-09-23T09:57:30+00:00
URL https://www.napiarfolyam.hu/robots.txt
Domain IPs 193.201.190.25
Response IP 193.201.190.25
Found Yes
Hash e1ce117ef430ab738ea602ec230d2cf819823e762788edafdf9631cce0925ad6
SimHash 6940c39357b5

Groups

*

Product Comment
* Disable all, but
Rule Path
Disallow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

google favicon

Rule Path
Allow /

google web preview

Rule Path
Allow /

google-adwords-instant

Rule Path
Allow /

google-certificates-bridge

Rule Path
Allow /

google-inspectortool

Rule Path
Allow /

google-physicalweb

Rule Path
Allow /

google-read-aloud

Rule Path
Allow /

google-safety

Rule Path
Allow /

google-site-verification

Rule Path
Allow /

google-structured-data-testing-tool

Rule Path
Allow /

google-xrawler

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googleother

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

msnbot

Rule Path
Allow /

slurp

Rule Path
Allow /

teoma

Rule Path
Allow /

gigabot

Rule Path
Allow /

nutch

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yahoo-mmcrawler

Rule Path
Allow /

yahoo-blogs/v3.9

Rule Path
Allow /