classical-scene.com
robots.txt

Robots Exclusion Standard data for classical-scene.com

Resource Scan

Scan Details

Site Domain classical-scene.com
Base Domain classical-scene.com
Scan Status Ok
Last Scan2024-09-21T04:23:02+00:00
Next Scan 2024-10-21T04:23:02+00:00

Last Scan

Scanned2024-09-21T04:23:02+00:00
URL https://classical-scene.com/robots.txt
Redirect https://www.classical-scene.com/robots.txt
Redirect Domain www.classical-scene.com
Redirect Base classical-scene.com
Domain IPs 35.215.109.27
Redirect IPs 35.215.109.27
Response IP 35.215.109.27
Found Yes
Hash f2b59cc779269a110f935c2fa8bde0fdb8dc1f51cb0151d829a3d66f8a1f6d93
SimHash 4314cd30d2f3

Groups

*

Rule Path
Disallow /wp-register.php
Disallow /wp-login.php
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 10

facebookexternalhit

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

mediapartners-google*
mediapartners-google*

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookexternal

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap http://classical-scene.com/sitemap.xml

Warnings

  • 2 invalid lines.