tvguide.vg.no
robots.txt

Robots Exclusion Standard data for tvguide.vg.no

Resource Scan

Scan Details

Site Domain tvguide.vg.no
Base Domain vg.no
Scan Status Ok
Last Scan2024-05-25T21:27:43+00:00
Next Scan 2024-06-24T21:27:43+00:00

Last Scan

Scanned2024-05-25T21:27:43+00:00
URL https://tvguide.vg.no/robots.txt
Domain IPs 13.226.2.118, 13.226.2.33, 13.226.2.68, 13.226.2.72, 2600:9000:21f8:1200:6:a737:7b00:93a1, 2600:9000:21f8:1e00:6:a737:7b00:93a1, 2600:9000:21f8:6400:6:a737:7b00:93a1, 2600:9000:21f8:7800:6:a737:7b00:93a1, 2600:9000:21f8:7a00:6:a737:7b00:93a1, 2600:9000:21f8:ac00:6:a737:7b00:93a1, 2600:9000:21f8:da00:6:a737:7b00:93a1, 2600:9000:21f8:f000:6:a737:7b00:93a1
Response IP 108.157.60.48
Found Yes
Hash 84430e42ae7ae233f69034237d9fe961bb0e487813f58251d0f6ccb53c1941b8
SimHash 7022f0d2e112

Groups

*

Rule Path
Allow /
Disallow *?*
Disallow /nokkelord/
Disallow /s/
Disallow /program/p_*
Disallow /program/*/sesong/
Disallow /program/*/1000
Disallow /kanal/*/2019-*
Disallow /kanal/*/2020-*
Disallow /kanal/*/2021-*
Disallow /kanal/*/2022-*
Disallow /kanal/*/2023-*
Disallow /kanal/*/1000
Disallow /kanal/*/mandags
Disallow /kanal/*/tirsdags
Disallow /kanal/*/onsdags
Disallow /kanal/*/torsdags
Disallow /kanal/*/fredags
Disallow /kanal/*/lordags
Disallow /kanal/*/sondags
Disallow /kanal-bibliotek/

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

Comments

  • start AI crawler block
  • end AI crawler block