tingit.nl
robots.txt

Robots Exclusion Standard data for tingit.nl

Resource Scan

Scan Details

Site Domain tingit.nl
Base Domain tingit.nl
Scan Status Ok
Last Scan2024-10-04T14:00:20+00:00
Next Scan 2024-10-18T14:00:20+00:00

Last Scan

Scanned2024-10-04T14:00:20+00:00
URL https://tingit.nl/robots.txt
Redirect https://www.tingit.nl/robots.txt
Redirect Domain www.tingit.nl
Redirect Base tingit.nl
Domain IPs 2001:7b8:65c::5, 212.114.96.5
Redirect IPs 2001:7b8:65c::5, 212.114.96.5
Response IP 212.114.96.5
Found Yes
Hash 5e901b89858ccf0a693b0149d5fcea200fb5759c5aa7cba891ce3c7c9fd4c84d
SimHash 6890d3e0c0bd

Groups

*

Rule Path
Disallow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

openai-gpt-3

Rule Path
Allow /

openai-gpt-4

Rule Path
Allow /

google-extended

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

adidxbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

pinterest

Rule Path
Allow /

slackbot

Rule Path
Allow /

instagram

Rule Path
Allow /

whatsapp

Rule Path
Allow /

telegrambot

Rule Path
Allow /

redditbot

Rule Path
Allow /

yahoo! slurp

Rule Path
Allow /

adsbot-amazon

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

ccbot

Rule Path
Allow /

slurp

Rule Path
Disallow /

yandex

Rule Path
Disallow /

applebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot-ba

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

omgili

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /