headliner.nl
robots.txt
Robots Exclusion Standard data for headliner.nl
Resource Scan
Scan Details
| Site Domain | headliner.nl |
| Base Domain | headliner.nl |
| Scan Status | Ok |
| Last Scan | 2024-11-14T07:06:08+00:00 |
| Next Scan | 2024-11-21T07:06:08+00:00 |
Last Scan
| Scanned | 2024-11-14T07:06:08+00:00 |
| URL | https://headliner.nl/robots.txt |
| Redirect | https://www.headliner.nl/robots.txt |
| Redirect Domain | www.headliner.nl |
| Redirect Base | headliner.nl |
| Domain IPs | 77.68.24.79 |
| Redirect IPs | 77.68.24.79 |
| Response IP | 77.68.24.79 |
| Found | Yes |
| Hash | 889040e1fad57d71d48fdf030c1e1a4690141d661ff41b5a6f9bf97104bde71f |
| SimHash | 731cf7d18851 |
Groups
semrushbot
semrushbot-sa
ahrefsbot
mj12bot
petalbot
owlin
trendictionbot
dotbot
scooperbot
spinn3r
twiceler
sistrix
wotbox
exabot
wbsearchbot
blexbot
yandex
seekportbot
ccbot
imagesiftbot
awariorssbot
awariosmartbot
amazonadbot
sirdatabot
mojeekbot
lamarkbot
geedobot
surdotlybot
dataforseobot
phxbot
zoominfobot
awariobot
hoaxybot
livelapbot
turnitinbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /sections.php |
| Disallow | /headliner.php |
| Disallow | /headliner2.php |
| Disallow | /follow.php |
| Disallow | /follow2.php |
| Disallow | /top.php |
| Disallow | /top2.php |
| Disallow | /file.php |
| Disallow | /redirect.php |
| Disallow | /redirect2.php |
Warnings
- 1 invalid line.