bokt.nl
robots.txt

Robots Exclusion Standard data for bokt.nl

Resource Scan

Scan Details

Site Domain bokt.nl
Base Domain bokt.nl
Scan Status Ok
Last Scan2024-06-22T01:53:58+00:00
Next Scan 2024-06-29T01:53:58+00:00

Last Scan

Scanned2024-06-22T01:53:58+00:00
URL https://bokt.nl/robots.txt
Redirect https://www.bokt.nl/robots.txt
Redirect Domain www.bokt.nl
Redirect Base bokt.nl
Domain IPs 5.75.240.112
Redirect IPs 5.75.240.112
Response IP 5.75.240.112
Found Yes
Hash 7d5b1f04367717c68651e9b6d761ab76b1a2e76230d592e0759f427dc91676a9
SimHash 1a1c1e004ed1

Groups

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /forums/posting.php
Disallow /forums/report.php
Disallow /forums/login.php
Disallow /forums/mcp.php
Disallow /forums/ucp.php
Disallow /forums/postfreq.php
Disallow /forums/viewonline.php
Disallow /forums/calendar.php
Disallow /forums/verkleinen.php
Disallow /forums/mods/egosearch.php
Disallow /upload/
Disallow /tools/postfreq.php
Disallow /w/
Allow /w/skins/
Disallow /wiki/Speciaal%3ARandom
Disallow /wiki/Speciaal%3ARandom
Disallow /wiki/Speciaal%3AWillekeurig
Disallow /wiki/Speciaal%3AWillekeurig
Disallow /wiki/Speciaal%3ASearch
Disallow /wiki/Speciaal%3ASearch
Disallow /wiki/Speciaal%3ARecentchangeslinked
Disallow /wiki/Speciaal%3ARecentchangeslinked/
Disallow /wiki/Speciaal%3AVerwijzingenNaarHier/
Disallow /wiki/Speciaal%3AVerwijzingenNaarHier/
Disallow /wiki/Speciaal%3AWhatlinkshere/
Disallow /wiki/Speciaal%3AWhatlinkshere/
Disallow /wiki/Overleg%3A
Disallow /wiki/Overleg%3A
Disallow /wiki/Gebruiker%3A
Disallow /wiki/Gebruiker%3A
Disallow /markt/search

*

Rule Path
Disallow /forums/memberlist.php
Disallow /forums/posting.php
Disallow /forums/report.php
Disallow /forums/login.php
Disallow /forums/mcp.php
Disallow /forums/ucp.php
Disallow /forums/search.php
Disallow /forums/postfreq.php
Disallow /forums/viewonline.php
Disallow /forums/calendar.php
Disallow /forums/verkleinen.php
Disallow /forums/mods/egosearch.php
Disallow /apc.php
Disallow /tools/postfreq.php
Disallow /w/
Allow /w/skins/
Disallow /flarum/
Disallow /wiki/Speciaal%3ARandom
Disallow /wiki/Speciaal%3ARandom
Disallow /wiki/Speciaal%3AWillekeurig
Disallow /wiki/Speciaal%3AWillekeurig
Disallow /wiki/Speciaal%3ASearch
Disallow /wiki/Speciaal%3ASearch
Disallow /wiki/Speciaal%3ARecentchangeslinked
Disallow /wiki/Speciaal%3ARecentchangeslinked/
Disallow /wiki/Speciaal%3AVerwijzingenNaarHier/
Disallow /wiki/Speciaal%3AVerwijzingenNaarHier/
Disallow /wiki/Speciaal%3AWhatlinkshere/
Disallow /wiki/Speciaal%3AWhatlinkshere/
Disallow /wiki/Speciaal%3AContributors/
Disallow /wiki/Speciaal%3AContributors/
Disallow /wiki/Overleg%3A
Disallow /wiki/Overleg%3A
Disallow /wiki/Gebruiker%3A
Disallow /wiki/Gebruiker%3A
Disallow /report/
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /themes/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips/
Disallow /node/add/
Disallow /search/
Disallow /user/register/
Disallow /user/password/
Disallow /user/login/
Disallow /user/logout/
Disallow /markt/search
Disallow /markt/their-ads
Disallow /taxonomy/
Disallow /?q=admin%2F
Disallow /?q=comment%2Freply%2F
Disallow /?q=filter%2Ftips%2F
Disallow /?q=node%2Fadd%2F
Disallow /?q=search%2F
Disallow /?q=user%2Fpassword%2F
Disallow /?q=user%2Fregister%2F
Disallow /?q=user%2Flogin%2F
Disallow /?q=user%2Flogout%2F

Other Records

Field Value
sitemap https://www.bokt.nl/sitemap.xml
sitemap https://www.bokt.nl/sitemap_forum.xml.gz

Comments

  • robots.txt for https://www.bokt.nl/
  • Allow /forums/memberlist.php
  • Symfony
  • Drupal
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)