trendstop.levif.be
robots.txt

Robots Exclusion Standard data for trendstop.levif.be

Resource Scan

Scan Details

Site Domain trendstop.levif.be
Base Domain levif.be
Scan Status Ok
Last Scan2024-10-23T12:20:37+00:00
Next Scan 2024-11-22T12:20:37+00:00

Last Scan

Scanned2024-10-23T12:20:37+00:00
URL https://trendstop.levif.be/robots.txt
Domain IPs 37.148.180.246
Response IP 37.148.180.246
Found Yes
Hash 39c0ca427de0c7f97a618a12f08905a1987bf2038845d700eebdbc46b8ffc0d1
SimHash 610cc84a8b13

Groups

*

Rule Path
Allow /sitemap-xml.ashx
Allow /sitemap-xml-default.ashx
Allow /sitemap-xml-companies.ashx
Disallow /*.axd
Disallow /*.ashx
Disallow /*/login-info.aspx
Disallow /*/signin.aspx
Disallow /*/tools/benchmark
Disallow /__detailbedrijf.aspx
Disallow /company.aspx
Disallow /genericform.aspx
Disallow /results.aspx
Disallow /server.aspx
Disallow /showarticle.aspx
Disallow /tracker.ashx
Disallow /trap.ashx
Disallow /ontop/
Disallow /services/

ahrefsbot
archive.org_bot
baiduspider
dotbot
gptbot
ia_archiver
orangebot
sapphirewebcrawler
trendictionbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://trendstop.knack.be/sitemap-xml.ashx
sitemap https://trendstop.levif.be/sitemap-xml.ashx

Comments

  • https://developers.google.com/search/docs/crawling-indexing/robots/intro
  • sitemap domain name is replaced dynamically by handler