nuwber.com
robots.txt

Robots Exclusion Standard data for nuwber.com

Resource Scan

Scan Details

Site Domain nuwber.com
Base Domain nuwber.com
Scan Status Ok
Last Scan2024-09-26T19:19:11+00:00
Next Scan 2024-10-26T19:19:11+00:00

Last Scan

Scanned2024-09-26T19:19:11+00:00
URL https://nuwber.com/robots.txt
Domain IPs 104.18.8.28, 104.18.9.28
Response IP 104.18.9.28
Found Yes
Hash 1e0493cf49c1539c670024ad6f18401fa20d0cba0aa2131378b1506784814534
SimHash 681c78f54618

Groups

*

Rule Path
Allow /removal/link
Allow /house/581889bde41b0e7c5ff80491
Disallow /user/*
Disallow /search*
Disallow /data-removal*
Disallow /removal*
Disallow /ad/*
Disallow /api/*
Disallow /redirect/*
Disallow /report/*
Disallow /account/*
Disallow /site/similar*
Disallow /address/house-people/*
Disallow /phone/*
Disallow /house/*
Disallow /subscription*
Disallow /checkout*
Disallow /checkout-with-report-review*
Disallow /report-review*
Disallow /unsubscribe/*
Disallow /offer/*
Disallow /web-reports/*
Disallow /cr*
Disallow /login
Disallow /magic-link
Disallow /verify-email
Disallow /urbust-late-visities-of-pine-our-Gues-not-the-pe
Disallow /_Incapsula_Resource
Disallow /accessibility
Disallow /fact-file/*
Disallow /property-fact-sheet/*
Disallow /three-d-secure-completed
Disallow /gtw/*
Disallow /auth*
Disallow /payment-proceed*
Disallow /feedback/*
Disallow /background-search
Disallow /advanced-search
Disallow /secure-removal
Disallow /blog/?s=

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

exabot

Rule Path
Disallow /

twitterbot

Rule Path
Allow /fact-file/*
Allow /property-fact-sheet/*

facebookbot

Rule Path
Allow /fact-file/*
Allow /property-fact-sheet/*

pinterestbot

Rule Path
Allow /fact-file/*
Allow /property-fact-sheet/*

Other Records

Field Value
sitemap https://nuwber.com/blog/sitemap_index.xml

Warnings

  • `host` is not a known field.