numlooker.com
robots.txt

Robots Exclusion Standard data for numlooker.com

Resource Scan

Scan Details

Site Domain numlooker.com
Base Domain numlooker.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-10T19:20:38+00:00
Next Scan 2025-08-17T19:20:38+00:00

Last Successful Scan

Scanned2025-08-02T19:19:58+00:00
URL https://numlooker.com/robots.txt
Domain IPs 104.21.72.76, 172.67.176.101, 2606:4700:3034::ac43:b065, 2606:4700:3037::6815:484c
Response IP 104.21.72.76
Found Yes
Hash 7ded6304fe0a7d2dc2d52cfabbdbf8eec14dcfd9fd4ee7dd3246594bfb8870f6
SimHash 8abe4160aed8

Groups

*

Rule Path
Disallow /search*
Disallow /results*
Disallow /report-review*
Disallow /register*
Disallow /data/image/

dataforseobot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

garlikcrawler/1.1 (http://garlik.com/, crawler@garlik.com)

Rule Path
Disallow /

linguee

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/