photographers.ua
robots.txt

Robots Exclusion Standard data for photographers.ua

Resource Scan

Scan Details

Site Domain photographers.ua
Base Domain photographers.ua
Scan Status Ok
Last Scan2024-10-04T07:46:59+00:00
Next Scan 2024-10-11T07:46:59+00:00

Last Scan

Scanned2024-10-04T07:46:59+00:00
URL https://photographers.ua/robots.txt
Domain IPs 178.162.209.249
Response IP 178.162.209.249
Found Yes
Hash f0ac0952a18d3125aa3626df0efd8dae8444dc75c5dc31e79e0b46cbf0fd2404
SimHash 2352d7414ef8

Groups

*

Rule Path
Disallow /*lang%3D
Disallow /*post%3D
Disallow /*strSortBy%3D
Disallow /*?profile
Disallow /*strTag%3D
Disallow /rss/user/*
Disallow /rss/lenta/*
Disallow /user/login/
Disallow /signup/
Disallow /stuff/
Disallow */sort/*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

mlbot

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

webalta

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

proximic

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sitecheckerbotcrawler

Rule Path
Disallow /

sitecheckerbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

dataforseobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

Warnings

  • `host` is not a known field.