new2.jp
robots.txt

Robots Exclusion Standard data for new2.jp

Resource Scan

Scan Details

Site Domain new2.jp
Base Domain new2.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-05-20T10:28:37+00:00
Next Scan 2024-08-18T10:28:37+00:00

Last Successful Scan

Scanned2023-04-03T20:31:31+00:00
URL https://new2.jp/robots.txt
Domain IPs 153.121.1.1
Response IP 133.167.7.4
Found Yes
Hash ccba6dbccb3b417b6fb0f5fae3112fc90b425e08e465744e5b097e16005e16c8
SimHash 805cd1d2ce99

Groups

*

Rule Path
Disallow /redirect/

ahrefsbot

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

megalodon

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

slurp

Rule Path
Disallow /

teoma

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

voltron

Rule Path
Disallow /

yahoo pipes 1.0

Rule Path
Disallow /

Other Records

Field Value
sitemap https://cdn.new2.jp/sitemap/sitemap-index.xml.gz

Warnings

  • 2 invalid lines.