Log In Sign Up

jinritoutiao.com
robots.txt

Robots Exclusion Standard data for jinritoutiao.com

Resource Scan

Scan Details

Site Domain jinritoutiao.com
Base Domain jinritoutiao.com
Scan Status Ok
Last Scan2024-07-04T09:26:45+00:00
Next Scan 2024-08-03T09:26:45+00:00

Last Scan

Scanned2024-07-04T09:26:45+00:00
URL http://jinritoutiao.com/robots.txt
Redirect https://www.toutiao.com/robots.txt
Redirect Domain www.toutiao.com
Redirect Base toutiao.com
Domain IPs 122.14.229.15, 122.14.229.17
Redirect IPs 23.209.46.11, 23.209.46.15
Response IP 42.99.140.218
Found Yes
Hash cac19574c23a7ee81b073b16adcd8b3397edb26b4498c4e4fcea8763e3a66be5
SimHash e145d9707f82

Groups

*

Rule Path
Disallow /search

bytespider

Rule Path
Disallow

toutiaospider

Rule Path
Disallow

baiduspider

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

baiduspider-image

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

googlebot

Rule Path
Allow /article/*
Allow /w/*
Disallow /search

bingbot

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

sogou web spider

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

sogou inst spider

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

sogou spider2

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

sogou blog

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

sogou news spider

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

sogou orion spider

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

sosospider

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search

yisouspider

Rule Path
Allow /article/*
Allow /w/*
Disallow /article/*?*
Disallow /w/*?*
Disallow /search
Back to top

Warnings

  • 6 invalid lines.
Back to top