arthing.org
robots.txt

Robots Exclusion Standard data for arthing.org

Resource Scan

Scan Details

Site Domain arthing.org
Base Domain arthing.org
Scan Status Ok
Last Scan2026-01-09T05:57:22+00:00
Next Scan 2026-01-16T05:57:22+00:00

Last Scan

Scanned2026-01-09T05:57:22+00:00
URL https://arthing.org/robots.txt
Domain IPs 96.44.182.178
Response IP 96.44.182.178
Found Yes
Hash c861bdf90600ece3e74b5e11336a1033853d6ac7ffb9544f815bea9af0084b97
SimHash eaaad6120aab

Groups

semrushbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /tag/

*

Rule Path Comment
Disallow /wp-admin/ -
Disallow /wp-includes/ -
Disallow /wp-content/plugins/ -
Disallow /wp-content/themes/*.php -
Disallow /wp-json/ -
Disallow /xmlrpc.php -
Disallow /search -
Disallow /*?s= -
Disallow /*?*wptouch_switch= -
Disallow /*?*wp_switch= -
Disallow /*-2048x*.jpg$ -
Disallow /*-2560x*.jpg$ -
Disallow /*.png$ 如想放行缩略图可删除此行
Disallow /*.webp$ 如å

googlebot

Rule Path
Allow /wp-content/uploads/*-300x*.jpg$
Allow /wp-content/uploads/*-1024x*.jpg$
Allow /wp-content/uploads/*-150x*.jpg$
Disallow /*-2048x*.jpg$

Other Records

Field Value
crawl-delay 30

bingbot

Rule Path
Disallow /feed
Disallow /tag/*/page/
Disallow /category/*/page/
Allow /archives/
Allow /wp-content/uploads/*-1024x*.jpg$

Other Records

Field Value
crawl-delay 30

sogou web spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

googlebot-image

Rule Path
Disallow /wp-content/uploads/

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /author/*/page/
Disallow /tag/*/page/
Disallow /archives/*/page/
Disallow /app/*
Disallow /arthingevent/*/page/
Disallow /arthingnews/*/page/
Disallow /page/*

Other Records

Field Value
crawl-delay 30

Other Records

Field Value
sitemap https://www.arthing.org/sitemap.xml
sitemap https://www.arthing.org/sitemap.html

Warnings

  • 1 invalid line.