Log In Sign Up

stmforum.com
robots.txt

Robots Exclusion Standard data for stmforum.com

Resource Scan

Scan Details

Site Domain stmforum.com
Base Domain stmforum.com
Scan Status Ok
Last Scan2024-05-18T20:04:35+00:00
Next Scan 2024-06-17T20:04:35+00:00

Last Scan

Scanned2024-05-18T20:04:35+00:00
URL https://stmforum.com/robots.txt
Redirect https://affiliateworldforum.com/robots.txt
Redirect Domain affiliateworldforum.com
Redirect Base affiliateworldforum.com
Domain IPs 45.79.188.79
Redirect IPs 104.21.93.138, 172.67.210.219, 2606:4700:3031::ac43:d2db, 2606:4700:3033::6815:5d8a
Response IP 104.21.93.138
Found Yes
Hash cc84ab2f23f6bfb5bbbeac1091fc5c4258f9715366d1ec7011f1347186e68c99
SimHash 2a31995165f0

Groups

adidxbot
applebot
applenewsbot
baiduspider
baiduspider-image
bingbot
bingpreview
ccbot
cliqzbot
coccoc
coccocbot-image
coccocbot-web
daumoa
dazoobot
deusu
duckduckbot
duckduckgo-favicons-bot
euripbot
exploratodo
facebot
feedly
findxbot
gooblog
apis-google
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
googlebot
googlebot-image
googlebot-mobile
googlebot-news
googlebot-video
mediapartners-google
haosouspider
ichiro
istellabot
jikespider
lycos
mail.ru
mojeekbot
msnbot
msnbot-media
orangebot
pinterest
plukkie
qwantify
rambler
seznambot
sosospider
slurp
sogou blog
sogou inst spider
sogou news spider
sogou orion spider
sogou spider2
sogou web spider
sputnikbot
teoma
twitterbot
wotbox
yacybot
yandex
yandexmobilebot
yeti
yioopbot
yoozbot
youdaobot

Rule Path
Disallow

*

Rule Path
Disallow /
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /trackback/
Disallow /wp-login.php
Disallow /wp-register.php
Allow /wp-content/uploads/
Allow /wp-admin/admin-ajax.php
Allow /wp-includes/js/
Allow /wp-includes/images/
Back to top

Other Records

Field Value
sitemap https://meetups.stmforum.com/meetups_sitemap.xml
Back to top

Comments

  • ROBOTS.TXT
  • Alphabetically ordered whitelisting of legitimate web robots, which obey the
  • Robots Exclusion Standard (robots.txt). Each bot is shortly described in a
  • comment above the (list of) user-agent(s).
  • Important: Blank lines are not allowed in the final robots.txt file!
  • Last update: 2018-09-8
  • so.com chinese search engine
  • bing ads bot
  • apple.com search engine
  • baidu.com chinese search engine
  • bing.com international search engine
  • commoncrawl.org open repository of web crawl data
  • cliqz.com german in-product search engine
  • coccoc.com vietnamese search engine
  • daum.net korean search engine
  • dazoo.fr french search engine
  • deusu.de german search engine
  • duckduckgo.com international privacy search engine
  • eurip.com european search engine
  • exploratodo.com latin search engine
  • facebook.com social network
  • feedly.com feed fetcher
  • findx.com european search engine
  • goo.ne.jp japanese search engine
  • google.com international search engine
  • google.com landing page quality check
  • google.com app resource fetcher
  • google.com adsense bot
  • so.com chinese search engine
  • goo.ne.jp japanese search engine
  • istella.it italian search engine
  • jike.com / chinaso.com chinese search engine
  • lycos.com & hotbot.com international search engine
  • mail.ru russian search engine
  • mojeek.com search engine
  • bing.com international search engine
  • orange.com international search engine
  • pinterest.com social networtk
  • botje.nl dutch search engine
  • qwant.com french search engine
  • rambler.ru russian search engine
  • seznam.cz czech search engine
  • soso.com chinese search engine
  • yahoo.com international search engine
  • sogou.com chinese search engine
  • sputnik.ru russian search engine
  • ask.com international search engine
  • twitter.com bot
  • wotbox.com international search engine
  • yacy.net p2p search software
  • yandex.com russian search engine
  • search.naver.com south korean search engine
  • yioop.com international search engine
  • yooz.ir iranian search engine
  • youdao.com chinese search engine
  • crawling rule(s)
  • disallow all other bots
  • Sitemaps
Back to top

Warnings

  • 3 invalid lines.
Back to top