4irelabs.com
robots.txt

Robots Exclusion Standard data for 4irelabs.com

Resource Scan

Scan Details

Site Domain 4irelabs.com
Base Domain 4irelabs.com
Scan Status Ok
Last Scan2025-10-18T17:20:53+00:00
Next Scan 2025-11-17T17:20:53+00:00

Last Scan

Scanned2025-10-18T17:20:53+00:00
URL https://4irelabs.com/robots.txt
Domain IPs 104.21.74.149, 172.67.159.100, 2606:4700:3033::ac43:9f64, 2606:4700:3034::6815:4a95
Response IP 172.67.159.100
Found Yes
Hash 8d5eb8ffe266bde3136a65cb8a13b16523d74b50bfadfab8de498e9f33ecaf85
SimHash 61144110c2b6

Groups

*

Rule Path
Allow /
Disallow /employee/
Disallow /reviews/
Disallow /ctypes/
Disallow /type/
Disallow /wp-comments-post.php
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /go/
Disallow /tag/
Disallow /author/
Disallow /cdn-cgi/
Disallow /*/feed/
Disallow /*/comments/
Disallow /*?s=
Disallow /*?p=
Disallow /license.txt

gptbot
chatgpt-user
oai-searchbot

Rule Path
Allow /

claudebot
claude-user
claude-searchbot

Rule Path
Allow /

perplexitybot
perplexity-user

Rule Path
Allow /

google-extended
bard-google
gemini

Rule Path
Allow /

meta-externalagent
facebookbot

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

bing-bingbot

Rule Path
Allow /

ccbot
youbot
duckassistbot
amazonbot

Rule Path
Allow /

ai2bot
cohere
diffbot
omgili
omgilibot
petalbot
timpibot
velenpublicwebcrawler
dataforseobot
imagesiftbot
bytespider
tiktokspider
kangaroo bot
sentibot
img2dataset
meltwater
seekr
peer39_crawler
duckassistbot
scrapy
cotoyogi
aihitbot
factset_spyderbot
firecrawlagent

Rule Path
Allow /

Other Records

Field Value
sitemap https://4irelabs.com/sitemap.xml
sitemap https://4irelabs.com/cases-sitemap.xml
sitemap https://4irelabs.com/articles-sitemap.xml
sitemap https://4irelabs.com/page-sitemap.xml

Comments

  • === AI BOTS - EXPLICITLY ALLOWED ===
  • OpenAI (ChatGPT, GPT models)
  • Anthropic (Claude AI)
  • Perplexity AI
  • Google AI (Bard, Gemini, AI training)
  • Meta AI (Facebook AI)
  • Apple Intelligence
  • Microsoft AI (Bing Chat)
  • Other Major AI Services
  • Additional AI Bots

Warnings

  • `host` is not a known field.