Enloq, my own project, measured on 27 and 29 October 2026
How Ask this group finds and checks an answer.
Ask this group is Enloq's retrieval-augmented generation (RAG) feature. This page shows how it retrieves, how it checks its citations, what was measured and what wasn't.
The result, and its limits.
| What it does | Answers a member's question from that group's own cached messages, as up to four statements with source links. When the messages don't settle a question, it says so. |
|---|---|
| Measured on | 100 authored questions over six synthetic groups. Not real groups or live traffic. |
| Result | 78 of the 80 answerable questions correct, graded by a model against authored answers. |
| Tool calling | 78 against 73 without tools, and none lost. That gain isn't statistically significant. |
| Not shown | Production use, adoption or quality on real groups. Enloq is a personal deployment, shared with a few approved members. |
What Ask does.
A daily briefing summarizes one day. Finding one answer across a group's recent days still means scrolling. Ask this group answers a member's question from that group's own messages, in up to four short statements. Each statement links to the messages it rests on. A link opens the group at that message, and Back returns to the answer.
It searches only what Enloq already caches for that group: text, captions, voice transcripts and image descriptions, from the group's first opt-in day, within a 30-day cache. Hidden and ignored bots, flagged ads and service events stay out. It reads no Telegram history, no other chat and no web. Enloq's database keeps neither the question nor the answer: they exist only on the open screen, and receipts keep usage without content.
How an answer is made.
A question goes through these steps. The first three are the fixed pass.
| Step | What happens |
|---|---|
| Rank | The question's embedding ranks the group's messages by similarity. |
| Rerank | A reranker reorders the first 40. If reranking fails, the embeddings' order stands. |
| Read | The writer model, gpt-6.1-sol, reads the best 25. |
| Search further | If those don't settle the question, the writer calls its tools. |
| Write | It writes its statements, citing only messages it has read. Any other ID fails validation. |
| Check | A decision model checks each statement against its citations. Nothing streams: an answer shows once checked. |
Embeddings, picked by measurement.
The measurement ran on 27 October 2026 over six synthetic groups, 665 messages in Russian, English and both. It used 100 authored questions, labelled before any run. 80 are answerable, 11 of those asked in another language than the group's, and 20 aren't.
Recall at k is the share of the relevant messages in the first k. MRR is the mean reciprocal rank of the first relevant one. nDCG at 10 rewards relevant messages ranked early.
| Retrieval | Recall at 5 | at 10 | at 25 | MRR | nDCG at 10 |
|---|---|---|---|---|---|
| Keywords alone (FTS5 trigram, BM25) | 0.53 | 0.63 | 0.67 | 0.38 | 0.41 |
| Qwen3 Embedding 8B | 0.80 | 0.85 | 0.92 | 0.63 | 0.66 |
| Perplexity Embed V1 4B (adopted) | 0.80 | 0.89 | 0.92 | 0.66 | 0.69 |
| bge-m3 | 0.76 | 0.87 | 0.94 | 0.58 | 0.62 |
| Qwen3 Embedding 8B with keywords | 0.75 | 0.81 | 0.88 | 0.61 | 0.63 |
| Perplexity Embed V1 4B with keywords | 0.75 | 0.83 | 0.87 | 0.62 | 0.63 |
| bge-m3 with keywords | 0.73 | 0.81 | 0.90 | 0.58 | 0.60 |
Perplexity Embed V1 4B ranked best on MRR and nDCG and was the fastest on these routes. One request of up to 64 messages took 1.3 seconds at the median, against 2.4 for bge-m3 and 9.2 for Qwen3 at Nebius. bge-m3 found slightly more by recall at 25.
The reranker stayed. Keyword search didn't.
Fusing keywords with the embeddings by reciprocal rank lowered every embedding model's ranking. Across languages, keywords found almost nothing: recall at 25 of 0.05 on those 11 questions, where the fusion reached 0.95. The final run compared the reranker's two possible lists.
| Candidates | Recall at 40 before reranking | nDCG at 10 before | Recall at 10 after | nDCG at 10 after |
|---|---|---|---|---|
| The embeddings' first 40 (adopted) | 0.95 | 0.69 | 0.93 | 0.78 |
| The keyword fusion's first 40 | 0.94 | 0.63 | 0.91 | 0.77 |
Reranking lifted nDCG at 10 from 0.69 to 0.78. After it the two lists rank about alike: one rerank in that run fell back after a provider refused it, and one question moves these means by up to 0.0125. Before it the embeddings' list is better. So the application reranks the embeddings' list and leaves keyword search out.
The reranker is Qwen3 Reranker 8B at Fireworks, the one zero-retention reranker listed when it was chosen. Voyage's rerank-3 ranked alike on the fused list, but on 27 October its terms allowed training on submitted content unless the customer opted out. So it ran on authored messages only and never receives real group messages.
When the fixed pass doesn't settle it.
After the 25, the writer can call three tools. It can search again in its own words, for the best 10. It can read up to five messages on each side of one. And it can read a message with the one it replies to and up to ten replies. The tools read only this group's searchable messages, and their results are framed as untrusted data, like the messages. Per question the writer gets at most five requests, four searches and twelve tool calls, and the fifth request must answer.
On 29 October 2026, each of the 100 questions was answered three ways through the application's own steps: the fixed pass, the tools alone with no messages to start with, and the fixed pass then the tools. One judge request graded the three answers together against the authored answer, in a shuffled order. The rule was fixed before the run: a way qualifies with at least two more answerable questions correct than the fixed pass, no fewer unanswerable ones and no more statements judged unsupported. The qualifying way with the most correct answers is adopted if its median time is at most twice the fixed pass's.
| Result | Fixed pass | Tools alone | Fixed pass, then tools |
|---|---|---|---|
| Answerable: correct | 73 of 80 (82.8% to 96.4%) | 74 of 80 (84.4% to 97.2%) | 78 of 80 (91.3% to 99.7%) |
| Unanswerable: correct | 20 of 20 | 20 of 20 | 20 of 20 |
| Unanswerable: said no answer | 3 of 20 | 3 of 20 | 3 of 20 |
| Statements judged unsupported | 4 of 135 | 2 of 124 | 2 of 127 |
| Seconds: median (95th) | 6.5 (12.8) | 13.8 (25.7) | 7.7 (22.4) |
| Relevant messages read | 0.95 | 0.96 | 0.99 |
| Outcome | The baseline | Didn't qualify | Adopted |
Ranges are two-sided 95% exact (Clopper-Pearson). Relevant messages read is the mean share of a question's labelled messages that its writer read, over the 80 answerable questions.
The fixed pass then the tools met the rule and was adopted. It answered five questions the fixed pass missed and lost none. An exact two-sided McNemar test gives p = 0.0625. So it met the practical bar set before the run, not significance at 5% on 80 questions. In those five, the message that settles the question wasn't among the 25: for example, a correction posted right after an instruction, or the reply claiming a lost multimeter. Tools alone didn't qualify, and the application doesn't use that way.
The price is waiting. A question took 17.9 seconds at the median when the writer called tools, against 6.7 when it didn't. Averaged over all 100 questions, the adopted way cost about $0.0014 on the metered routes, against $0.0009 for the fixed pass. That leaves out the writer, which runs on my ChatGPT subscription. Vectors were computed before the run, so these figures also leave out a group's first question, which pays for its messages' vectors.
The citation check.
The check is a decision model, Perplexity's Decider. It answers fixed yes-or-no questions with probabilities and writes no text. For each statement it's asked whether each cited message supports part of it, and whether together they support all of it.
A citation below 0.5 is removed, and a statement left with none is dropped. A statement its citations don't fully support shows a muted line: "Not all of this is in its sources." If the check can't run, no answer shows, only "Couldn't check the answer."
| On 70 authored claims, labelled before the run | Count (95% range) |
|---|---|
| Supporting citations removed | 8 of 93 (3.8% to 16.3%) |
| Unsupported citations kept | 1 of 69 (0.04% to 7.8%) |
| Correct claims judged unsupported | 1 of 32 (0.08% to 16.2%) |
| Wrong claims judged supported | 3 of 38 (1.7% to 21.4%) |
So a wrong claim can get through. In the tool-calling run the check dropped no statement and marked 6 of the adopted way's 127 as not fully supported. The judge found 2 of those 127 unsupported. One added "that date has passed" to a power cut, which the messages don't say.
When the messages hold no answer.
If the group has nothing to search, no model is called. If the writer returns no statements, or no statement survives the check, the answer is one line that names the searched range: "No answer in this group's messages from 3 Oct to today."
Edited and deleted messages.
A message's vector is computed when a question first needs it, and kept per message revision with the 30-day cache. An edit or a deletion drops it. An edited message is embedded again the next time a question needs it, and a deleted one leaves the search.
Where the messages go.
The embedding and rerank requests go through OpenRouter. Each is pinned to one provider with no fallback and requires zero data retention: Perplexity for embeddings and Fireworks for reranking. That rests on how OpenRouter lists those routes.
Only those two requests require zero retention. The writer runs on my ChatGPT subscription, so messages reach OpenAI under that plan's data settings. The check goes to Perplexity's decision model. On 27 October, Perplexity published no retention terms for it, and OpenRouter listed it as zero retention. My own proxy logs the check's requests, which hold each statement and its cited messages, for up to 30 days.
What this doesn't show.
These are authored questions over synthetic groups, not real groups or live traffic. Each group holds about 110 messages, so the writer's 25 are nearly a quarter of it. A busy group's 30 days can hold far more, and nothing here measures retrieval or waiting time there.
The same questions chose each component, so 78 of 80 isn't a held-out estimate. A model graded the answers, the same one that wrote them, so the grades aren't an independent human rating. The samples are small, as the ranges show, and the times and costs are indicative, not a benchmark. Tool calling's gain isn't statistically significant.
This isn't production use. Enloq is a personal deployment, shared with a few approved members, and nothing here measures how they use Ask.
Evidence basis.
This account is based on Enloq's public decision records, measurements and code. The authored groups, questions and labels are public in the repository, and the measurements list the tool-calling run's misses. The run reports, which hold each model's answers, are private.