Jev 能用来做什么：从概率门到可计算的信念模型
What Jev can be used for: from probability gates to computable belief models
总结前两篇的理论和实验，提出一种分工：Jev 只估计最基础的概率，组合与推断全部交给数学，准确度靠真实数据校准。按这种分工，可以把大模型的知识整理成一个彼此自洽、可以编辑和推演的概率模型。文章讨论它和直接问大模型的区别，以及几类应用方向。
This post brings together the theory and experiments of the previous two and proposes a division of labour. Jev estimates only the most basic probabilities, math does all combination and inference, and real data calibrates accuracy. With this split, a language model's knowledge can be turned into a probability model that is self-consistent, editable and explorable. It discusses how this differs from asking a language model directly, and several application areas.
前两篇《概率门：从比特到概率的电路》和《概率门实验：Jev 的概率能直接用吗》分别讨论了理论和实验。这一篇把结论放在一起，看 Jev 适合用来做什么。
目前的结论
实验部分的发现可以归成六条：
- 快，便宜，可以批量问。服务端单次约 0.2 秒；一次请求同时问十几个问题，彼此基本不影响。
- 输出几乎是确定的。同一个问题重复问，答案几乎不变，所以误差主要是系统误差，重复调用不能减小误差。
- 严格互补的是否问题基本自洽。「是」和「不是」两个概率之和与 1 平均只差约 0.04。
- 直接问组合问题不可靠。分别问 A、B、A 且 B、A 或 B，81% 的组合违反概率规则；「或」被系统性低估约 0.10。
- 排序准，数值偏向中间。在 33 组人口普查数据上，Jev 估计的高收入比例与真实比例的排序相关约 0.97–0.98，但低比例被高估；用少量真实数据校准后，误差从 0.119 降到 0.042。
- 分步组合不会自动更准。方向不一的误差会抵消，同向偏差会原样传下去，只能靠校准处理。
一种分工
这些结论指向一种分工：
- Jev 只估计最基础的概率：单个命题的概率，以及在明确条件下的条件概率。单个对象的判断用是否问题，群体比例用按档打分，互斥的类别用多选题；
- 组合、推断、抽样全部交给数学：「且」「或」、边缘化、贝叶斯更新、抽样，都由程序按概率规则计算，不再问 Jev；
- 准确度靠真实数据校准：用已知结果修正 Jev 的系统偏差，而且在使用的那一层上校准；
- 结构由大模型、人或现成的命题库提供：有哪些命题，谁依赖谁。其中依赖关系也可以交给 Jev 判断，见下文。
这样分工之后，自洽由数学保证，不依赖 Jev；准确度由校准决定，可以用数据持续检验。
用是否问题确定一个大模型
只用是否问题，能不能确定一个包含大量命题的完整概率模型？可以，有现成的数学结构。
n 个是否命题共有 2ⁿ 种组合，直接描述完整的联合分布需要 2ⁿ − 1 个数，不可能逐个去问。贝叶斯网络用链式法则把它拆开：每个命题只依赖少数几个「父命题」，需要的参数是「已知父命题取某组值时，这个命题成立的概率」。每个参数正好是一个带条件的是否问题。
这组参数有一个关键性质：它们在数学上相互独立。每个参数取 0 到 1 之间的任何值，合起来都是一个合法、自洽的联合分布。所以参数可以分别问、分别校准、分别替换，自洽由结构本身保证。实验里看到的矛盾，只会在同一个量被重复估计的时候出现。
如果问了很多相互重叠的问题，比如单项概率和两两同时成立的概率，答案之间可能不一致。这时可以用最大熵方法：找一个尽量满足所有答案、此外不做多余假设的分布。数学上它是一个玻尔兹曼机：每个命题有一个偏好，每两个命题之间有一个耦合强度。
这和训练模型参数的方式不同。玻尔兹曼机和神经网络的参数，通常要从大量数据中学习，或者用强化学习调整。这里 Jev 直接给出概率，程序据此算出参数，不需要训练。知识仍然来自一个训练好的模型（Jev 本身），但它被整理成了一组有名字、有含义、可以单独检查的数字。
结构也可以大部分交给 Jev
结构包括两部分：有哪些命题，以及谁依赖谁。
依赖关系可以用是否问题确定。对每一对命题问「A 会直接影响 B 吗」，n 个命题需要 n(n−1) 个问题。50 个命题约 2,450 个问题，这些问题读的是同一段背景，一次请求可以问十几个，总共一两百次请求。还要处理两件事：
- 间接影响。如果 A 通过 B 影响 C，Jev 也可能对「A 直接影响 C」回答是。可以再问「已知 B，A 还会影响 C 吗」，去掉多余的边。这就是结构学习中 PC 算法的做法，只是把独立性检验从统计检验换成了 Jev 的判断。
- 环。如果「A 影响 B」和「B 影响 A」都被判断为是，贝叶斯网络不允许这种情况。可以问「A 发生在 B 之前吗」来确定方向，或者由程序保留概率更高的方向。
有哪些命题，Jev 自己提不出来，因为它不生成文字。但它可以从现成的命题库里挑选：对库里的每一条问「它和这个问题相关吗」。如果为某个领域事先建好一个足够大的命题库（这一步由大模型或人完成，只做一次），运行时就只需要 Jev 和数学：挑选命题、确定依赖、估计概率、计算结论。这样的系统在运行时快、便宜，结果可以复现。
和直接问大模型有什么不同
模型建好之后，能做到几件直接问大模型做不到的事：
- 所有回答彼此自洽。对同一个模型问 100 个不同的问题，答案都来自同一个分布，不会相互矛盾。分别问大模型 100 个问题，答案之间常常对不上。
- 假设分析即时完成。设定「假设这件事已经发生」，其余所有命题的概率按贝叶斯规则精确更新，不需要再调用任何模型。
- 能看到结论依赖哪些判断。可以计算结论对每个参数的敏感程度，列出「这个结论主要取决于这几个判断」。用户不同意哪一个，就改哪一个，结论随之更新。
- 能给出结论的可靠程度。同一个参数用几种问法分别估计，把它们的差异传到结论上，得到的是一个区间，比如「62%，区间 55%–70%」，而不只是一个数。
- 可以给出多种结局。从联合分布中抽样，每次得到一个内部自洽的「可能世界」，再由大模型写成一段描述。各个结局的概率由同一个分布决定，不是凭空给出的百分比。
- 一次建模，反复使用。一个上万参数的模型，用 Jev 实例化一次大约需要几分钟，费用在一美元以内；之后的查询都在本地计算，即时返回。
这条路有人走过
决策分析在 1970 到 1980 年代就形成了一套成熟方法：影响图、信息价值、敏感性分析，斯坦福的 Ron Howard 是这个领域的重要推动者。它和贝叶斯网络遇到的是同一个问题：每个参数都需要专家估计，成本很高，模型因此做不大。Jev 提供了一个便宜、随时可用、可以调用上万次的估计来源。方法是现成的，过去缺的是这个来源。
应用方向
按落地难度从低到高：
- 业务判断流水线。消息路由、工单分诊、操作审批、风险分级。每一步给出校准过的概率，不确定的交给人，每次判断都有记录。这是 Jev 最直接的用法。
- 验收评测。按验收标准逐条判断 AI 系统的输出是否合格，先用人工评分校准，之后每次改动都自动跑一遍全部样例。
- 可推演的信念模型。用户提出一个问题，比如「这个项目年底前会不会延期」：大模型列出关键命题和依赖关系，Jev 填写全部条件概率，界面上显示一张命题图。点任一命题设为「已发生」，其余概率立刻更新；侧边列出结论最依赖的判断和分歧最大的参数；还可以抽出几种可能的结局。事情有了结果之后回填，用于下一次校准。它和单个 Jev（问一个问题，得到一个数）、和大模型（问一个问题，得到一段话）都不一样，是在一个自洽的模型里探索。
- 其他实现：
- 诊断：每一步选出最能缩小不确定性的问题，用尽量少的提问定位原因；
- 事件追踪：跟踪某个事件的概率，随新消息逐条更新，并显示是哪条消息改变了概率；
- 用自然语言建立概率模型：大模型提出变量和依赖，Jev 填写参数，少量数据校准。
在中国大陆向客户提供服务时，把数据发给境外模型涉及数据出境。概率门是一层抽象，也可以用国内模型加校准实现，同一套结构和工具都能沿用。
边界
- 数学上独立，不等于误差独立。所有参数来自同一个 Jev，系统偏差是共有的，必须校准。
- 结构可能出错。大模型提出的命题和依赖关系不一定对，结构错误比参数误差影响更大。
- 规模带来的不是更强的能力。参数再多，也不会超出 Jev 本身的知识；规模换来的是覆盖面、一致性和可解释性。
- 开放问题缺少校准数据。没有已知结果的问题无法直接校准，只能借用同类已揭晓问题的校准，在此之前概率主要用于比较和排序。
下一步
- 用 Adult 数据搭一个六七个变量的小型贝叶斯网络，参数全部由 Jev 估计，和真实数据、直接询问的结果比较；
- 做一个「可推演的信念模型」原型；
- 继续前瞻预测实验，检验开放问题上的校准。
说明：Jev 通过 Vercel AI Gateway 按量付费调用。作者和榫卯 Tenon AI 与 TypeSafe AI 没有合作关系，也没有收到任何费用。
The previous two posts, "Probability gates: from bits to probabilities" and "Probability gate experiments: can Jev's probabilities be used as they are?", covered the theory and the experiments. This post puts the conclusions together and looks at what Jev is suited for.
What we know so far
The experimental findings come down to six points:
- Fast, cheap and batchable. A call takes about 0.2 seconds on the server, and one request can ask a dozen or more questions with little interaction between them.
- The output is almost deterministic. Asking the same question again barely changes the answer, so the errors are mainly systematic and repeated calls do not reduce them.
- Strictly complementary yes/no questions are mostly consistent. The "yes" and "no" probabilities sum to within about 0.04 of 1 on average.
- Asking compound questions directly is unreliable. Asking separately for A, B, A and B, and A or B, 81% of combinations violated the rules of probability, and "or" was systematically underestimated by about 0.10.
- The ranking is accurate; the values are pulled toward the middle. On 33 census groups, Jev's estimated shares of high earners had a rank correlation of about 0.97–0.98 with the true shares, but low shares were overestimated. Calibrating with a small amount of real data cut the error from 0.119 to 0.042.
- Composing in steps does not automatically improve accuracy. Errors in different directions cancel; bias in a shared direction passes straight through and can only be handled by calibration.
A division of labour
These conclusions point to a division of labour:
- Jev estimates only the most basic probabilities: the probability of a single proposition, and conditional probabilities under stated conditions. Yes/no questions for judgments about one case, a scale for the share of a population, multiple choice for mutually exclusive categories.
- Math does all combination, inference and sampling: "and", "or", marginalization, Bayesian updating and sampling are computed by code according to the rules of probability, without asking Jev again.
- Real data calibrates accuracy: known outcomes correct Jev's systematic bias, calibrated at the level where the numbers are used.
- A language model, a person or an existing proposition library supplies the structure: which propositions exist and what depends on what. The dependencies can also be judged by Jev; see below.
With this split, consistency is guaranteed by the math and does not depend on Jev; accuracy depends on calibration and can be checked against data continuously.
Building a large model from yes/no questions
Can yes/no questions alone determine a complete probability model over many propositions? Yes; there is an existing mathematical structure for it.
n yes/no propositions have 2ⁿ combinations, so describing the full joint distribution directly takes 2ⁿ − 1 numbers, far too many to ask one by one. A Bayesian network breaks it down with the chain rule: each proposition depends on only a few "parent" propositions, and the parameters needed are "the probability that this proposition holds, given a particular set of values for its parents". Each parameter is exactly one conditional yes/no question.
These parameters have a key property: they are mathematically independent of each other. Whatever value between 0 and 1 each one takes, together they always form a valid, consistent joint distribution. So parameters can be asked, calibrated and replaced one at a time, and consistency comes from the structure itself. The contradictions seen in the experiments only appear when the same quantity is estimated more than once.
If you ask many overlapping questions, such as single probabilities and the probabilities of pairs holding together, the answers may disagree. Then you can use the maximum entropy method: find the distribution that satisfies all the answers as closely as possible and assumes nothing beyond them. Mathematically it is a Boltzmann machine: each proposition has a bias, and each pair of propositions has a coupling strength.
This differs from how model parameters are usually obtained. The parameters of Boltzmann machines and neural networks are normally learned from large amounts of data or adjusted with reinforcement learning. Here Jev gives the probabilities directly, and code derives the parameters from them, with no training. The knowledge still comes from a trained model (Jev itself), but it is organized into a set of named, meaningful numbers that can each be checked.
Most of the structure can also go to Jev
The structure has two parts: which propositions exist, and what depends on what.
Dependencies can be settled with yes/no questions. For every pair of propositions, ask "does A directly affect B?"; n propositions need n(n−1) questions. Fifty propositions need about 2,450 questions, and since they all read the same background, one request can ask a dozen or more, for a total of one or two hundred requests. Two things still need handling:
- Indirect effects. If A affects C through B, Jev may also answer yes to "does A directly affect C?". A further question, "given B, does A still affect C?", removes the redundant edge. This is how the PC algorithm for structure learning works, with Jev's judgment replacing the statistical independence test.
- Cycles. If both "A affects B" and "B affects A" are judged true, a Bayesian network cannot represent it. Asking "does A happen before B?" can settle the direction, or code can keep the direction with the higher probability.
Jev cannot come up with the propositions themselves, because it does not generate text. But it can choose from an existing library by asking, for each entry, "is this relevant to the question?". If a large enough proposition library is built for a domain in advance (done once, by a language model or by people), then at run time only Jev and math are needed: choose the propositions, settle the dependencies, estimate the probabilities and compute the conclusions. Such a system is fast and cheap at run time, and its results are reproducible.
How this differs from asking a language model directly
Once the model is built, it can do several things that asking a language model directly cannot:
- Every answer is consistent with every other. Ask the same model 100 different questions and all the answers come from one distribution, so they never contradict each other. Ask a language model 100 separate questions and the answers often disagree.
- What-if analysis is instant. Set "suppose this has happened", and the probabilities of all other propositions update exactly by Bayes' rule, without calling any model again.
- You can see which judgments a conclusion depends on. You can compute how sensitive the conclusion is to each parameter and list "this conclusion depends mainly on these judgments". If the user disagrees with one, they change that one and the conclusion updates.
- It can say how reliable a conclusion is. Estimate the same parameter with several wordings and propagate the spread to the conclusion, which gives a range such as "62%, range 55%–70%" instead of a single number.
- It can give several outcomes. Sampling from the joint distribution gives an internally consistent "possible world" each time, which a language model can then write up as a description. The probabilities of the outcomes all come from the same distribution rather than being made-up percentages.
- Build once, use many times. A model with ten thousand parameters takes a few minutes and under a dollar to instantiate with Jev; after that every query is computed locally and returns instantly.
Others have walked this path
Decision analysis developed a mature set of methods in the 1970s and 1980s: influence diagrams, value of information and sensitivity analysis, with Ron Howard at Stanford as a leading figure. It hit the same problem as Bayesian networks: every parameter had to be estimated by an expert, which was expensive and kept models small. Jev provides an estimation source that is cheap, always available and can be called tens of thousands of times. The methods already exist; what was missing was the source.
Application areas
From easiest to hardest to put into practice:
- Business judgment pipelines. Message routing, ticket triage, action approval, risk grading. Each step gives a calibrated probability, uncertain cases go to a person, and every judgment is recorded. This is the most direct use of Jev.
- Acceptance evaluation. Check an AI system's output against acceptance criteria one by one, calibrate against human ratings first, then rerun all the examples automatically on every change.
- Explorable belief models. A user poses a question such as "will this project be late by the end of the year?". A language model lists the key propositions and dependencies, Jev fills in all the conditional probabilities, and the interface shows a graph of propositions. Click any proposition to mark it as "happened" and the other probabilities update immediately; a side panel lists the judgments the conclusion depends on most and the parameters with the widest disagreement; and you can sample several possible outcomes. When the real result is known, it is fed back for the next calibration. This is different from a single Jev (ask one question, get one number) and from a language model (ask one question, get a paragraph): you explore inside a consistent model.
- Other uses:
- Diagnosis: at each step, choose the question that most reduces uncertainty, to find the cause with as few questions as possible.
- Event tracking: follow the probability of an event, update it with each new piece of news, and show which item changed it.
- Building probability models in natural language: a language model proposes variables and dependencies, Jev fills in the parameters, and a small amount of data calibrates them.
When serving clients in mainland China, sending data to a model abroad is a cross-border data transfer. A probability gate is an abstraction and can also be implemented with a domestic model plus calibration, keeping the same structure and tools.
Limits
- Mathematically independent does not mean independent errors. All parameters come from the same Jev, so systematic bias is shared and calibration is required.
- The structure can be wrong. The propositions and dependencies a language model proposes are not necessarily right, and structural errors matter more than parameter errors.
- Scale does not bring more capability. However many parameters there are, they cannot exceed Jev's own knowledge; scale buys coverage, consistency and explainability.
- Open questions lack calibration data. Questions without known outcomes cannot be calibrated directly; they can only borrow calibration from similar resolved questions, and until then the probabilities are mainly useful for comparison and ranking.
Next
- Build a small Bayesian network with six or seven variables on the Adult data, with every parameter estimated by Jev, and compare it with the real data and with asking directly.
- Build a prototype of an explorable belief model.
- Continue the forward forecasting experiment to test calibration on open questions.
Note: Jev was called and paid for per use through the Vercel AI Gateway. The author and Tenon AI have no business relationship with TypeSafe AI and have received no payment.