我以前聊 AI 就像聊天气,只能忍着,没法设计。让我改变想法的是,我注意到任何干活的人,不管是人还是机器人,都能选择真把活干完,或者只让活看起来干完了,而他会挑更省事的那条路。

造假不花成本,就只会得到假货。所以我开始要求每一步都交回执:一种不真干活就写不出来的东西,比如带出处的原文原句。一旦造假变贵,真干活反倒成了更便宜的选择。 我抱怨的问题,归根结底是四个骑士根子上的问题。下面是我针对每一个用的规则。

1. 它说自己做了,其实没做 “打开报告,查一下三月的支出。”“打开了,340k。”报告上写的是 430k。 说一句“我做了”,模型不花任何代价,而我以前还真就信了它。

When you say you did something — opened, read, checked, calculated — show the thing itself: quote the line of the document the number came from; show the first lines of what you opened; show the calculation. If you say
"done" without showing the thing, I treat it as not done.
Give any number, standard, date, quote or link only with the document it came from, so I can open it and check. If there is no document or you can't open it, write "no source" and don't fill in a number from memory. In your
answer, label: "from the document", "from general knowledge", "my estimate".

Before you say "checked", prove your check works: plant two errors of the kind you're looking for in a copy of my text, run the check, and show that it caught them. Only then give a verdict on the real text.

一个埋进去的错误,被检查真的抓出来了——这是我找到的最好的回执。

更新:正如 depotwerkstatt 在评论中指出的,错误要你自己埋,而不是让模型来埋。

2. 每次答案都不一样 同一个问题问下去,每次给的答案都不同,还塞满了空话和套话。我只要结果、没给步骤,模型就每次自己现编一套流程。 现在我让它先说出既有的标准流程(采购方怎么比供应商、律师怎么读合同、编辑怎么校对),先给我看,再照着走。流程一旦摆在眼前,哪一步被跳过了就一目了然。查我的化验结果时,它走的是教科书上的鉴别诊断法,得出的各种假设我会拿去给我的主治医生看。

Don't answer right away. First describe how professionals solve this kind of task: what steps and criteria they use. Show me those steps. Then go
through them in order, and at each one write what you took from my data and what conclusion follows. If there's no data for a step, write "no data" — don't fill the gap by guessing. At the end: how confident you are, and why you're that confident.

3. 它记不住规则

「简短点」和「先给数字」这两条,顶多管两轮回复。为了让它记住上下文而一直用同一个超长对话,反而让我这边更糟。我没法检查它到底读没读我的规则,但我能检查它有没有点出适用的是哪条规则、以及那条规则改变了什么。 每个任务开始前:

Before you start, list which of my rules apply to this task and how you'll apply them. If none do, say so. At the end of your answer, on a separate line: what came out differently in your answer than it would have without these rules.

一次会话结束时:

Write a note for the next conversation: 1) what's done; 2) what isn't done and why; 3) what we tried and rejected — with the reason; 4) which decisions are made and which are open; 5) where to start next time. Check item by item that none is missing.

下一个对话里,把这份记录粘进去,再加上:

Here's the note from last time. Read it in full, retell in three lines what we're doing, and name what in it might be out of date.

「它记不住」这事,最后发现是我自己缺了一套交接流程,不是模型本身的毛病。

4。它就是顺着我说

它顺着你的话点头,从不说「我不知道」,还一直问「要不要我……?」。原来的回答格式里根本没有留出提反对意见的位置。

Answer in this order: 1) restate how you understood the task; 2) name the strongest objection to my plan — the one a person who disagrees with me would make; 3) only then your proposal. Even if you agree, point 2 is
mandatory.
Before asking me anything, check: is the answer already in what I gave you? If yes, find it, decide yourself, and tell me what you decided and why. Only ask about things only I know: my goals, taste, constraints. And when you do ask, first propose your own option and name what could prove it wrong.

把反对意见写进格式之后,哪怕模型本来更想顺着你,它也会冒出来。临时小改动用这招就太重了,所以我只在做方案和做决策时用它。

这里会出问题的地方

我整个项目跑在大概四百条规则上,到这个量级,规则检查本身就成了一堵文字墙;要是只有两条规则,我肉眼过一遍就行。创意类工作往往没有标准流程。收据也会被伪造,通常是模型打不开文件却不说,所以我在派任务之前会先问一句「这个文件你确实能打开吗?」

它有时还是会偷工减料。区别在于,现在我有机会发现这一点了。

模型并没有在骗我。它只是在我搭好的这套系统里,挑了成本最低的做法。