Rendered at 20:29:09 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
wongarsu 3 hours ago [-]
That writing style might be a tad too tense
If I got it correct (appending B from https://stolen-thoughts.com/paper.pdf is essential) they are the authors of the well-known exploit to recover readable CoT from OpenAI and Anthropic models. They use that to find hints of distillation, by running a benchmark with a SotA model, recovering the CoT, then taking the first 1% of the CoT and running the open-source model as if that was the start of its own CoT. In the paper they found that Kimi-K3 gets a lot closer to Claude 4.8 answers when prefilled with the start of Claude 4.8 reasoning, suggesting that Claude 4.8 was used in its post-training. This blog post is the follow-up with results that suggest that Qwen3.8 was post-trained with the help of GPT-5.5 Pro (or some similarly responding GPT model, it's unclear how many models they tested)
c7b 56 minutes ago [-]
I wasn't aware that we have access to raw reasoning tokens? I thought what you get is a kind of summary. Does the author have some kind of privileged access or was my assumption wrong?
But for the question studied here it probably doesn't matter - overlaps in the publicly available output may be indicative of distillation (or not), regardless of what it is. I would just find it surprising that the Chinese labs would use it so trustingly. The publicly released reasoning trace is the first place where I would suspect some distillation poisoning to be injected.
cristoperb 7 minutes ago [-]
They reference this paper which describes a method to decrypt reasoning traces (by sending the encrypted trace back to the model and asking it to transcribe it):
I wonder if that’s why 3.8 got so much better? Mixing the reasoning traces from both sides seems to be effective.
7734128 3 hours ago [-]
The problem with this is obviously that the only GPT 5.5 thoughts that we have access to are from stolen thought.
Qwen 3.8 0902 was trained after the release of the paper on August 10, so it should have seen those specific thoughts.
usernomdeguerre 2 hours ago [-]
seems like only the companies in question could run this sort analysis long-term; since they have full access to their CoTs not in public datasets.
verdverm 2 hours ago [-]
and we have to "trust them bro" to be fair and accurate, something I am very unlikely to do given their other false / misleading statements to date
sureMan6 2 hours ago [-]
And the only end result would be that the Chinese trained on their data just like OAI and Anthropic trained on our data so who cares
verdverm 1 hours ago [-]
capitalism ensures I get high marx on my Ai bill
refulgentis 2 hours ago [-]
The thoughts trick was known before their paper / August.
I "independently" "invented" it for the first Anthropic reasoning models because the API required you have thoughts for each assistant message. My app lets you switch AIs within a chat, and their API used to require thinking for all messages if thinking was enabled, so I needed to get a valid thinking stub to insert.
Time has flew by for me the last 3 years, but, I'd guess it's been at least 18 months. And IMHO it wasn't very complicated to work through how to do once you were dead set on making it happen. I expect it was well-known to distillers before the paper.
7734128 1 hours ago [-]
Sure, but TFA is trying to use Qwen's reaction to the thoughts as proof that they did indeed extract thoughts to train on.
My point is that any model trained after August 10 will know of those specific thoughts.
refulgentis 59 minutes ago [-]
I'm sorry, it's going over my head still - my reading is "all models with any training after August 10 know how GPT 5.5 Pro thinks", but I'm not sure why - my initial guess was that's when GPT 5.5 was released, but that doesn't seem to be the case (it was released April 23rd).
7734128 40 minutes ago [-]
They would know the specific thoughts released by the "stolen thought" paper, which became part of the public internet on August 10.
Unfortunately those are the only thought examples you can use to perform this experiment, as no other are availible.
But as the model should have seen those specific examples, it's not a good signal that Qwen was exfiltrating thinking traces.
hermitShell 38 minutes ago [-]
As a user of local models, does this mean that there are 'magic incantations' that can increase the performance of some local models?
I see some details about recovering information via whatever technique. It's interesting, but appears not generalized.
So for a specific question, yes, but this is not about techniques like adding a good embedding that just generally tends to improve open model performance on certain tasks.
c7b 30 minutes ago [-]
I don't think that follows from the published results. Would have been an interesting hypothesis to add though, and quite easy. Just throw the same setup at some benchmarks.
spijdar 1 hours ago [-]
It's interesting that someone else noticed this. A week or two ago, GPT-5.6 Sol starting leaking reasoning into a tool call in Pi. I don't really know what happened, but it was ... interesting:
Attach. Use hub debugger. Ensure source binary perhaps same. start. todo init. parallel no. two tool calls in same turn sequential is okay. immediately. exactly. Need not mention apologies yet final. [...] Let's do. [...] Do tools. Use commentary. Let's initiate. rambling no. use tool. searching now. okay. Really must call. Let's send. done. why stuck? generate. Sorry. go. no more. (The answer engine expects tool). [...] I think no hidden issue. Go. I'll type tool. now. Stop internal repetition. We have 8000 tokens. tool. sorry. I'll produce call. need include i. Great. final. no. Let's send.gpt. This may be bug. I'll consciously construct tool message next.
It eventually triggered some error state and stopped.
Nevertheless, this was the first time I'd seen Sol's CoT. I looked up the stolen thought's paper, aaaaand yep, that's Sol's CoT alright.
But it occurred to me, hey, Qwen3.8 27B's CoT seems ... very similar.
I compared the geometry problem in the paper, which had a reasoning block open with:
We need solve. Need reason geometry Weber point? Given pentagon sides and angles. Need find min sum distances. Likely construct rotations / Fermat point lower bound via vectors calibration, maybe triangulation.
I passed the same prompt to Qwen, which opened with:
We need solve geometry optimization. We need provide final answer. Let's analyze thoroughly.
This proves nothing, but it does seem an awful lot like they did use GTP-5.5/6 reasoning traces...
beacon294 46 minutes ago [-]
I've read an embarrassing amount of Qwen 3.8 27b cot and it's nothing like this. I'm not refuting the OP, though, which is about continuation.
spijdar 40 minutes ago [-]
This is Unsloth's UD-Q4_K_S quantization (edit -- on llama.cpp, via the Vulkan backend, on an RX 7900 XT, with Unsloth's recommended sampler config), for "as replicable as LLMs can be" disclosure, done through llama.cpp's web UI. In another thread, via Open WebUI (both with no custom prompt, so whatever the "default" system prompt is), a reasoning block opens as such:
We need answer user's request: write complete short story scene 2000-3000 words, third-person limited with Rowan, no outline preamble analysis. Need satisfy plot elements: raccoon, Milo returns, storm power failure, elderly customer knocks, thermos somewhere. End scene feels finished. Need produce only story. Need think in English (user English). Need craft 2000-3000 words. Need no meta. Let's plan.
Now, I will add that when I use Qwen3.8 in a coding harness, the reasoning tends to NOT look like this, and instead looks more Qwen3.6-ish.
But when I give short prompts to Qwen3.8 with no system prompt or previous instructions, it seems to strongly gravitate towards grug-talk.
stymaar 58 minutes ago [-]
> Qwen3.8 27B's CoT seems ... very similar.
What? I've never seen garbled CoT like the one you posted when using Qwen3.8-27B.
polotics 45 minutes ago [-]
I have seen plenty of Qwen 3.8 27B's caveman-like "Need doing this & that" thoughts. And on cerebras now I've seen them come real fast!
stymaar 1 minutes ago [-]
[flagged]
jari_mustonen 2 hours ago [-]
> Qwen barely moved toward Opus 4.8 in the earlier experiment, but moved by +20.58 points toward GPT-5.5 Pro here, including a large effect on the private synthetic puzzles. The data suggest that Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model, rather than from Opus.
If Qwen and other Chinese labs are just copying reasoning traces, then those labs are more than a year behind the frontier.
levocardia 49 minutes ago [-]
It's been pretty obvious to me that the Chinese labs are operating mostly on a fast-follow strategy. The distillation attacks are well-documented, and there is good reason to believe they are able to copy architectural innovations as well. If US labs stagnate I would expect Chinese labs to stagnate as well. Their engineering is great, but in terms of frontier innovation (which requires heavy compute to search for new strategies that work at frontier scale) they are very far behind.
atomicnumber3 1 hours ago [-]
It makes me happy because it means that these misanthropic technofascists have no moat. They can spend trillions of dollars only for it to be largely copied in short order.
Even if they weren't political adversaries of freedom, I would still feel 0% bad given all their training is already on data they got for free.
Information continues to want to be free. To the benefit of us all.
vipa123 1 hours ago [-]
Preach brother, they stole everything on the internet, and beyond, to train their models. They thought all that information was free, and everyone a few months beyond them is just following their example.
polotics 41 minutes ago [-]
They didn't actually steal in the sense that the information is still there on the internet.... About these shredded rare books, now we're talking.
If I may propose instead of "steal" I think we could agree to write they "Aaron-Swartz'ed" the information from the internet, what do you think, is this too harsh on Sam Altman or Carmen Ortiz ?
vipa123 30 minutes ago [-]
Is anything too harsh for these new robber barons?
Daishiman 48 minutes ago [-]
The way I see it the Chinese labs are optimizing for other things, including effective compact models that don't need to run on top-of-the-line nVidia hardware.
nater5000 37 minutes ago [-]
Well, yeah? I didn't think anybody seriously thought otherwise?
CamperBob2 2 hours ago [-]
News flash: people who scraped the Internet without permission to build their product complain when something vaguely similar is done to them. Water still wet, sky still blue. Film at 11.
(slibhb: Don't get me wrong, I agree with you 99%. But the frontier labs have zero moral authority here.)
slibhb 42 minutes ago [-]
I understand people just get off posting stuff like this. But creating LLMs from the entire corpus of human text was a huge achievement. Distilling those models is much less of an achievement. It means China is further behind than we thought.
1 hours ago [-]
noir_lord 2 hours ago [-]
I'd send them the worlds smallest violin but Rufus is getting in the way of me finding it.
UberFly 45 minutes ago [-]
I had to install an add-on in Waterfox to stop Rufus from following me around and interjecting every 2 minutes.
verdverm 2 hours ago [-]
I would not be surprised in the slightest if we later find out they are running those same open models to find useful traces or bits to incorporate into their own training. Lots of rules for thee but not for me from Big Ai
I look forward to a day when open models are so dominant that we stop considering traces to be some form of intellectual property that must be hidden from / manipulated for paying users.
It's that manipulation of inputs and outputs that really rubs me the wrong way
vezycash 1 hours ago [-]
They are copying useful parts of open models 100% especially from deepseek.
vlyan 2 hours ago [-]
not just the internet, but every commercially published written work in existence, and I doubt their highly publicized destructive scanning thing had managed to legitimize even a fraction of a percent.
this what is permissible for Jupiter is not permissible for a cow bullshit alone should tell people all they need to know about what kind of greasy sociopaths run "open"ai and (mis)anthropic, and how seriously you should take their purported stances on "safety" and other self-serving shit.
RivieraKid 1 hours ago [-]
For some reason, what China is doing seems worse. Part of it is that I want the US to stay ahead of China.
If I got it correct (appending B from https://stolen-thoughts.com/paper.pdf is essential) they are the authors of the well-known exploit to recover readable CoT from OpenAI and Anthropic models. They use that to find hints of distillation, by running a benchmark with a SotA model, recovering the CoT, then taking the first 1% of the CoT and running the open-source model as if that was the start of its own CoT. In the paper they found that Kimi-K3 gets a lot closer to Claude 4.8 answers when prefilled with the start of Claude 4.8 reasoning, suggesting that Claude 4.8 was used in its post-training. This blog post is the follow-up with results that suggest that Qwen3.8 was post-trained with the help of GPT-5.5 Pro (or some similarly responding GPT model, it's unclear how many models they tested)
But for the question studied here it probably doesn't matter - overlaps in the publicly available output may be indicative of distillation (or not), regardless of what it is. I would just find it surprising that the Chinese labs would use it so trustingly. The publicly released reasoning trace is the first place where I would suspect some distillation poisoning to be injected.
https://stolen-thoughts.com/paper.pdf
Qwen 3.8 0902 was trained after the release of the paper on August 10, so it should have seen those specific thoughts.
I "independently" "invented" it for the first Anthropic reasoning models because the API required you have thoughts for each assistant message. My app lets you switch AIs within a chat, and their API used to require thinking for all messages if thinking was enabled, so I needed to get a valid thinking stub to insert.
Time has flew by for me the last 3 years, but, I'd guess it's been at least 18 months. And IMHO it wasn't very complicated to work through how to do once you were dead set on making it happen. I expect it was well-known to distillers before the paper.
My point is that any model trained after August 10 will know of those specific thoughts.
Unfortunately those are the only thought examples you can use to perform this experiment, as no other are availible.
But as the model should have seen those specific examples, it's not a good signal that Qwen was exfiltrating thinking traces.
I see some details about recovering information via whatever technique. It's interesting, but appears not generalized.
So for a specific question, yes, but this is not about techniques like adding a good embedding that just generally tends to improve open model performance on certain tasks.
Nevertheless, this was the first time I'd seen Sol's CoT. I looked up the stolen thought's paper, aaaaand yep, that's Sol's CoT alright.
But it occurred to me, hey, Qwen3.8 27B's CoT seems ... very similar.
I compared the geometry problem in the paper, which had a reasoning block open with:
I passed the same prompt to Qwen, which opened with: This proves nothing, but it does seem an awful lot like they did use GTP-5.5/6 reasoning traces...But when I give short prompts to Qwen3.8 with no system prompt or previous instructions, it seems to strongly gravitate towards grug-talk.
What? I've never seen garbled CoT like the one you posted when using Qwen3.8-27B.
How does this suggest anyting of the sorts?
Score go up. Probability go up. Conclusion.
If Qwen and other Chinese labs are just copying reasoning traces, then those labs are more than a year behind the frontier.
Even if they weren't political adversaries of freedom, I would still feel 0% bad given all their training is already on data they got for free.
Information continues to want to be free. To the benefit of us all.
If I may propose instead of "steal" I think we could agree to write they "Aaron-Swartz'ed" the information from the internet, what do you think, is this too harsh on Sam Altman or Carmen Ortiz ?
(slibhb: Don't get me wrong, I agree with you 99%. But the frontier labs have zero moral authority here.)
I look forward to a day when open models are so dominant that we stop considering traces to be some form of intellectual property that must be hidden from / manipulated for paying users.
It's that manipulation of inputs and outputs that really rubs me the wrong way
this what is permissible for Jupiter is not permissible for a cow bullshit alone should tell people all they need to know about what kind of greasy sociopaths run "open"ai and (mis)anthropic, and how seriously you should take their purported stances on "safety" and other self-serving shit.