this post was submitted on 23 Oct 2026
24 points (78.6% liked)
LocalLLaMA
5662 readers
180 users here now
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
If the kink you're referring to is the amount of time Qwen's "thinking" about your greeting, that's not necessarily actually a problem. I've seen common misconceptions about exactly this kind of scenario.
Models like these are trained to follow instructions. When they receive a user input they think "what is the user asking me to do, and how do I go about doing it?"
If all you've said is "Hi", that leaves them with a huge puzzle and no hints how to solve it. What does the user want them to do? Is this a test? Did they start writing an actual instruction and accidentally hit enter partway through? Do they want me to write a "hello world" app, and if so in what language? Is this the outout of a "hello world" app that I wrote in a previous round that I now need to check up on?
Qwen's models are known to be rather "thinky" in the first place - it's part of how they're able to hit so high above their weight class - so it's not surprising that it's taking its time to figure out this challenge you've presented it with.
Oh thank fuck. I thought I was going crazy... My first attempt at local models was a Qwen. A base model. I swear to god all I said was hello and the thing started and never stopped. I guess that's what a filibuster must be like? It was both amazing, yet terrifying. I've a bit more xp now, but yeah Qwen and really really thinky, dear gods ! My colleague got himself an abliterated one and just leave it to run overnight trying to pentest shit in his home network, but I guess I prefer something a bit more hands on, myself.
Heh. I use Qwen3.8-27B as my main coding agent, and I sometimes feel really bad for it as I watch the stream of thought go by filled with "wait... am I doing this right? What if I'm completely wrong about this? Is this really what I should be doing??" I want to pet it and assure it that it should have more confidence in itself. :)
But it's the tradeoff that Qwen made to cram that much smarts into such a small bundle of parameters, I guess.
Yeah, I can see that but it's horrendously slow, I find. I tried a 3.8 online and I just dont' like the way it works, overall :( It goes on and on and on and never stops to ask despite that being one of my main rules.
There's a fine-tuned version called Swift that's been trained to use 58.3% fewer reasoning tokens, apparently with almost no loss of quality. I've not been in a hurry with my personal usage (I set my coding agent loose on a task and then go have tea or whatever) so I haven't experimented with it much but it might be worth checking out if the verbosity of reasoning is a problem.
Qwen4 will be coming out soon too, it has a 27B version, so perhaps they'll have incorporated more efficient reasoning into it from the start. I'm looking forward to testing it out.
Good tip! I'll definitely be checking this out, as someone who also uses Qwen3.8 27B. The capability leap over the model I used before is definitely worthwhile, but wow does it overthink really easily.