It’s conceivable that running at a lower wattage could crud up the coils more quickly, so charge it all the way. I hope this goes without saying, don’t make a disposable your main vape it’s obscenely wasteful

It really is, another thing I find remarkable is that all the magic vectors (features) were produced automatically without looking at the actual output of the model, only activations in a middle layer of the network, and using a loss function that is purely geometric in nature, it has no idea the meaning of the various features it is discovering.

And the fact that this works seems to confirm, or at least almost confirm, a non trivial fact about how transformers do what they do. I always like to point out that we know more about the workings of the human brain than we do about the neural networks we have ourselves created. Probably still true, but this makes me optimistic we’ll at least cross that very low bar in the near future.

The voice chat feature they show in the demo isn’t rolled out yet

I do my best not to kill them, but it’s rather a lot of effort and I just don’t see the practice going widespread, it’s hard enough to convince people to buy fewer tortured birds and mammals

That’s a perfectly reasonable position, the question of how complex a human brain is compared with the largest NNs is hard to answer but I think we can agree it’s a big gap. I happen to think we’ll get to AGI before we get to human brain complexity, parameter wise, but we’ll probably also need at least a couple architectural paradigms on top of transformers to compose one. Regardless, we don’t need to achieve AGI or even approach it for these things to become a lot more dangerous, and we have seen nothing but accelerating capability gains for more than a decade. I’m very strongly of the opinion that this trend will continue for at least another decade, there’s are just so many promising but unexplored avenues for progress. The lowest of the low hanging fruit has been, while lacking in nutrients, so delicious that we haven’t bothered to do much climbing.

Idk if we can ever see eye to eye here… if we were to somehow make major advances in scanning and computer hardware to the point where we could simulate everything that biologists currently consider relevant to neuron behavior and we used that to simulate a real person’s entire brain and body would you say that A) it wouldn’t work at all, the simulation would fail to capture anything about human behavior, B) it would partly work, the brain would do some brain like stuff but would fail to capture our full intelligence, C) it would capture human behaviors we can measure such as the ability to converse but it wouldn’t be conscious, or D) something else?

Personally I’m a hard core materialist and also believe the weak version of the church turing thesis, I’m quite strongly wedded to this opinion, so the idea that being made of one thing vs another or being informational vs material says anything about the nature of a mind is quite foreign. I’m aware that this isn’t shared by everyone but I do believe it’s the most common perspective inside the hard sciences, though not universal, Roger Penrose is a brilliant physicist who doesn’t see this way.

Huh? a human brain is a complex as fuck persistent feedback system

Every time-limited feedback system is entirely equivalent to a feed-forward system, similar to how you can unroll a for loop.

No see this is where we’re disagreeing… It is doing string manipulation which sometimes looks like maths.

String manipulation and computation are equivalent, do you think not just LLMs but computers themselves cannot in principal do what a brain does?

…you may as well say human reasoning is a side effect of quark bonding…

No because that has nothing to do with the issue at hand. Humans and LLMs and rocks all have this in common. What humans and LLMs do have in common is that they are a result of an optimization process and do things that weren’t specifically optimized for as side effects. LLMs probably don’t understand anything but certainly it would help them to predict the next token if they did understand, describing them as only token predictors doesn’t help us with the question of whether they have understanding.

…but that is not evidence that it’s doing the same task…

Again, I am not trying to argue that LLMs are like people or that they are intelligent or that they understand, I am not trying to give evidence of this. I’m trying to show that this reasoning (LLMs merely predict a distribution of next tokens -> LLMs don’t understand anything and therefore can’t do certain things) is completely invalid

The analogy is only there to point out the flaw in your thinking, the lack of persistence applies to both humans (if we shoot them quickly) and LLMs and so your argument applies in both cases. And I can do the very same trick to the clock analogy. You want to say that a clock is designed to keep time and that’s all it does therefore it can’t understand time. But I say, look, the clock was designed to keep time yes but that is far from all it does, it also transforms electrical energy into mechanical and uses it to swing around some arms at constant speed, and we can’t see the inside of the clock who knows what is going on in there, probably nothing that understands the concept of time but we’d have to look inside and see. LLMs were designed to predict the next token, they do actually do so, but clearly they can do more than that, for example they can solve high school level math problems they have never seen before and they can classify emails as being spam or not. Yes these are side effects of their ability to predict token sequences as human reasoning is a side effect of their ability to have lots of children. The essence of a task is not necessarily the essence of the tool designed specifically for that task.

If you believe LLMs are not complex enough to have understanding and you say that head on I won’t argue with you, but you’re claiming that their architecture doesn’t allow it even in theory then we have a very fundamental disagreement

Not the weights, the activations, these depend on the input and change every time you evaluate the model. They are not fed back into the next iteration, as is done in an RNN, so information doesn’t persist for very long, but it is very much persisted and chewed upon by the various layers as it propagates through the network.

I am not trying to claim that the current crop of LLMs understand in the sense that a human does, I agree they do not, but nothing you have said actually justifies that conclusion or places any constraints on the abilities of future LLMs. If you ask a human to read a joke and then immediately shoot them in the head before it’s been integrated into their long term memory they may or may not have understood the joke.

This is just a restatement of the second example argument I gave, trying to assert something about the internals of a model (it doesn’t understand) based on the fact that it was optimized to predict the next token

Every argument that refers to stochastic parrots is terrible. First off, people are stochastic, animals are stochastic, any sufficiently advanced AI is going to be stochastic, that part does no work. The real meat is in the parrot, parrots produce very dumb language that is mostly rote memorization, maybe a smidge of basic pattern matching thrown in, with little understanding of what they’re saying. Are LLMs like this? No.

Idk if I can really argue with people who think they’re so stupid as to be compared to a bird, I actually think they can be a bit clever, even exhibiting rare sparks of creativity, but this is just, like, my opinion after interacting with them a lot, other people have a different impression and I really think this is pretty subjective. I’ll grant that even the best of them can be really dumb sometimes, and I really don’t think it matters as this technology is in its infancy, unless we think they are necessarily dumb for some reason we will just have to wait to see how smart they will become. So we’re down to the rote memorization / basic pattern matching part. I’ve seen various arguments here. Pointing and waving at examples of LLMs seemingly using wrong patterns or regurgitating something almost verbatim found on the internet, but there are also many examples of them not obviously doing this. Then there’s claiming that because the loss function merely incentivizes the system to predict the next token that it therefore can’t produce anything intelligent but this just doesn’t follow. The loss function for humans merely incentivizes us to produce more offspring, just because it doesn’t directly incentivize intelligence doesn’t mean it won’t produce it as a side effect. And I’m sure more arguments, all of them are flawed…

…because the idea that LLMs are just big lookup tables with some basic pattern matching thrown in is, while plausible, demonstrably false. The internals of these models are really really hard to interrogate but it can be done if you know what you’re looking for. I think the clearest example of this would be in models trained on games of chess/othello, people have pointed out that some versions of chatgpt are kind of okay at chess but fail hard if weird moves are made in the opening, making illegal moves and not understanding what pieces are on the board, suggesting that they are just memorizing common moves and extracting basic patterns from a huge number of game histories. Probably this is to some extent true for ChatGpt 3.x, but version 4 does quite a bit better and LLMs specifically trained to mimic human games do better still, playing generally reasonably no matter what their opponent does. It could still technically be that they somehow pattern matching… better… but actually no, this question has been directly resolved. Even quite tiny LLMs trained on board game moves develop the ability to, at the very least, faithfully represent the board state, like you can just look inside at the activations the right way and see what piece is on each square. This result has been improved upon and also replicated with chess. What are they doing with that board state, how are they using it? Unknown, but if you’re building an accurate model of something not directly accessible to you using incidental data, you’re not just pattern matching. That’s just one example, and it’s never been proven, to my knowledge, that ChatGPT and the like do something like this, but it shows that it’s possible and does sometimes happen under natural conditions. Also, it would be kind of weird if a ~1 trillion parameter model was not at the very least taking advantage of something accessible to a 150 million parameter one, I’d expect it to be doing that plus a lot more clever and unexpected stuff.

CombineZP looks like it’s just for this

Eliezer has some horrible opinions and is possibly the most arrogant person alive, but he also wants to nationalize Nvidia and force them to stop turning out GPUs

This is not arrogance it’s playful language for humorous effect. The real reason to not eat fake meat is that it usually doesn’t taste very good no matter how you prepare it, whereas traditional vegan food can be amazing