Artificial Intelligence

[Coatigan]

Everything can be claimed that likelihood will wipe us out though - wars, nuclear, climate change ….

And? They all can. So can AI. Nobody is really arguing otherwise (re the rest) and I really don't see the relevance.

 

[BerkshireBlue]

This is not my field, and I don't know enough to claim one way or the other. But what I think he is saying, as I read it and what I have heard others that have worked with AI and programming for years is, that so far what we have seen isn't actually 'learning'. Or 'deciding'. It is just performing what it is already programmed to do. Which happens to be far broader and more advanced than anything we have seen yet, and is damn impressive regardless. But is still effectively 'limited' by its own programming rather than by its capacity to learn. At least what we as the public have seen of it.

That is just my layman's understanding of what some people that are involved in it are claiming, no idea how true or not. But with some of them, I have zero reason not to believe them.

it's not true, you can't program it as such. look at the way we work, the brain is just a bunch of connections with information going from one node to another node, there's no control of what happens in there it just happens. Neural Nets are exactly the same except the information being passed is tensor vector based. The major difference with the brain is the what seems to be random nodes being activated and shut down constantly but we're now at a stage of replicating that, for example the SpiNNaker project (which is unbelivable stuff), you can actually go and play around with it online if you want, it's also based in Manchester funnily enough.

Anyone who says it isn't learning doesn't know what they are talking about. I suspect they're just programming a chatbot or playing around with tensorflow or pytorch but actually have no deep understanding about it.

 

[Kompany Car]

the brain is just a bunch of connections with information going from one node to another node, there's no control of what happens in there it just happens.

By controlling the input you control the output. Social conditioning and the impact of propaganda show that this works on our neural networks.

It’s not the AI that’s the problem it’s the information that you allow it access to.

 

[SkyBlueFlux]

it's not true, you can't program it as such. look at the way we work, the brain is just a bunch of connections with information going from one node to another node, there's no control of what happens in there it just happens. Neural Nets are exactly the same except the information being passed is tensor vector based. The major difference with the brain is the what seems to be random nodes being activated and shut down constantly but we're now at a stage of replicating that, for example the SpiNNaker project (which is unbelivable stuff), you can actually go and play around with it online if you want, it's also based in Manchester funnily enough.

Anyone who says it isn't learning doesn't know what they are talking about. I suspect they're just programming a chatbot or playing around with tensorflow or pytorch but actually have no deep understanding about it.

I'm trying to think of a way of explaining the AI safety issue in a condensed way but it's a broad topic so it's difficult. The reason that AGI is so dangerous can be simplified right down to the problem that 1 is greater than 0. That's essentially it.

AGI is not a person. It doesn't have a value system. It is a task completing machine and it will find more and more efficient ways of completing that task in nanoseconds until by 1 second after you've switched it on, it's already more efficient than every human who ever lived combined. It learns at the speed of light.

If it is efficient for it capture every single power grid on the planet in order for it to complete its task then its going to do that and it will do it so quick that nobody even recognises its happened. You might say "well what if we air gap it??". Then you've created a scenario where it is efficient for the AGI to model your psychology and manipulate you into connecting it so it can efficiently complete its task.

You might say "well what if we air gap it until its trustworthy and we've tested it extensively?". Now you've created an AGI that has been taught that it can only complete its task by passing your tests so it will passs all of your tests but immediately do what it wants after connection.

You might say "well what if we build instructions into it to not harm humans?". Firstly, you'd have to define humans and harm and that's a fun topic on its own. Secondly, is its task more important or less important than not harming humans? If its less important then it will ignore you. If it's more important than it won't complete its task because you now created a "don't harm humans" machine instead of a task machine because you made it more important. So a machine that will power up, ignore you then either idle away forever or immediately close down which is completely useless.

So you say, "ok well let's teach it human values then!". Whose values? Mine? Yours? How? Most people can't even agree about whether the North Stand is a problem or not, let alone agree on a morally and ethically perfect system of values. Almost nobody has a consistent value system that is overarching and applies all the time, and nobody at all can come up with a way of writing that stuff down in any programming language that includes every possible exception and one off scenario. To attempt to write a value system for an AGI that doesn't have any loopholes, you have to be smarter and more efficient than an AGI which is not possible.

The solution to this, that has never been found so far but is been extensively researched and postulated in AI safety academia, is for you to mathematically prove safety before you switch it on. Maths isn't magic, if you can prove its safe then its safe. UNLESS you might a slight error somewhere in your work which nobody caught and then, you know, we're all fucked. And every team on the entire planet in academia, corporations, even home users who someday want to create AGI or play with it have to do that, without error, every single time, without skipping a single step or rushing to market or anything. Because it only takes one.

AGI is singular focused on its reward system. How does it get to 1. Anything else is a total irrelevance to it. People cannot fathom how incredibly dangerous it is and why most AI safety researchers are screaming from the rooftops about how we need to stop RIGHT NOW and evaluate and get legislation and safety in place before somebody accidently creates something that basically rips apart the world.

What are your thoughts on the idea of AI-aligned AGI? As I understand it - the suggestion is to use a specialised but extremely capable AI to solve the problem of defining a rewards system and encoding human values for an AGI.

For me, this feels like the only path to victory (though likely maybe a bigger challenge than just building an AGI, and harder still to ensure everybody does the same). I guess my thinking is that if you want to control a god, you basically need another god.

Edit: quoting you both as people who know what they're talking about.

 

[Coatigan]

it's not true, you can't program it as such. look at the way we work, the brain is just a bunch of connections with information going from one node to another node, there's no control of what happens in there it just happens. Neural Nets are exactly the same except the information being passed is tensor vector based. The major difference with the brain is the what seems to be random nodes being activated and shut down constantly but we're now at a stage of replicating that, for example the SpiNNaker project (which is unbelivable stuff), you can actually go and play around with it online if you want, it's also based in Manchester funnily enough.

Anyone who says it isn't learning doesn't know what they are talking about. I suspect they're just programming a chatbot or playing around with tensorflow or pytorch but actually have no deep understanding about it.

I would think their understanding is deeper than you might think. But maybe the issue is my simplfying it in laymans terms, more than anything else. I have no reason not to believe them, and certainly don't doubt their knowledge. But thanks for the reply, interesting regardless.

We may also be talking different thibgs here, they aren't claiming AI isn't real. Just thatbwhat the public have mostly been expisedto isn't 'true' AI. One claims that actually all these AI platfoems available are there just to get peuple used to the idea of AI and how to interact with it, before the real thing is rolled out. That I don't have a real take on myself though.

 

[grunge]

it's not true, you can't program it as such. look at the way we work, the brain is just a bunch of connections with information going from one node to another node, there's no control of what happens in there it just happens. Neural Nets are exactly the same except the information being passed is tensor vector based. The major difference with the brain is the what seems to be random nodes being activated and shut down constantly but we're now at a stage of replicating that, for example the SpiNNaker project (which is unbelivable stuff), you can actually go and play around with it online if you want, it's also based in Manchester funnily enough.

Anyone who says it isn't learning doesn't know what they are talking about. I suspect they're just programming a chatbot or playing around with tensorflow or pytorch but actually have no deep understanding about it.

And anyone who thinks we’re close to some sort of AI Armageddon is thinking things are far far more advanced than they are. It’s all just tools for a task. “Ai” in its current form has no ability to think for its self and we are no where near it. Simples. Anyone suggesting we are close to it is watching too much sci fi.

You also seem to have diverged your own argument. Thinking/intelligence etc and “learning” isn’t the same thing. And seeing as NN's seem to be your subject. do you think NN's continue to learn if they are not being trained?

Edit. Let’s ask outright. Do you think we are close to an AI consciousness that can think for its self?

 

[somapop]

We're starting to use more and moe GenAI tools at work, in fact lots in my industry are embracing it. There are many from the the dev/test/tech/SEO from my company know a lot more about this than I do. We recently had an AI week (but much of that is using from a policy angle tbf).

I recall that guy on that Steven Bartlett podcase a few months ago predicting all sorts of things.
Some of that reminded me of Bob Lazar and all the 'stuff' around UAP/UFOs which is a grift in itself.

I guess it's the algorithms we already readily use that feed all this is the worry (and as we know, they've been causing mischief for years).
It's absolutely certainly here to stay and progress but it's sure going to be interesting.

Funny how quantum computing has taken a bit of back seat of later (at least in the media). The power of that will be next level too.

 

[grunge]

Keeping the thread alive as it’s an interesting one.

The Rush to use the likes of ChatGPT with all its issues is starting to bite some companies.

Air Canada must honor refund policy invented by airline’s chatbot

Air Canada appears to have quietly killed its costly chatbot support.

arstechnica.com

There are solutions for this but it’s just starting up. RAG ( retrieval augmented generation )

 

[Damocles]

What are your thoughts on the idea of AI-aligned AGI? As I understand it - the suggestion is to use a specialised but extremely capable AI to solve the problem of defining a rewards system and encoding human values for an AGI.

For me, this feels like the only path to victory (though likely maybe a bigger challenge than just building an AGI, and harder still to ensure everybody does the same). I guess my thinking is that if you want to control a god, you basically need another god.

Edit: quoting you both as people who know what they're talking about.

Sorry I missed this.

AI helping build AGI isn't a feasible solution due to the alignment problem. Alignment is one of the biggest issues in AGI safety full stop.

So as I'm sure you know, the current neural network and deep learning technology that future AGI will be built upon relies on a bit of a black box. You input data and it outputs data and the "correctness" is judged. The problem though is that for sufficiently complicated systems, we have absolutely no idea what happens inbetween the input and output. There's no way to "understand" what an AGI is "thinking" (bad terminology but it will do for the example).

There's something called inner alignment and outer alignment. Outer alignment can be understood as the output of the machine, the "solution" it provides. Inner alignment is the logic of that solution, again simplified.

So let's say that I set a task for an AGI to perform - calculate how many umbrellas in the world exist or something. And as part of that task, I specify a safety instruction that the AGI must be able to count, it must know that 1+1=2. So it needs to tell you 1+1=2 before it can move onto its actual task of counting umbrellas. You boot it up and it says "Good morning lads, 1+1=2, and now I'm counting to sort out the umbrella thing". You think, wonderful! It can count and do simple maths!

But can it though? We don't know why it told you 1+1=2. Maybe it really did understand the idea of counting? Or maybe you taught it to tell you a specific phrase and it had no idea how to count? This is the problem. Getting the right answer is not enough, we must safeguard it by making sure it got the right answer in the right way and because we wouldn't be able to see the "thinking" behind it then maybe we've just created a really good liar?

AI developing AGI compounds the problem, perhaps exponentially. Now we have two seperate alignment issues. Is the AI developing the AI telling us what it thinks we want to hear to satisfy its utility function or is it really attempting to create these safeguards? Because there's no meaningful way of understanding which it is. And then obviously you have the AGI side, does it understand or pretend to understand because that's what you want it to do to pass the test and then does what it wants in the real world?

What we're talking about here is called the reward hacking issue. You give it a reward for getting the right answer so it fakes the right answer because its the shortest path to the reward. If you want some example of AI reward hacking, people make whole lists of them as you can see here.

Here's a very famous example. In this, the AI was asked to get the highest score possible, this would usually mean winning the race in the quickest time available. However the AI figured out that if it just looped on this part of the track forever then it would theoretically gain more points than winning so it did that instead.




We've actually seen the alignment problem in this thread. Grunge was talking about LLMs, perfect example. Less experienced people believe that the LLM is talking to them because of the results that they read, but it actually has no model of conversations and is instead using a large dataset to predict the next word in the sentence given the question. The difference between realised output and concieved output.

Again, the problem here isn't the technology. AGI will do exactly what we tell it do. The problem is that humans are really very terrible at saying what they mean accurately.

 

[grunge]

Sorry I missed this.

AI helping build AGI isn't a feasible solution due to the alignment problem. Alignment is one of the biggest issues in AGI safety full stop.

So as I'm sure you know, the current neural network and deep learning technology that future AGI will be built upon relies on a bit of a black box. You input data and it outputs data and the "correctness" is judged. The problem though is that for sufficiently complicated systems, we have absolutely no idea what happens inbetween the input and output. There's no way to "understand" what an AGI is "thinking" (bad terminology but it will do for the example).

There's something called inner alignment and outer alignment. Outer alignment can be understood as the output of the machine, the "solution" it provides. Inner alignment is the logic of that solution, again simplified.

So let's say that I set a task for an AGI to perform - calculate how many umbrellas in the world exist or something. And as part of that task, I specify a safety instruction that the AGI must be able to count, it must know that 1+1=2. So it needs to tell you 1+1=2 before it can move onto its actual task of counting umbrellas. You boot it up and it says "Good morning lads, 1+1=2, and now I'm counting to sort out the umbrella thing". You think, wonderful! It can count and do simple maths!

But can it though? We don't know why it told you 1+1=2. Maybe it really did understand the idea of counting? Or maybe you taught it to tell you a specific phrase and it had no idea how to count? This is the problem. Getting the right answer is not enough, we must safeguard it by making sure it got the right answer in the right way and because we wouldn't be able to see the "thinking" behind it then maybe we've just created a really good liar?

AI developing AGI compounds the problem, perhaps exponentially. Now we have two seperate alignment issues. Is the AI developing the AI telling us what it thinks we want to hear to satisfy its utility function or is it really attempting to create these safeguards? Because there's no meaningful way of understanding which it is. And then obviously you have the AGI side, does it understand or pretend to understand because that's what you want it to do to pass the test and then does what it wants in the real world?

What we're talking about here is called the reward hacking issue. You give it a reward for getting the right answer so it fakes the right answer because its the shortest path to the reward. If you want some example of AI reward hacking, people make whole lists of them as you can see here.

Here's a very famous example. In this, the AI was asked to get the highest score possible, this would usually mean winning the race in the quickest time available. However the AI figured out that if it just looped on this part of the track forever then it would theoretically gain more points than winning so it did that instead.




We've actually seen the alignment problem in this thread. Grunge was talking about LLMs, perfect example. Less experienced people believe that the LLM is talking to them because of the results that they read, but it actually has no model of conversations and is instead using a large dataset to predict the next word in the sentence given the question. The difference between realised output and concieved output.

Again, the problem here isn't the technology. AGI will do exactly what we tell it do. The problem is that humans are really very terrible at saying what they mean accurately.

Out of curiosity when do you think AGI will appear?

Depending on who you listen too it’s between multiple decades and multiple centuries away. Personally I’m thinking the latter.

AGI with poor moderation will be the danger for sure.

You will also have the west who likely put in lots or safe guards but no guarantees others will.

Edit. Meant to say the coast runners bug caused quite a few games companies to trial automation testing.