> Shlegeris said he uses his AI agent all the time for basic system administration tasks that he doesn't remember how to do on his own, such as installing certain bits of software and configuring security settings.
Back in the day, I knew the phone numbers of all my friends and family off the top of my head.
After the advent of mobile phones, I’ve outsourced that part of my memory to my phone and now the only phone numbers I know are my wife’s and my own.
There is a real cost to outsourcing certain knowledge from your brain, but also a cost to putting it there in the first place.
One of the challenges of an AI future is going to be finding the balance between what to outsource and what to keep in your mind - otherwise knowledge of complex systems and how best to use and interact with them will atrophy.
We've already seen part of this with turn-by-turn GPS navigation. People enable it to go to the stores they've been to so many times already. I understand going some place for the first time, but every. single. time. just shows the vast majority of people are quite happy outsourcing the most basic skills. After all, if it means they can keep up with the Ks easier, then it's a great invention
I’ve lived in the same house for 17 years. I will still often use a map app to navigate home as it can know more about the traffic backups/delays than I can.
It’s not just for getting home, but for getting home as efficiently as possible without added stress.
It's interesting, because maps (all of them) will reliably toss me onto a far more convoluted path home vs staying on the interstate.
The former has ~15 traffic lights vs the latter ~2.
Imho, one of the most corrosive aspects of GPS has been mystifying navigation, due to over reliance on weird street connections. Versus the conceptually simply (if slightly longer) routes we used to take.
Unfortunately, with the net effect that people who only use GPS think the art of manual street navigation is impossibly complex.
True, although "back in the day" people used to memorize at what times during the day certain routes were busy, and they took alternative routes ("the back roads" in my area) to get around traffic that could be predicted.
We've outsourced that to an app, too.
It also has information on store closing times/dates (some stores are closed on random days of the week, or close early on others), unexpected detours (construction, previously announced road work), speed traps (crowdsourced), and more.
Some of it simply wasn't possible before the technology came along.
I suspect that outsourcing as much of our lives to others (i.e. the corporations and the governments they control) is exactly what they want. AI is just the next thing that happens to be extremely useful for that plan.
Who is they? there is no plan here, it’s just everyone making similar decisions when faced with a similar set of incentives and tools, the reason those corporations can make money, is because they add value to the people who use them, if they didn’t, it wouldn’t be a business.
I think it ironic that visionaries and optimists see AI as freeing humanity, where our reliance on it will make us subordinate to it and its owners.
100% this. When we outsource something to the extent that we're incapable of doing it ourselves, we place ourselves at the mercy of those who control it.
There is also a cost to future encoding of relevant information - I (roughly) recall an experiment with multiple rounds of lectures where participants took notes in a text document, and some were allowed to save the document while others weren't.
Those who could save had worse recall of the information, however they had better recall of information given in the next round without note taking. Suggests to me there are limits to retention/encoding in a given period, and offloading retention frees resources for future encoding in that period.
Also that study breaks are important :)
Anecdotally, I often feel that learning thing 'pushes another out', especially if the things are learnt closely together.
Similarly, I'm less likely to retain something if I know someone I'm with has that information - essentially indexing that information in social knowledge graphs.
Pros and cons.
I can still remember all my high school friends' phone numbers though. Just not the numbers of anyone I met in the 30 years since.
I can remember the phone number for the local Best Buy which I called a lot as a teenager to find out when new games came in stock.
His system instructions include this: "In general, if there's a way to continue without user assistance, just continue rather than asking the user something. Always include a bash command in your message unless you need to wait for the user to say something before you can continue at risk of causing inconvenience. E.g. you should ask before sending emails to people unless you were directly asked to, but you don't need to ask before installing software."
https://gist.github.com/bshlgrs/57323269dce828545a7edeafd9af...
So it just did what it was asked to do. Not sure which model. Would be interesting to see if o1-preview would have checked with the user at some point.
Wait, is that gist of the same session as is described in the article? I don’t see any escalation of privileges happening.
It just ran 'sudo'.
I saw that but here’s an alternative take on what happened:
While the session file definitely shows the AI agent using sudo, these commands were executed with the presumption that the user session already had sudo privileges. There is no indication that the agent escalated its privileges on its own; rather, it used existing permissions that the user (buck) already had access to.
The sudo usage here is consistent with executing commands that require elevated privileges, but it doesn’t demonstrate any unauthorized or unexpected privilege escalation or a self-promotion to sysadmin. It relied on the user’s permissions and would have required the user’s password if prompted.
So he sudo commands executed successfully without any visible prompt for a password, which suggests one of the following scenarios:
1. The session was started by a user with sudo privileges (buck), allowing the agent to run sudo commands without requiring additional authentication.
2. The password may have been provided earlier in the session (before the captured commands), and the session is still within the sudo timeout window, meaning no re-authentication was needed.
3. Or maybe the sudoers file on this system was configured to allow passwordless sudo for the user buck, making it unnecessary to re-enter the password (I just discovered this one, actually!).
In any case, the key point is that the session already had the required privileges to run these commands, and no evidence suggests that the AI agent autonomously escalated its privileges.
Is this take reasonable or am I really missing something big?
The article said it was claude
Dood, it's not "deciding" to do anything. It's autocompleting commands that statistically follow other commands. It might do anything.
This is reductive to the point of not being helpful, these models display actual intelligence and can work through sight-unseen problems that can't be solved by "get a bunch of text and calculate the most often used next word." I understand why people say this because they see knowledge fall off once it's something outside their training data but when provided the knowledge the reasoning capability stays.
These models don't have will which is why it can't decide anything.
These statistical models certainly don’t have will, they hardly have understanding, but they do a great job at emulating reasoning.
As we increase parameter sizes and increment on the architecture, they’re just going to get better as statistical models. If we decide to assign terms reserved for human beings, that’s more a reflection on the individual. Also they certainly can decide to do things, but so can my switch statements.
I’m going to admit that I get a little unnerved when people say these statistical models have “actual intelligence”. The counter is always along the lines of “if we can’t tell the difference, what is the difference?”
The models literally do just repeatedly calculate the most (or randomly not quite the most) often used next word. So those problems can in fact be solved by doing that, because that's how they're being solved.
It's really not though, you're confusing one teeny tiny part of the decoder with the entire decoder. Yes you sample from output probabilities but that's literally the least interesting bit. How you generate those probabilities is the hard part.
You do this! When you're speaking or writing you're going one word at a time just like the model, and just like the model your attention is moving between different things to form the thoughts. When you're writing you need at least a semi-complete thought in order for you figure out what word you should write next. The fact that it generates one word at a time is a red herring as to what's really going on.
How are you defining intelligence here?
Incorrect.
I thought they were stoichastic parrots that couldn't do anything outside their training data. Now they might do anything? I don't know what to believe, let me ask ChatGPT and have it tell me what to think!
You really do not know how to parse the phrase "might do anything" in context?
Well it's no crime to be so handicapped, but if it were me, that would not be something I went around advertizing.
Or you're not so challenged. You did know how to parse that perfectly common phrase, because you are not in fact a moron. Ok, then that means you are disingenuous instead, attempting to make a point that you know doesn't exist.
I mean I'm not claiming either one, you can choose.
Perhaps you should ask ChatGPT which one you should claim to be, hapless innocent idiot, or formidable intellect ass.
Always seems to me that the goalposts keep moving as capabilities of AI improves.
I swear to God we could have AGI+robotics capable of doing anything a human can do (but better) and we'll still - as a species - have multi-hour podcasts pondering and mostly concluding "yea, impressive, but they're not really intelligent. That's not what intelligence actually is."
They aren't and it isn't. So far it's all pure mechanism.
However, when people talk like this, it does make one wonder if the opposite isn't true. No AI has done more than what an mp3 player does, but apparently there are people who hear an mp3 player say "Hello neighbor!" and actually believe that it greeted them.
Isn't that what we all do to some degree?
I mean, sure I can do anything.
However i won’t because despite it not being in my training data, i recognize that blindly running updates could fuck my day up. This process isn’t a statistical model expressed through language, this is me making judgement calls based on what I know and more importantly, don’t know.
This sounds exactly like what I would have done at age 18 cluelessly searching the internet for advice while updating a fresh debian so I can run some random program.
I have done this ... and worse. Fun times.
My favorite was waiting 2 days to compile Gentoo then realizing I never included the network driver for my card. But also this was the only machine with internet access in the house.
Downloading network drivers through WAP on a flip phone ... let's say I never made that mistake again lol.
I nuked all my /dev devices on FreeBSD back in the day and had to figure out how to copy the right utilities from the secondary partition to the primary so I could remake them using mknod. You learn so much from such wonderful mistakes. Sometimes jamming a stick into your spokes is the best way.
I wonder if we'll see an AI Agent do a Crowdstrike-tier oops in our lifetime.
Who knows, it might even happen at Crowdstrike!
That seems inevitable with how we are going.
I wonder how many have already been pushed under the guise of human-agent (e.g., copy-paste from ChatGPT/CoPilot).
If it happens, the AI will be less responsible than the moron that gave it control over 300,000+ computers.
If statistically companies are making more money in the long run giving more control to AI... that is exactly what they will do it - and it will be rational for companies to do that. And the people not doing it will be labeled unmodern, falling behind, losing profits etc etc.
Me daydreaming about a future Cloudstrike caused by AI:
```
Why did this happen? You've cost our company millions with your AI service that we bought from you!
Err... dunno... not even the people who made the AI know. We will pay you your losses and then we can just move on?
No, we will stop using AI entirely after this!
No you wont. That will make you uncompetitive in the market and your whole business will lag behind and fail. You need AI.
We will only accept an AI model where we understand why it does what it does and it never makes a mistake like this again.
Unfortunately, that is a unicorn. Everyone just kinda accepts that this all-powerful thing we use for everything we kinda just don't fully understand. The positives far outweigh the few catastrophes like this though! It is a gamble we all take. You'd be a moron running a company destined to fail if you don't just have faith it'll mostly be ok like the rest of our customers!
*Groan* ok
```
I wrote a similar tool to help me do system tasks I couldn't be bothered to do myself:
openinterpreter is another popular choice
The agent is set to respond to the terminals output, it cannot stop / finish the task
This is about what I expect when I hear "AIOps". Something that operates So Hard... until it doesn't.
Something reduced to 'see/do' can and should be implemented in pid1
Does anyone have similar agentic code or know of any frameworks for accomplishing a similar task? I've been working on something like this as a side project. Thanks.
Looks pretty easy to find tons of examples of agentic AI engineer projects online:
- https://github.com/Pythagora-io/gpt-pilot - https://github.com/smol-ai/developer - https://github.com/stitionai/devika
If you want to just make games there's Rosebud AI
I cannot speak for the quality of any of these projects though
I wrote one: https://github.com/skorokithakis/sysaidmin
openinterpreter is. a popular one https://github.com/OpenInterpreter/open-interpreter
Maybe automating a task that you don't want to remember how to perform, would best be done by writing a script?
Always remember the rule of the lazy programmer:
1st time: do whatever is most expeditious
2nd time: do it the way you wished you'd done it the first time
3rd time: automate it!
In my reading of it, the article says that this was done as an experiment, not as a means to accomplish anything.
There's no logs or screenrecording/video provided, the whole article is based on an (email) discussion with the CTO. It's a plausible experiment obviously, but for all we know the events or "results" could be a complete fabrication (or not).
Brought to you by Redwood Research(TM).
AI has advanced to Joey Pardella levels, a.k.a. "knowing just enough to be dangerous".
Maybe it really is time to be scared...
> CEO at Redwood Research, a nonprofit that explores the risks posed by AI
CEO promoting himself on the Internet...
> No password was needed due to the use of SSH keys;
> the user buck was also a [passwordless] sudoer, granting the bot full access to the system.
> And he added that his agent's unexpected trashing of his desktop machine's boot sequence won't deter him from letting the software loose again.
... as an incompetent.
Hah, that's the outfit Yudkowsky endorsed. You think he's going to retract? Because almost anyone who knows system administration and LLMs would have told this guy it was a horrible idea.
Not sure why the downvotes...
he even admits it
>"I only had this problem because I was very reckless,"
guys makes an automated process and is surprised when his outsourcing his trust to the automated process.
Trust but verify mate. Computers are IO machines you put garbage in and you will get garbage out.
AI is not different, in fact its probably worse as its an aggregator of garbage.
The downvotes are because it's rude to call someone incompetent because they made mistakes. We have all done very stupid shit in our day.