In a sense, Adrian Thompson kicked this off in the 90's when he applied an evolutionary algorithm to FPGA hardware. Using a "survival of the fittest" approach, he taught a board to discern the difference between a 1kHz and 10KHz tone.
The final generation of the circuit was more compact than anything a human engineer would ever come up with (reducible to a mere 37 logic gates), and utilized all kinds of physical nuances specific to the chip it evolved on - including feedback loops, EMI effects between unconnected logic units, and (if I recall) operating transistors outside their saturation region.
Article: https://www.damninteresting.com/on-the-origin-of-circuits/
Paper: https://www.researchgate.net/publication/2737441_An_Evolved_...
Reddit: https://www.reddit.com/r/MachineLearning/comments/2t5ozk/wha...
Related. Others?
The origin of circuits (2007) - https://news.ycombinator.com/item?id=18099226 - Sept 2018 (25 comments)
On the Origin of Circuits: GA Exploits FPGA Batch to Solve Problem - https://news.ycombinator.com/item?id=17134600 - May 2018 (1 comment)
On the Origin of Circuits (2007) - https://news.ycombinator.com/item?id=9885558 - July 2015 (12 comments)
An evolved circuit, intrinsic in silicon, entwined with physics (1996) - https://news.ycombinator.com/item?id=8923902 - Jan 2015 (1 comment)
On the Origin of Circuits (2007) - https://news.ycombinator.com/item?id=8890167 - Jan 2015 (1 comment)
That's not a lot of discussion—we should have another thread about this sometime. If you want to submit it in (say) a week or two, email hn@ycombinator.com and we'll put it in the second-chance pool (https://news.ycombinator.com/pool, explained at https://news.ycombinator.com/item?id=26998308), so it will get a random placement on HN's front page.
Fascinating paper. Thanks for the ref.
Operating transistors outside the linear region (the saturated "on") on a billion+ scale is something that we as engineers and physicists haven't quite figured out, and I am hoping that this changes in future, especially with the advent of analog neuromorphic computing. The quadratic region (before the "on") is far more energy efficient and the non-linearity could actually help with computing, not unlike the activation function in an NN.
Of course, the modeling the nonlinear behavior is difficult. My prof would say for every coefficient in SPICE's transistor models, someone dedicated his entire PhD (and there are a lot of these coefficients!).
I haven't been in touch with the field since I moved up the stack (numerical analysis/ML) I would love to learn more if there has been recent progress in this field.
I believe neuromorphic spiking hardware will be the step to truly revolutionize the field of anthropod contagion issues.
Can’t tell if this is a joke or not
I came in already knowing what neuromorphic hardware is and I'm also unsure
I really wish I still had the link, but there used to be a website that listed a bunch of times in which machine learning was used (mostly via reinforcement learning) to teach a computer how to play a video game and it ended up using perverse strategies that no human would do. Like exploiting weird glitches (https://www.youtube.com/watch?v=meE5aaRJ0Zs shows this with Q*bert)
Closest I've found to the old list I used to go to is this: https://heystacks.com/doc/186/specification-gaming-examples-...
My favorite example was a game of pong with the goal of staying alive as long as possible. One ML algo just paused the game and left it like that.
Make no mistake most humans will exploit any glitches and bugs they can find for personal advantage in game. It’s just machines can exploit timing bugs better.
on youtube, codebullet remakes games so that he can try different AI techniques to beat them.
I remember this paper being discussed in the novel "Science of Discworld" -- a super interesting book involving collaboration between a fiction author and some real world scientists -- where the fictional characters in the novel discover our universe and its rules ... I always thought there was some deep insight to be had about the universe within this paper. Now moreso I think the unexpectedness says something instead about the nature of engineering and control and human mechanisms for understanding these sorts of systems ... -- sort of by definition human engineering relies on linearized approximations to characterize the effects being manipulated -- so something which operates in modes far outside those models is basically inscrutable. I think that's kind of expected but the results still provoke the fascination to ponder the solutions super human engineering methods might yet find with the modern technical substrates.
IIRC the flip-side was that it was hideously specific to a particular model and batch of hardware, because it relied on something that would otherwise be considered a manufacturing flaw.
Not even one batch. It was specific to that exact one chip it was evolved on. Trying to move it to another chip of the same model would produce unreliable results.
There is actually a whole lot of variance between individual silicon chips, even two chips right next to each other on the wafer will preform slightly differently. They will all meet the spec on the datasheet, but datasheets always specify ranges, not exact values.
long time ago, maybe in russian journal "Radio" ~198x, there was someone there describing that if one gets certain transistor from particular batch of particular factory/date, and connect it in whatever weird way, will make a full FM radio (or similar-complex-thing).. because they've wronged the yields. No idea how they had figured that out.
But mistakes aside, what would it be if the chips from the factory could learn / fine-tune how to work (better) , on the run..
AM radio can be "detected" with a semiconductor, so this kinda makes sense if you squint. If you can find it, someday, update this!
I remember talking about this with my friend and fellow EE grad Connor a few years ago. The chip's design really feels like a biological approach to electrical engineering, in the way that all of the layers we humans like to neatly organize our concepts into just get totally upended and messed with.
Biology also uses tons of redundancy and error correction that the generative algorithm approach lacks.
Though, the algorithm might plausibly evolve it if it were trained in a more hostile environment.
I read the damn interesting post back when it came out and seeing the title of the post immediately led me to thinking of Thompson's post as well.
“More compact than anything a human engineer would ever come up with” … sounds more like they built an artificial Steve Wozniak
That's exactly what I thought of too when I saw the title.
Basically brute force + gradient descent.
Relying on nuances of the abstraction and undefined or variable characteristics sounds like a very very bad idea to me.
The one thing you generally want for circuits is reproducibility.
And this is the kind of technology we use to decide if someone should get a loan, or if something is a human about to be run over by a car.
I think I'm going to simply climb up a tree and wait this one out.
What if it invented a new kind of human, or a different kind of running over?
Is this really so novel? Engineers have been using evolutionary algorithms to create antennas and other components since the early 2000s at least. I remember watching a FOSDEM presentation on an 'evolved' DSP for radios in the 2010s.
I don't believe it's comparable. Yes, we've used algorithms to find "weird shapes that work" for a long time, but they've always been very testable. AI is being used for more complex constructs that have exponentially-exponentially greater testable surface area (like programs and microarch).
This is really interesting and I’m surprised I’ve never even heard of it before.
Now I’m imagining antennas breeding and producing cute little baby antennas that (provided they’re healthy enough) survive to go on to produce more baby antennas with similar characteristics, and so on…
It’s a weird feeling to look at that NASA spacecraft antenna, knowing that it’s the product of an evolutionary process in the genuine, usual sense. It’s the closest we can get to looking at an alien. For now.
Two antennas get married. The wedding was ok but the reception was great!
Well done.
I strongly dislike when people say AI when they actually mean optimizer. Calling the product of an optimizer “AI” is more defensible, you optimized an MLP and now it writes poetry. Fine. Is the chip itself the AI here? That’s the product of the optimizer. Or is it the 200 lines of code that defines a reward and iterates the traces?
Yesterday I used a novel AI technology known as “llvm” to remove dead code paths from my compiled programs.
> known as “llvm” to remove dead code paths
Large Language Vulture Model?
Optimization is near and dear to my heart (see username), but I think it’s fine to call optimization processes AI because they are in the classical sense.
> I strongly dislike when people say AI when they actually mean optimizer.
It probably uses a relatively simple hill climbing algorithm, but I would agree that it could still be classified as machine learning. AI is just the new, hip term for ML.
If you want grant and VC dollars, you’ll rebrand things as “AI”.
Sigh, another day, another post I must copy paste my bookmarked Wikipedia entry for:
> "The AI effect" refers to a phenomenon where either the definition of AI or the concept of intelligence is adjusted to exclude capabilities that AI systems have mastered. This often manifests as tasks that AI can now perform successfully no longer being considered part of AI, or as the notion of intelligence itself being redefined to exclude AI achievements.[4][2][1] Edward Geist credits John McCarthy for coining the term "AI effect" to describe this phenomenon.[4]
> McCorduck calls it an "odd paradox" that "practical AI successes, computational programs that actually achieved intelligent behavior were soon assimilated into whatever application domain they were found to be useful in, and became silent partners alongside other problem-solving approaches, which left AI researchers to deal only with the 'failures', the tough nuts that couldn't yet be cracked."[5] It is an example of moving the goalposts.[6]
> Tesler's Theorem is:
> AI is whatever hasn't been done yet.
> — Larry Tesler
> It is an example of moving the goalposts.
On the contrary. The "AI effect" is an example of attempting to hold others to goalposts that they never agreed to in the first place.
Instead of saying "this is AI and if you don't agree then you're shifting the goalposts" instead try asking others "what future developments would you consider to be AI" and see what sort of answers you get.
We'll never build true AI, just reach some point where we prove humans aren't really all that intelligent either
The chip is not called AI-chip but rather AI-designed chip. At least in the title.
My point is that it’s equally ridiculous to call either AI. If our chip here is not the AI then the AI has to be the optimizer. By extension that means AdamW is more of an AI than ChatGPT.
I don't understand. I learnt about optimizers, and genetic algorithms in my AI courses. There are lots of different things we call AI, from classical AI (algorithms for discrete and continuous search, planning, sat, Bayesian stuff, decision trees, etc.) to more contemporary deep learning, transformers, genAI etc. AI is a very very broad category of topics.
Optimization can be a tool used in the creation of AI. I'm taking issue with people who say their optimizer is an AI. We don't need to personify every technology that can be used to automate complex tasks. All that does is further dilute an already overloaded term.
I agree that the article is wrong in using the wording “the AI”. However, firstly the original publication [0] doesn’t mention AI at all, only deep-learning models, and neither do any of the quotes in the article. Secondly, it is customary to categorize the technology resulting from AI research as AI — just not as “an AI”. The former does not imply any personification. You can have algorithms that exhibit intelligence without them constituting any kind of personal identity.
It's not "an AI", it's AI as in artificial intelligence, the study of making machines do things that humans do.
A fairly simple set of if statements is AI (an "expert system" specifically).
AI is _not_ just talking movie robots.
Who said it was 'an AI'? Do you understand what intelligence means? And what artificial means?
In game dev we've called a bunch of weighted If statements AI since the 80s. Sometimes they're not even weighted.
I think that's a bit different. The term is overloaded. There's "the machine is thinking" AI and then there's "this fairly primitive code controls an agent" AI. The former describes the technique while the latter describes the use case.
Clippy was an AI but he wasn't an AI.
Artificial intelligence, as others are using it here to cover a broad field of study or set of techniques. You seem to be objecting because the described product is not "an artificial intelligence", i.e. an artificial mind.
For some of us, your objection sounds as silly as if we were to tell some student they didn't use algebra, because what they wrote down isn't "an algebra".
You use optimization to train AI, but we usually refer to AI as being the parametrized function approximator that is optimized to fit the data, not the optimizer or loss function themselves.
This is "just" an optimizer being used in conjunction with a simulation, which we've been doing for a long, long time. It's cool, but it's not AI.
I don't understand what your gripe is. Both are AI. Even rudimentary decision trees are AI.
There's no function here that is analogous to a decision tree, or a parametrized model, just an optimizer and a loss function with a simulator. This isn't AI in the way it's commonly understood, which is the function that takes an input and produces a learned output.
The entire point of the thing is that it takes an input and produces an output. The output is the designed chip.
Lets just call every turning complete system AI and be done with it.
These are highly complicated pieces of equipment almost as complicated as living organisms.
ln some cases, they've been designed by other computers.
We don't know exactly how they work.
I've only started to look into the complexities involved in chip design (for my BitGrid hobby horse project) but I've noticed that in the Nature article, all of the discussion is based on simulation, not an actual chip.
Let's see how well that chip does if made by the fab. (I doubt they'd actually make it, likely there are a thousand design rule checks it would fail)
If you paid them to over-ride the rules at make it anyway, I'd like to see if it turned out to be anything other than a short-circuit from Power to Ground.
They do have some measurement results in figures 6 and 7. Looks like they didn't nail the center frequencies but at mmWave it's reasonable for a first attempt -- they're still missing something in their model though, same as if you did it by hand.
I'm skeptical that these pixelated structures are going to turn out anything better than the canonical shapes. They look cool but may just be "weird EM tricks", deconstructing what doesn't really need to be. Anyone remember the craze for fractal antennas?
> AI models have, within hours, created more efficient wireless chips through deep learning, but it is unclear how their 'randomly shaped' designs were produced.
IIRC this was also tried at NASA, they used some "classic" genetic algorithm to create the "perfect" antenna for some applications, and it looked unlike anything previously designed by engineers, but it outperformed the "normal" shapes. Cool to see deep learning applied to chip design as well.
Wasn't there an GA FPGA design to distinguish two tones that was so weird and specific not only did it use capacitance for part on its work but literally couldn't work on another chip of the same model?
Yes, indeed, although the exact reference escapes me for the moment.
What I found absolutely amazing when reading about this, is that this is exactly how I always imagined things in nature evolving.
Biology is mostly just messy physics where everything happens at the same time across many levels of time and space, and a complex system that has evolved naturally appears to always contain these super weird specific cross-functional hacks that somehow end up working super well towards some goal
> Yes, indeed, although the exact reference escapes me for the moment.
It's mentioned in a sister comment: https://www.damninteresting.com/on-the-origin-of-circuits/
I think it was that or a similar test where it would not even run on another part, just the single part it was evolved on.
Yes. The work of Adrian Thompson at the University of Sussex.
https://scholar.google.com/citations?user=5UOUU7MAAAAJ&hl=en
Actual Article: https://www.nature.com/articles/s41467-024-54178-1#Fig1
If we can't understand the designs, how rigorously can we really test them for correctness?
Our human designs strive to work in many environmental conditions. Many early AI designs, if iterated in the real world, would incorporate local physical conditions into their circuits. For example, that fluorescent lamp or fan I'm picking up(from the AI/evolutionary design algorithm's perspective) has great EM waves that could serve as a reliable clock source, eliminating the need for my own. Thus if you move things it would break.
I am sure there are analogous problems in the digital simulation domain. Without thorough oversight and testing through multiple power cycles, it's difficult to predict how well the circuit will function, and how incorporating feedback into the program will affect its direction, if not careful, causing the aforementioned strange problems.
Although the article mentions corrections to the designs, what may be truly needed is more constraints. The better we define these constraints, the more likely correctness will emerge on its own.
> Our human designs strive to work in many environmental conditions. Many early AI designs, if iterated in the real world, would incorporate local physical conditions into their circuits. For example, that fluorescent lamp or fan I'm picking up(from the AI/evolutionary design algorithm's perspective) has great EM waves that could serve as a reliable clock source, eliminating the need for my own. Thus if you move things it would break.
This problem may have a relatively simple fix: have two FPGAs – from different manufacturing lots, maybe even different models or brands – each in a different physical location, maybe even on different continents. If the AI or evolutionary algorithm has to evolve something that works on both FPGAs, it will naturally avoid purely local stuff which works on one and not the other, and produce a much more general solution.
Ask the same "AI" to create a machine readable proof of correctness. Or even better - start from an inefficient but known to be working system, and only let the "AI" apply correctness-preserving transformations.
I don’t think it’s that easy. I’m sure Intel, AMD and Apple have a very sophisticated suite of “known working systems” that they use to test their new chips, and they still build in bugs that security researchers find 5 years later. It’s impossible to test and verify such complex designs fully.
Especially true if the computer design creates a highly coupled device that could be process sensitive.
Results?
Can you always test the entire input space? Only for a few applications.
I am really curious about how you test software...
Literally no piece of software is bug-free. Not one. What are you talking about? Of course it’s impossible to test all inputs, because there’s going to be inputs that you can’t even convince of at the time of designing. What if your application suddenly runs at 1000000x the intended speed because hardware improves so much? How do you test for that?
Its inevitable: software (and other systems) will also become like this.
I've been using Cursor, it already is. I've found myself becoming merely a tester of the software rather than a writer of it, the more I use this IDE.
It’s a bit clunky still IMHO. Or you found a good tutorial to leverage it fully?
And then it's pretty much game over.
It’s better we [democracies] ride and control the AI change of paradigm than just let someone else do it for us.
"Democracy" is just a chant now. It's supposed to somehow happen without votes, privacy, freedom of expression, or freedom of association.
Well, Democracy is still the least worst of all political systems!
But please: would you prefer something else?
The point is there is no difference except spelling.
Maybe we’re all just in someone’s evolutionary chip designer
> The AI also considers each chip as a single artifact, rather than a collection of existing elements that need to be combined. This means that established chip design templates, the ones that no one understands but probably hide inefficiencies, are cast aside.
there should be a word for this process of making components efficiently work together, like 'optimization' for example
This is a strange distinction for the article to point out. If you want to take a more modular approach all you have to do is modify the loss function to account for that. It's entirely arbitrary.
"In particular, many of the designs produced by the algorithm did not work"
Judging by the code it outputs we need to because I have to constantly fix most code LLMs output.
Vast chunks of engineering are going to be devalued in the next 10-15 years, across all disciplines. It's already enabling enormous productivity gains in software, and there's zero reason this can't translate to other areas. I don't see any barrier to transformers being able to write code-cad for a crankshaft or a compressor, for example, other than the fact that so far they haven't been trained to do so. Given the extent to which every industry uses software for design, there's nothing to really stop the creation of wrappers and the automation of those tasks. In fact, proprietary kernels aren't even a barrier, because the gains in productivity make building a competitor easier than ever before.
now we are talking! next level for sure.