This is embarrassing. I would say Hopfield networks aren't even very revolutionary in neuroscience, but they're so old I can't tell. In terms of AI... they've been irrelevant for thirty years. I guess you could argue a transformer is a generalized Hopfield network, but of course that's a post-hoc understanding. None of this has anything to do with physics.
So what if an energy function lets you approximate the number of macro-states it can capture? Should every mathematics paper with Lagrange multipliers be put up for nomination? Every poll that uses the law of large numbers, and thus, entropy? Surely the computer scientists building the internet need to be included as well, since their work is based in information theory.
Or maybe, hear me out, we reserve the Nobel Prize in physics for advances in the physical sciences, understanding physical reality or how to bend it to our will.
Neither Hopfield networks nor Restricted Boltzmann Machines/Deep Belief Networks really panned out for any purposes outside some theoretical niches.
The prize was awarded for "AI" and the tenous links to physics of some irrelevant models are just an excuse.
I agree this is rather bizarre.
But let's not forget that the brain is a physical system and that neural networks are part of the reason we understand the brain as well as we do.
There was a long period where people like Chomsky thought the brain couldn't learn fast enough and that knowledge had to be innate.
> neural networks are part of the reason we understand the brain as well as we do
We don’t understand much then.
This is the only plausible reason:
“These artificial neural networks have been used to advance research across physics topics as diverse as particle physics, materials science, and astrophysics,” Ellen Moons, chair of the Nobel Committee for Physics, said at a press conference this morning.
Must be, because in the field of artificial intelligence, if these techniques are not in production and considered obsoletes it's for a good reason.
It may have been state-of-the-art in 1980s, but now is a bit late.
Very smart people in their time though.
In current times, a global prize to the transformers folks at least make more sense considering the context (despite it not being Physics).
The goal here is to attribute a very important area in contemporary technology to physicists. This prize advances physics in terms of giving it higher importance in the minds of lay people and journalists.
Had they wanted a good ML relevant physics Nobel, the committee had decades to award a prize to Marshall and Arianna Rosenbluth for the Markov Chain Monte Carlo method. Would have been self-evidently important and relevant to both physics and ML. Too late now -- Arianna died in 2020.
Curious, would Peter Shor qualify? I struggle if his work is really just CS or enough physics to be in the discussion.
There were some predictions that Peter Shor could win this year for quantum computation. I'd say his work is a lot closer to physics than Hinton's or Hopfield's.
As a physicist, my reaction to this is how bizarre is that. Maybe he deserves a nobel prize but in physics?
Also arguing that NN is used in physics so we can argue nobel prize is okay is like asking for Stephan Wolfram to be awarded Nobel prize for Mathematica which is much more used in physics as a tool. And he is a physicist and had contributions to the field of numerical relativity (The reason he created Mathematica in the first place).
The royal science academy fucked up so much with this choice.
By this definition Claude Shannon (the father of Information Theory) clearly deserves a Nobel in Physics. The central concept in Information Theory is Entropy which is defined literally the same way as in Physics. And Shannon's Information Theory clearly revolutionized our life (tele-communications) much more than Hopfield network or Hinton's Boltzmann machine.
Fun historical fact: Claude Shannon did win the Noble prize
That's different local prize by the American society of civil engineers.
Yes. That's the joke.
In 1939 Claude Shannon won the "wrong" Nobel prize -- The Alfred Noble Prize award presented by the American Society of Civil Engineers [0]. It causes a lot of confusion. Claude Shannon never won a "real Nobel".
[0] https://en.wikipedia.org/wiki/Alfred_Noble_Prize
EDIT: typos
It feels a bit like the field of physics claiming the invention of AI, where we all know that mathematics and/or CS deserve the honor.
Someone changed the Wikipedia article today to call Hopfield a "physicist". Previously the article called him simply a scientist, because his main work wasn't limited to physics. I changed it back now, let's see if it holds up.
>where we all know that mathematics and/or CS deserve the honor
Or semiconductor manufacturers.
All the math and CS needed for AI can fit on a napkin, and had been known for 200+ years. It's the extreme scaling enabled by semiconductor science that really makes the difference.
That's absurd. The computer science needed for AI has not been known for 200 years. For example, transformers were only invented in 2017, diffusion models in 2015.
(When the required math was invented is a different question, but I doubt all of it was known 200 years ago.)
TBF backpropagation was introduced only in the 1970's, although in hindsight it's a quite trivial application of the chain rule.
There were also plenty of "hacks" involved to make the networks scale such as dropout regularization, batch normalization, semi-linear activation functions (e.g. ReLU) and adaptive stochastic gradient descent methods.
The maths for basic NNs is really simple but the practice of them is really messy.
Residual connections are also worth mentioning as an extremely ubiquitous adaptation, one will be hard-pressed to find a modern architecture that doesn't use those at least to some extent, to the point where the original Resnet paper sits at over 200k citations according to google scholar[1].
[1] https://scholar.google.com/citations?view_op=view_citation&h...
> All the math and CS needed for AI can fit on a napkin, and had been known for 200+ years.
This isn't really true. If you read a physics textbook from the early 1900s, they didn't really have multivariate calculus and linear algebra expressed as concisely as we do now. It would take several napkins. Plus, statistical mechanics was quite rudimentary, which is important for probability theory.
I don’t think calculus existed at sufficient rigor 200 years ago.
Computer science wasn’t even a thing 100 years ago.
Calculus has been around for quite some time.
If Newton had the machinery to fit large models to data, he would have done so. No doubt.
Cauchy’s main work was under 200 years ago; and there’s been quite a lot of work since.
Again, I’m unsure that calculus existed at sufficient level 200 years ago — it didn’t appear in modern form from either Leibniz or Newton.
The Nobel Committee doesn’t represent the field of physics. I talked to a few former colleagues (theoretical physicists) just now and every one of them found this bizarre.
I think the Nobel prize doesn't want any scientific advance to fall outside the range of awards entirely.
> Maybe he deserves a nobel prize but in physics?
Which category fits better?
Right; the Nobel Committee has officially "jumped the shark".
Reminds me of the old classic Physics and Politics by Walter Bagehot.
I don't know if this is a travesty that they awarded the prize to work on non-physical systems to jump on a bandwagon or that there was nothing else obvious enough to the board in actual physics to give this to.
If I was the awardee I'd consider declining just out of respect to the field.
The linked document connects their work to physics as follows:
"The Hopfeld network utilises physics that describes a material’s characteristics due to its atomic spin – a property that makes each atom a tiny magnet. The network as a whole is described in a manner equivalent to the energy in the spin system found in physics, and is trained by fnding values for the connections between the nodes so that the saved images have low energy"
"Hinton used tools from statistical physics, the science of systems built from many similar components."
Then it should be awarded for the result.
Given the line of reasoning this has now opened up Alan Turing should be awarded one posthumously in every field.
He unironically should
100% agreed as I can't think of any one individual since(1) who has done as much for all of science and engineering as he ultimately did; alas, they are not awarded posthumously.
(1) Newton would be a strong contender on a "for all time" basis, but even he would've probably needed to share it with Leibniz, which would have driven him absolutely ~b o n k e r s~, like wet hornet in a hot outhouse mad, LOL.
Neither of these models were never really influential though beyond some theoretical niches.
Yup. This prize is indistinguishable from "we saw AI in all the headlines, let's see what's the most plausible reasons we could hand it out for that"
It seems like a hallucination generated from a ChatGPT prompt...
I think this is the Royal Academy of Sciences way to admit that Physics as a research subject has ground to a halt. String theory suffocated theoretical high energy physics for nearly half a century with nothing to show for it, and a lot of other areas of fundamental physics are kind of done.
I think this is (very) inaccurate. It feels more like them trying to jump on a "hot topic" bandwagon (machine learning/AI hype is huge).
Physics as a discipline hasn't really stalled at all. Fundamental physics arguably has, because no one really has any idea how to get close to making experimental tests that would distinguish the competing ideas. But even in fundamental physics there are cool developments like the stuff from Jonathan Oppenheim and collaborators in the last couple of years.
That said "physics" != "fundamental physics" and physics of composite systems ranging from correlated electron systems, and condensed matter through to galaxies and cosmology is very far from dead.
> trying to jump on a "hot topic" bandwagon
I don't know exactly what they hope to gain by jumping on that bandwagon though; neither the physicists nor the computer scientists are going to value this at all. And dare I say, the general populace associated with the two fields isn't going to either - case in point, this hn post.
If there weren't any noble-worthy nominations for physics, maybe skip it? (Although that hasn't happened since 1972 across any field)
One possibility is that they think this will access hype funding. Put "AI" in a physics paper and watch the grants roll in.
It really has not, though. There is more to physics than high-energy and cosmology, and there is no shortage of deserving contributions of smaller scope. It really is bizarre that deep learning would make it to the top of the list.
Could you give me some examples of areas of fundamental physics that are vital and have done some significant discoveries lately? I genuinely would like to know, because I really can't think of any.
I'm probably not the right person to ask, but off the top of my head: superconductivity of high-pressure hydrides; various quantum stuff like quantum computing, quantum cryptography, quantum photonics, quantum thermodynamics; topological phases; rare decays (double beta, etc.); new discoveries in cosmic rays, etc.
My point was that physics is a big and active field, stagnation at the smallest and largest scales notwithstanding. Note also that the Nobel committee is not in any way limited to "newsworthy" stuff and has in many cases awarded prizes decades after the fact.
"Vital" is completely subjective but I'd throw stuff around quantum information into the ring. Maybe you'd consider the loop-hole free Bell tests performed in 2015 and awarded the 2022 Nobel prize to count?
I think the prize in 2022 was a nice prize, but it could still be considering just tidying the corners. In the end it just proved that things really work as most of us has thought it worked for decades.
What if the next breakthrough is complex and not directly accessible from our current state of math/physics thought?
I see no reasons to expect steady progress. Nobody knows how long it would take to prove Riemann hypothesis, for example.
> Physics as a research subject has ground to a halt
Max Planck was told by his professor to not go into Physics because "almost everything is already discovered". Planck said he didn't want to discover anything, just learn the fundamentals.
String theory and the foundations are not the only area of physics. It would be nice for theorists to remember that.
My sense is that we might have reached the limits of what we can do in high-energy or fundamental physics without accessing energy levels or other extreme states that we currently can't access as they are beyond our capacity to generate.
From what I've read (not a professional physicist) string theory is not testable unless we can either examine a black hole or create particle accelerators the size of the Moon's orbit (at least). Many other proposed theories are similar.
There is some speculation that the hypothetical planet nine -- a 1-5 Earth mass planet predicted in the far outer solar system on the basis of the orbits of comets and Kuiper Belt / TNO objects -- could be a primordial black hole captured by the solar system. A black hole of that mass would be about the size of a marble to a golf ball, but would have 1-5g gravity at the distance of Earth's radius.
If such an object did exist it would be within space probe range, which would mean we could examine a black hole. That might get us un-stuck.
If we can't do something like that, maybe we should instead focus on other areas of physics that we can access and that have immense practical applications: superconductivity, condensed matter physics, plasmas / fusion, etc.
Even Sheldon Cooper stopped researching string theory at one point.
What bizarre choice. Hinton’s work has nothing to do with physics.
Nobel prize jumping on the bandwagon, just like they did for mRNA after covid. At least that was related to medicine.
The first 2 paragraphs of the linked pdf read like a joke. Like it’s a parody announcement.
Hinton did introduce the Boltzmann machine networks that kind of resemble thermodynamics if you squint really hard. And those didn't really pan out.
Of course the price is really for MLP backpropagation (or how it found applications) but I guess it's not physics enough.
Hopfield networks never really found use either, but they are sort of related to Ising models and NNs, so I guess it's physics then.
At least they already recognized actual spin-glasses research in physics in 2021. That makes this award for this by-product more puzzling though.
Or Obama’s peace prize.
Boltzmann machine does quote physics as its inspiration, they didn't award him for his work in BP and stuff.
But I agree, this feels like a stretch.
This is totally bizarre, no precedent for it really. The reality of the prize means that less and less are the winners names every physicist has heard of, but even today they're still big names in each subfield. For e.g. Kosterlitz, Thouless and Haldane weren't exactly household names but they really deserved the prize in 2016.
In this case, there's a good argument that Hopfield had conducted strong work as a physicist and in physics, but Geoffrey Hinton has never worked as a Physicist, at best adopting some existing things from physics into cognitive science use cases. In any case, what they've been given the prize for is work where they've not contributed to the understanding of the world of physics - it's not even really an arguable case where this is work that crosses over between Physics and another field either. It'd be like if Black or Scholes had been given the Physics prize rather than Economics because their famous equation can be re-written in Schrodinger equation form.
This does indeed smell of desperation. Which is really, really sad. Advances in _real_ physics are central to the absolutely needed sustainability transition. In a sane society that values its self-preservation you would not need to grasp at second-order straws to justify the need for all sorts of both fundamental and applied physics research.
We need to think seriously whether our collective hallucinations (pun) have got us to some sort of tipping point, undermining our very ability to act according to our best long-term interests.
ps. not to imply anything negative about the worthiness of the awardees in general
For what it's worth, the "Advanced information" PDF does a somewhat better job of trying to explain the rationale than the linked press release:
https://www.nobelprize.org/uploads/2024/09/advanced-physicsp...
Interesting. LeCun, Bengio, Schmidhuber and Hochreiter are (amongst others) also mentioned in this article.
Eagerly waiting for Schmidhuber's comment on the prize.
May be I should know better, but is there no Nobel category for computer science or mathematics? This isn't physics, this is absolutely embarrassing. May be all those bitter elder physicists who didn't get a prize can feel a little justified in their derision of the institution.
There no Nobel category for computer science or mathematics.
Related reading
https://hsm.stackexchange.com/questions/24/why-isnt-there-a-...
Computer science has the Turing award and mathematics the Fields medal. Neither is exactly equivalent to the Nobel but they're similar levels of prestige.
The Nobel prize fields and criteria are a bit random, they're essentially just whatever Alfred Nobel wrote in his will.
How about a "Nobel Memorial Prize in Computer Sciences" similar to the https://en.m.wikipedia.org/wiki/Nobel_Memorial_Prize_in_Econ...
Next year, the creators of Excel will get the prize, because it is an implementation of the mathematical universe.
You know what, I can support this for its predecessor, Lotus123. If ANNs are worthy of the prize in physics, then so is this
Physicist here. I'm sure Hinton deserves some sort of prize, but in Physics is really bizarre.
He did get the Turing award already in 2018.
It’s time to consider adding computer science as a category for the Nobel Prize. They have already been awarded prizes for economic sciences and peace, so why not computer science? It's not the same as physics, and its impact on modern life is undeniable
How about a "Nobel Memorial Prize in Computer Sciences" similar to the https://en.m.wikipedia.org/wiki/Nobel_Memorial_Prize_in_Econ...
> They have already been awarded prizes for [..] peace
Since 1901.
Why though? The Turing award exists.
Is that even "possible"? As in, they have to follow the rules of the organization, which have some criteria laid out. Not sure who could stop them from changing, though. Like, I think the original intent was to have done the most the preceding year, but now it's more of a lifetime award. So perhaps they can change or add different categories if wanted.
In some sense it makes sense to award AI researchers on behalf of the physics community because I know many physics PhD’s who thank their job to AI; they work as data scientist now.
Jokes aside, physics is a bit stuck because it’s hard to do experiments at the boundaries of what we know, as far as I understand. So then it makes sense I guess to award people who made useful tools for physics.
> it’s hard to do experiments at the boundaries of what we know
this applies primarily to fundamental physics. There are many areas of applied physics (materials, fusion, biophysics, atmospheric physics, etc. etc.) where the main constrain is understanding complex systems. These areas are quite crucial for society.
As evidenced by the search for room temperature low pressure superconductors.
Moreso genuine curiosity than as a gotcha: A lot of comments are saying this was the wrong choice. I'd find it really interesting to hear who the nomination should have gone to instead, in your opinions.
This reminded me of my Hopfield networks implementation in Go [1]. The algorithm is rather simple but fascinating nevertheless and works surprisingly well for reconstructing noisy images. I actually blogged about it as well [2]. But as many are discussing here Deep Memory networks based on Boltzmann networks are more powerful yet they don't seem to have found much use case either
[1] https://github.com/milosgajdos/gopfield [2] https://cybernetist.com/2017/03/12/hopfield-networks-in-go/
My perspective as a PhD in theoretical physics, who's been doing deep learning in the last 4 years:
1. The prize itself makes zero sense as a prize in _physics_. Even the official announcement by the Nobel Prize Committee, taken at a face value, reads as a huge stretch in trying to link neural networks to physics. When one starts asking questions about the real impact on physics and whether the most important works of Hinton and Hopfield were really informed by physics (which is a dubious link to the Nobel prize anyway), the argument stops holding water at all.
2. Some of the comments mention that giving prize for works in AI may make sense, because physics is currently stalled. This is wrong for several reasons: 2.1. While one can argue that string theory (which is, anyway, only a part of high-energy theoretical physics) is having its "AI winter" moment, there are many other areas of physics which develop really fast and bring exciting results. 2.2. The Nobel Prize is often awarded with quite some delay, so there are many very impactful works from 80s which haven't been awarded with a Nobel prize (atomic force microscopy is a nice example). 2.3. It is wrong to look at the recent results in some sub-field and say "okay, there was nothing of value in this field". For example, even if one completely discards string theory as bogus, there were many important results in theoretical physics such as creation of conformal field theory, which was never recognized with a Nobel Prize (which is OK if Nobel Prize is given to other important physical works, but is quite strange in the light of today's announcement).
To finish on a lighter mood, I'll quote a joke from my friend, who stayed in physics: "Apparently the committee has looked at all the physicists who left academia and decided that anything they do is fair game. We should maybe expect they will give a prize for crypto or high-frequency trading some time later".
I'd rather they withhold the prize this year, if there isn't really anything interesting happening in physics.
This must piss Schmidhuber off like nothing else.
A very long and detailed blog post incoming. Spoiler alert: the Nobel prize should have been awarded to Gödel/Hilbert (and Schmidhuber).
To the point of nuclear fusion even.
What an odd rationale. Please do not devalue the Physics Nobel Prize to the status of the Nobel Peace Prize.
This is overtly driven by expediency or business interests and ignores all societal problems.
Be happy that you get your billions for CERN and keep a low profile.
this is a painfully accurate description of how we feel we are treated(CERN physicists that is) are you an ex colleague or something :)
Maybe the story here isn't so much whether Hinton is a physicist or not, but rather a lack of groundbreaking progress in physics to give the Nobel to.
This is not physics. Does that mean there was no Nobel Price worthy research happening in the field of physics?
The contrast in discoveries made in 'core' physics in the first 25 years of the last centuries compared to this century is quite insane, it was never going to be sustainable. If it did sustain we would be colonising a new galaxy by now.
Consider that in 1900 the atom wasn't discovered yet, within around 25 years the basic principles of quantum physics were established, to say nothing about discoveries in cosmology (GR + big bang).
Chemists everywhere are wondering where this furor has been for the forty years or so.
Physicists stole the chemistry prize, now computer scientists are stealing the physics prize, I guess
Has anyone else won a Turing award as well as the Nobel prize in physics?
Baffling. If this isn't a sign of a major crisis in physics, then I don't know what is.
Real physics and its resulting breakthrough technologies have been hidden from society for a very long time. And so they simply need somebody they can give that price to.
Physics Nobel prize now also covers computer science, I guess.
Meh, neuroscience.
Peak AI Hype! Winter is coming!
Since most papers nowadays are written by AI, and peer reviewed by AI, it only seemed logical for AI to be used by the Nobel committee to award the godfather of AI.
Thanks to AI, you now only have to to ask any GPT for the source code of the universe to get the code. Since physics is now a solved problem, we should recenter ourselves on more important questions like why did AI create the universe ?
Hopefully AI will have an answer soon.
I wonder if this'll help physics PhDs get industry jobs (I hear Myron Scholes is up for the 2025 prize)
You mean like the ones working on Wall Street?
If you google who Myron Scholes is you'll see the person you replied to made the same assertion already.
Turing award and now Nobel prize in physics, something which he would have never expected. What an amazing career!
In physics...?
Waiting for Sabine's comment: "Told you so, physics is dead"
"
And the 2024 Nobel Prize in Physics does not go to physics...
"
Naturally she's not happy https://x.com/skdh/status/1843592351736050053
Honestly, I am stunned by today's Nobel committee announcement. Hinton's Boltzmann machine is a clever construct that nobody, repeat nobody, in the AI and ML is using anymore in actual practice.
EDIT: add minor clarification.
Imagine how Schmidhuber's feeling right now.
I'm sure he writes an article pointing all the errors in the official justification for the prize.
That was my almost immediate reaction to the headline! Rarely credited but rarely forgotten.
Alas, poor Jurgen! I knew him, Horatio. A fellow of infinite jest, of most excellent fancy…
haha!
As a former physicist (PhD in Experimental condensed matter physics), I have to ask WTF?
I get the importance of AI development, but Physics?
I'm equally as confused; huge WTF moment. Seems like a paradigm breakthrough, in that Nobel Prizes can be given for discoveries in tangential fields. Or perhaps it's due to Dr. Hopfields physicist status, that all his discoveries are considered physics related? Or that NNs are considered a part of physics / nature?
Yeah I was wondering the same, I get that this was fundamental work, but how was this strongly tied to physics?
Terrible. Hopfield networks nobody cares about, and few heard of?
Maybe just scrap the categories and have set or prizes for whatever sort of advanced humanity the most?
"The Nobel Prize for Stuff We Think is Cool."
What a bunch of BS, yet another field trying to steal the thunder of CS. How often have I had to listen to physicists sneer at CS as not a proper science!
thanks the feeling is mutual since comp scientists sneer at physicists too, just for other reasons
"This year’s laureates used tools from physics to construct methods that helped lay the foundation for today’s powerful machine learning."
Does this mean if I'd use a deep understanding of birds to design way more aerodynamic airplanes, I could get the Nobel prize in physiology/medicine? Don't get me wrong, their work is probably prize worthy, but shouldn't the Nobel prize in physics be awarded for discoveries in the _physical world_?
> Don't get me wrong, their work is probably prize worthy
I would strongly disagree with you there. It's the exact same idea as the least squares approximation or conjugate gradient method: create an energy function from a quadratic and minimize it.
Next: Authors of "Attention is all you need" paper get Nobel Prize in Literature for writing one of the most read papers in past decade.
Bob Dylan already won that prize to similar controversy.
Nobel Prize in Literature has always been controversial. It seems that the Physics Prize is headed that way (it’s not just this year)…
this is the best comment
Sweden Yes!