𝙲𝚑𝚊𝚒𝚛𝚖𝚊𝚗 𝙼𝚎𝚘𝚠

  • 0 Posts
  • 150 Comments
Joined 2 years ago
cake
Cake day: August 16th, 2023

help-circle
  • That’s applying existing solutions to a different programming language or domain, but ultimately every single technique used already exists. It only applied what it knew, it did not come up with something new. The problem as stated is also not really “new” either, image extraction, conversion and rendering isn’t exactly a “new problem”.

    I’m not disputing that LLMs can speed up some work, I know it occasionally does so for me as well. But what you have to understand is that the LLM only remembered similar problems and their solutions, it did not at any point invent something truly new. I understand the distinction is difficult to make.


  • You’re referring to more generic machine learning, not LLMs. These are vastly different technologies.

    And I have used them for programming, I know their limitations. They don’t really transfer solutions to new problems, not on their own anyway. It usually requires pretty specific prompting. They can at best apply solutions to problems, but even then it’s not a truly generalised thing, even if it seems to work for many cases.

    That’s the trap you’re falling into as well; LLMs look like they’re doing all this stuff, because they’re trained on data produced by people who actually do so. But they can’t think of something truly novel. LLMs are mathematically unable to truly generalize, it would prove P=NP if they did (there was a paper from a researcher in IIRC Nijmegen that proved this). She also proved they won’t scale, and lo and behold LLM performance is plateauing hard (except in very synthetic, artificial benchmarks designed to make LLMs look good).


  • Well the thing is, LLMs don’t seem to really “solve” complex problems. They remember solutions they’ve seen before.

    The example I saw was asking an LLM to solve “Towers of Hanoi” with 100 disks. This is a common recursive programming problem, takes quite a while for a human to write the answer to. The LLM manages this easily. But when asked to solve the same problem with with say 79 disks, or 41 disks, or some other oddball number, the LLM fails to solve the problem, despite it being simpler(!).

    It can do pattern matching and provide solutions, but it’s not able to come up with truly new solutions. It does not “think” in that way. LLMs are amazing data storage formats, but they’re not truly ‘intelligent’ in the way most people think.