• 0 Posts
  • 187 Comments
Joined 2 years ago
cake
Cake day: June 30th, 2023

help-circle
  • The thrust of this article seems to be that the important thing is that automatic transcription services be compliant with unspecified “governance standards”. It goes on to give a generally glowing review of a specific medical transcription service:

    This software, Accurx Scribe, has been developed and deployed in line with all current NHS England requirements for AVT, and there is no suggestion this product breaches any rules, standards or guidance.

    Indeed, the company which developed it meets weekly with NHS England on creating a standardised approach to scale the benefits across the NHS.

    However their website seems to indicate that their privacy practices are garbage as transcriptions are implied to happen on company servers:

    At Accurx, our employees may need to see patient data that we store for for strictly limited purposes.

    This seems pretty absurd to me since the technology is at the point where effective on-device transcription is a reality. Why look at whether bureaucrats have rubber stamped something instead of looking at the actual commonsense properties of who has access to the data? That could easily be the doctor and no one else. The question of what constitutes good security and privacy isn’t even something this article wants to bring up for consideration.











  • What you confuse here is doing something that can benefit from applying logical thinking with doing science.

    I’m not confusing that. Effective programming requires and consists of small scale application of the scientific method to the systems you work with.

    the argument has become “but it seems to be thinking to me”

    I wasn’t making that argument so I don’t know what you’re getting at with this. For the purposes of this discussion I think it doesn’t matter at all how it was written or whether what wrote it is truly intelligent, the important thing is the code that is the end result, whether it does what it is intended to and nothing harmful, and whether the programmer working with it is able to accurately determine if it does what it is intended to.

    The central point of it is that, by the very nature of LKMs to produce statistically plausible output, self-experimenting with them subjects one to very strong psychological biases because of the Barnum effect and therefore it is, first, not even possible to assess their usefulness for programming by self-exoerimentation(!) , and second, it is even harmful because these effects lead to self-reinforcing and harmful beliefs.

    I feel like “not even possible to assess their usefulness for programming by self-exoerimentation(!)” is necessarily a claim that reading and testing code is something no one can do, which is absurd. If the output is often correct, then the means of creating it is likely useful, and you can tell if the output is correct by evaluating it in the same way you evaluate any computer program, without needing to directly evaluate the LLM itself. It should be obvious that this is a possible thing to do. Saying not to do it seems kind of like some “don’t look up” stuff.


  • Are you saying that it is not possible to use scientific methods to systematically and objectively compare programming tools and methods?

    No, I’m saying the opposite, and I’m offended at what the author seems to be suggesting, that this should only be attempted by academics, and that programmers should only defer to them and refrain from attempting this to inform their own work and what tools will be useful to them. An absolutely insane idea given that the task of systematic evaluation and seeking greater objectivity is at the core of what programmers do. A programmer should obviously be using their experience writing and testing both typing systems to decide which is right for their project, they should not assume they are incapable of objective judgment and defer their thinking to computer science researchers who don’t directly deal with the same things they do and aren’t considering the same questions.

    This was given as an example of someone falling for manipulative trickery:

    A recent example was an experiment by a CloudFlare engineer at using an “AI agent” to build an auth library from scratch.

    From the project repository page:

    I was an AI skeptic. I thought LLMs were glorified Markov chain generators that didn’t actually understand code and couldn’t produce anything novel. I started this project on a lark, fully expecting the AI to produce terrible code for me to laugh at. And then, uh… the code actually looked pretty good. Not perfect, but I just told the AI to fix things, and it did. I was shocked.

    But understanding and testing code is not (necessarily) guesswork. There is no reason to assume this person is incapable of it, and no reason to justify the idea that it should never be attempted by ordinary programmers when that is the main task of programming.


  • The problem, though, with responding to blog posts like that, as I did here (unfortunately), is that they aren’t made to debate or arrive at a truth, but to reinforce belief. The author is simultaneously putting himself on the record as having hardline opinions and putting himself in the position of having to defend them. Both are very effective at reinforcing those beliefs.

    A very useful question to ask yourself when reading anything (fiction, non-fiction, blogs, books, whatever) is “what does the author want to believe is true?”

    Because a lot of writing is just as much about the author convincing themselves as it is about them addressing the reader. …

    There is no winning in a debate with somebody who is deliberately not paying attention.

    This is all also a great argument against the many articles claiming that LLMs are useless for coding, in which the authors all seem to have a very strong bias. I can agree that it’s a very good idea to distrust what people are saying about how programming should be done, including mistrusting claims about how AI can and should be used for it.

    We need science #

    Our only recourse as a field is the same as with naturopathy: scientific studies by impartial researchers. That takes time, which means we have a responsibility to hold off as research plays out

    This on the other hand is pure bullshit. Writing code is itself a process of scientific exploration; you think about what will happen, and then you test it, from different angles, to confirm or falsify your assumptions. The author seems to be saying that both evaluating correctness of LLM output and the use of Typescript is comparable to falling for homeopathy by misattributing the cause of recovering from illness. The idea that programmers should not use their own judgment or do their own experimentation, that they have no way of telling if code works or is good, to me seems like a wholesale rejection of programming as a craft. If someone is avoiding self experimentation as suggested I don’t know how they can even say that programming is something they do.





  • It’s not quite the same thing as deploying soldiers against protesters, but technically all of those things are done ultimately through the use of coercive and violent force. Don’t want to go to school? Your parents will make you, because if they don’t they could be imprisoned. Slightly inconvenience drivers by walking across a busy street not at a crosswalk? Could be fined or arrested for jaywalking. Pose a hazard to rocket launches by flying a makeshift aircraft in federal airspace with no flight plan? You know the drill. That’s not to mention the funding for all those things, the violence inherent in which doesn’t stop at taxes, but also is a central factor in maintaining the value of a currency in a variety of different ways.


  • This kind of reminds me of that time Apple made a big show of resisting court efforts to get them to unlock iphone data; they have every reason to cultivate an impression of caring about privacy, but this isn’t actually evidence that they do. Giving them all this info about your life by holding a continual ongoing conversation about what you’re doing on their servers is inherently pretty bad for your privacy in a way reassurances can’t really fix.

    There’s a lot of reasons to prefer local AI instead and this is a big one.