I find that it basically can’t do decent architecture. My last attempt to use it ended with it using casbin, but then rewriting it’s own authorization framework and trying to use both at the same time 😶.

I think there is a lot of power here, but it needs very heavy guidance and handholding to do it well. Otherwise it makes very stupid intern level decisions

Ideally they’d compare time to write + time to fix. My experience is that if you use test driven development, LLM isn’t too bad. No worse than an intern.

I think it comes down to who is using the LLM. I had a junior dev once “presumably” AI gen a ton of code (broken trash). Then to fix it, they wrapped each function in a try catch block that dropped the error. Unit tests were mocked out to the extent they didn’t test anything.

When I use an LLM, I have tests and hard constraints on the LLM. It isn’t good enough to do everything, but it can generate about 80% of a simple app