The way you teach an LLM, outside of training your own, is with rules files and MCP tools. Record your architectural constraints, favored dependencies, and style guide information in your rule files and the output you get is going to be vastly improved. Give the agent access to more information with MCP tools and it will make more informed decisions. Update them whenever you run into issues and the vast majority of your repeated problems will be resolved.
If it’s doesn’t work for you, it’s because you’re a failure!
Still not convinced these LLM bros aren’t junior developers (at best) who someone gave a senior title to because everyone else left their shit hole company.
More to the point, that is exactly what the people in this study were doing.
They don’t really do into a lot of detail about what they were doing. But they have a table on limitations of the study that would indicate it is not.
We do not provide evidence that: There are not ways of using existing AI systems more effectively to achieve positive speedup in our exact setting. Cursor does not sample many tokens from LLMs, it may not use optimal prompting/scaffolding, and domain/repository-specific training/finetuning/few-shot learning could yield positive speedup.
Back to this:
even if it did it’s not any easier or cheaper than teaching humans to do it.
In my experience, the kinds of information that an AI needs to do its job effectively has a significant overlap with the info humans need when just starting on a project. The biggest problem for onboarding is typically poor or outdated internal documentation. Fix that for your humans and you have it for your LLMs at no extra cost. Use an LLM to convert your docs into rules files and to keep them up to date.
Your argument depends entirely on the assumption that you know more about using AI to support coding than the experienced devs that participated in this study. You want to support that claim with more than a “trust me, bro”?
Do you think that like nobody has access to AI or something? These guys are the ultimate authorities on AI usage? I won’t claim to be but I am a 15 YOE dev working with AI right now and I’ve found the quality is a lot better with better rules and context.
And, ultimately, I don’t really care if you believe me or not. I’m not here to sell you anything. Don’t use it the tools, doesn’t matter to me. Anybody else who does use them, give my advice a try an see if it helps you.
Again, read and understand the limitations of the study. Just the portion I quoted you alone is enough to show you that you’re leaning way too heavily on conclusions that they don’t even claim to provide evidence for.
That is a moronic take. You would be better off learning to structure your approach to SW development than trying to learn how to use a glorified slop machine to plagiarize other people’s works.
In practice I find the more stuff like this you throw at it the more rope it has to hang itself with. And you spend so much time prompt adjusting so it doesn’t do the wrong things that you were better off just doing half of the tasks yourself.
This is not really true.
The way you teach an LLM, outside of training your own, is with rules files and MCP tools. Record your architectural constraints, favored dependencies, and style guide information in your rule files and the output you get is going to be vastly improved. Give the agent access to more information with MCP tools and it will make more informed decisions. Update them whenever you run into issues and the vast majority of your repeated problems will be resolved.
Well, that’s what they say, but then it doesn’t actually work, and even if it did it’s not any easier or cheaper than teaching humans to do it.
More to the point, that is exactly what the people in this study were doing.
If it’s doesn’t work for you, it’s because you’re a failure!
Still not convinced these LLM bros aren’t junior developers (at best) who someone gave a senior title to because everyone else left their shit hole company.
They don’t really do into a lot of detail about what they were doing. But they have a table on limitations of the study that would indicate it is not.
Back to this:
In my experience, the kinds of information that an AI needs to do its job effectively has a significant overlap with the info humans need when just starting on a project. The biggest problem for onboarding is typically poor or outdated internal documentation. Fix that for your humans and you have it for your LLMs at no extra cost. Use an LLM to convert your docs into rules files and to keep them up to date.
Your argument depends entirely on the assumption that you know more about using AI to support coding than the experienced devs that participated in this study. You want to support that claim with more than a “trust me, bro”?
Do you think that like nobody has access to AI or something? These guys are the ultimate authorities on AI usage? I won’t claim to be but I am a 15 YOE dev working with AI right now and I’ve found the quality is a lot better with better rules and context.
And, ultimately, I don’t really care if you believe me or not. I’m not here to sell you anything. Don’t use it the tools, doesn’t matter to me. Anybody else who does use them, give my advice a try an see if it helps you.
These guys all said the same thing before they participated in a study that proved that they were less efficient than their peers.
Again, read and understand the limitations of the study. Just the portion I quoted you alone is enough to show you that you’re leaning way too heavily on conclusions that they don’t even claim to provide evidence for.
Codex literally lies about being connected to configured MCP servers.
Are you trying to make a point that agents can’t use MCP based off of a picture of a tweet you saw or something?
I’m talking from my personal, daily experience using codex.
That is a moronic take. You would be better off learning to structure your approach to SW development than trying to learn how to use a glorified slop machine to plagiarize other people’s works.
In theory yes.
In practice I find the more stuff like this you throw at it the more rope it has to hang itself with. And you spend so much time prompt adjusting so it doesn’t do the wrong things that you were better off just doing half of the tasks yourself.