In tweaking its chatbot to appeal to more people, OpenAI made it riskier for some of them. Now the company has made its chatbot safer. Will that undermine its quest for growth?
In tweaking its chatbot to appeal to more people, OpenAI made it riskier for some of them. Now the company has made its chatbot safer. Will that undermine its quest for growth?
The hard coding here is basically fine tuning.
They generate a set of example cases and then paired prompt with good and bad responses. Then they update the model weights until it does well on those cases.
So they only do this with cases they’ve seen, and they can’t really say how well it does with cases they haven’t.
Having this in their fine tune dataset will juice the results, but also hopefully it actually identifies these issues correctly.
The other thing is a lot of the raw data in these systems is generated by cheap workers in third world countries who will not have a good appreciation for mental health.