i.e. they need to be trained with real data in a very specific/narrow field,
Imo, it depends on the task, or prompt, and what's expected to be done with the response. The other issue is that, in order to keep up with current information on anything, it'll be necessary to use the internet. Boom. There's the problem
.
For ex, will limiting a tcc AI to the info on RSF make it more or less accurate? Other than pure math, is there a subject where there can be universal agreement on answers?
Remember that movie "War Games'? It'd be scary to have an AI controlling missile defense if it just relied on mathematical calculations. In the movie, for some reason, the AI decides it's senseless to fire the missiles. That might be considered model collapse to the designer, or it might be considered a kind of computer-based morality. Otoh, chess AI have no problem sacrificing everything to win.
"A man is rich when he has time and freewill. How he chooses to invest both will determine the return on his investment."