It did not sound like that's the only preprocessing step, but even with that, how "costly" would that be for a model comparable to ChatGPT 4 or 5?

Also, the comment was not related to LLMs only.

Note that the goal is to get comparable performance, iow to compare like for like.