Nice read. I think we need to clearly call out the quality of the training data on the foundational models. Also, what are the limitations of the specific foundation models, and what is the reason we are fine -tuning the model. Is it for small footprint of resources or domain specific knowledge scenarios ? We will see different mechanism for these and target segments - one where we need smaller LLMs fine-tuned vis-a-vis fine-tuning for domain specific knowledge. New architectures will emerge, as we keep moving the bottleneck by bringing in additional layers for specific requirements for customization potential, cost structure, skills required, privacy and ownership requirements.

Expand full comment