In a recent interview, Mustafa Suleyman, Microsoft's head of AI, explored the current trends in the field of artificial intelligence. He predicts that in the forthcoming years, AI models will exhibit a trend where both large and small-scale models will develop simultaneously.
On one hand, the race for scaling up large models is set to continue, incorporating more modalities of data, such as video and images. On the other hand, technology that leverages larger models to train smaller ones, such as distillation, is on the rise. These efficient small-scale models are expected to play substantial roles in specific contexts. Suleyman noted that in the future, knowledge will be condensed into smaller, more affordable models, which will be embedded across various devices, heralding a true ambient sensing revolution.
Suleyman accentuated the importance of prompt engineering for entrepreneurs. By crafting high-quality instruction sets, entrepreneurs can guide pre-trained models to align with their brand values, crafting unique products. Furthermore, small-scale models present significant opportunities; entrepreneurs can harness their low costs and efficiency to develop applications tailored for specific use cases.
During the discussion, Suleyman also underlined the significance of data integration. Synthetic data is expected to become pivotal in training models, yet how to acquire and integrate these data sources requires further exploration.
Additionally, he addressed the integration of new modalities like video and images, as well as the understanding and data collection of action trajectories across complex digital interfaces. Such advancements, he believes, will yield impressive results. For entrepreneurs, the key to future success lies in how they can leverage these new trends and technologies to innovate.