3.1. Context windows #

Current generative AI models, whether textual or graphical, still face significant challenges when it comes to maintaining context. While they can remember things within a single session, these models are quick to forget once the session is lost or a new one starts, necessitating the need to prompt them with relevant details all over again.

For instance, if I’m using GPT-4 to brainstorm marketing strategies for the launch of a new feature but fail to save the conversation or simply lose it, ChatGPT will not recall the context I’ve already shared. Details such as my business description or my target audience profiles will need to be re-provided when I approach OpenAI for help with launching another feature.

Custom instructions do offer some level of control over how ChatGPT responds, allowing you to set your preferences and have them remembered for future conversations. I’m not particularly fond of this feature, given that it requires you to prepare in advance by laying out as much context as possible up-front. This feels like a chore, detracting from the overall magic of the experience. It would be far more beneficial, I believe, if the model could summarize key points at the end of a conversation and store them as “core memories.” Ideally, it should learn from our conversations, but this seems unfeasible due to the immense context window needed for the model to retain everything we discuss, or the impracticality of retraining a personalized model for each user.

Another possible solution to the context problem is the implementation of integrations. For instance, an AI-driven sales manager could learn about your customers and product by linking it to your Google account and accessing your past emails. And integrations will be a huge game-changer in the world of large language models, opening up incredible opportunities for startups. When a company like Google launches an AI assistant, that assistant is limited to operating solely within Google. But imagine an independent assistant that could seamlessly integrate with other platforms. It could pull brand assets from your cloud storage, whip up UI mockups in Figma, and then send them over to an AI programmer to code the frontend.

This kind of cross-functionality is currently unachievable with the walled garden approach favored by large corporations, but it represents a significant potential avenue for disruption by generative AI. OpenAI knows this, too. You can tell by their announcement of the ChatGPT Enterprise offering. One notable feature under development is customization, which will allow companies to securely augment ChatGPT's knowledge base with their own data through integration with existing applications. I think this is a sound strategic direction.