At Polar Labs, we're constantly seeking ways to improve existing processes. It's not an uncommon thing when working with us to hear: “There must be a better way to do this…”.
Since the pandemic, we've witnessed a massive shift towards remote learning, impacting everything from corporate training to professional certifications to classrooms. This surge in remote training presents both opportunities and challenges. One key challenge is recreating the dynamic, interactive element of being able to ask questions in real-time to a teacher who has knowledge of both the course material and the wider field of study.
We believe we've found the seed of a solution in our new prototype, Professor Prompt - an AI-powered tool that simulates the student-teacher experience in a digital environment. It's like being able to raise your hand in a lecture and get immediate answers, even when that lecture is a pre-recorded video.
What We Built: A Glimpse into the Future of Training
Imagine using an online training app where you can pause the video at any point and ask a question - about the content, a specific term, or even something related but not explicitly covered - and receive an immediate, contextually relevant answer.
To showcase this potential, we built a demo that brings this vision to life, just in a limited context. We chose an education-focused scenario simply because it's universally relatable and allowed us to quickly generate a knowledge base by scraping Wikipedia. We then gave our AI a catchy (if a little silly) name: Professor Prompt.
The demo lets you watch a video on the solar system, pause it at will, and ask Professor Prompt questions, receiving answers that demonstrate a deep understanding of the video's content and the broader subject matter.
Built on Snowflake's powerful data platform and utilizing their new Terraform provider, Professor Prompt transforms passive video watching into an interactive learning experience. The application features a very simple interface with a video player and a chat interface side by side, allowing for quick switching between absorbing information and seeking clarification.
How It Works
Professor Prompt combines several technologies to create an interactive learning experience. Here's a more detailed look at how it works:
Key Components
- Frontend: The user interface is built with Streamlit. It allows users to watch the video and interact with the chat interface in a single window. We implemented custom JavaScript to accurately track the video's current time, even when paused, which is essential for providing relevant answers.
- Backend: Snowflake is the foundation of the application. It stores the knowledge base and provides access to AI functionalities through Snowflake Cortex.
- Knowledge Base Generation: We built a system to automatically retrieve and process Wikipedia content related to the video's topic in order to generate a knowledge base automatically based on the video's transcript (using Cortex to generate search terms). This allows us to quickly build a knowledge base for any topic.
- AI: We use the Mistral large language model, integrated via Snowflake Cortex. Mistral is responsible for understanding user questions and generating responses. We use Cortex Search for semantic search capabilities, allowing Professor Prompt to find relevant information in our knowledge base to enhance the greater context for answering questions.
- Infrastructure: We use Terraform to define the infrastructure as code. This allows us to recreate the entire environment in any Snowflake account quickly and consistently, which is much more reliable/reusable than SQL worksheets for generating demos and deploying the application.
The Process
Here's a step-by-step breakdown of what happens when a user asks a question:
- Capture Video Context: Professor Prompt identifies the precise timestamp in the video the user is referencing.
- Question Enhancement: The user's question is rephrased and expanded upon using a dedicated prompt. This helps improve the accuracy of the information retrieval process.
- Information Retrieval: Snowflake Cortex Search is used to find relevant information. It searches both the video transcript and the external knowledge base (Wikipedia in the demo) for content related to the enhanced question.
- Answer Generation: The Mistral large language model takes all the gathered information, including the user's current timestamp, the video transcript, relevant knowledge base entries, and a limited chat history, and uses it to formulate a clear and concise answer.
- Display Answer with References: The generated answer is displayed in the chat interface. We also include references to the source materials used, providing transparency and allowing users to explore further. We also encouraged Professor Prompt to use images from Wikipedia as often as possible which greatly enhances the experience.
Beyond the Demo: Revolutionizing Corporate Training
While our demo uses an educational scenario and Wikipedia as its knowledge source, the true potential of Professor Prompt lies in its application to corporate training. Both our demo and our larger vision are content agnostic, meaning they can be adapted to suit the specific needs of any organization, from small businesses to large corporations, using any content source as a knowledge base. We believe this demo shows the potential for a comprehensive AI-augmented training platform, encompassing the following features:
- Modular Learning: The full version of this tool would allow companies to create training programs by grouping videos into modules, offering curated learning paths.
- Progress Tracking and Learning Activities: This tool could track learner progress, providing insights into individual and group performance. Integrating interactive learning activities and quizzes could further reinforce knowledge and assess comprehension.
- Hybrid Learning Integration: The tool could seamlessly bridge the gap between online and in-person training. Imagine completing a series of modules on a new product, then registering for an in-person session directly within the platform.
- Instructor Support: Before an in-person session, instructors would have access to a summarized overview of each participant's learning journey, including areas where they excelled, struggled, and any specific topics they've requested to focus on. This will enable instructors to have more personalized, and productive sessions before they've even shaken a student's hand.
- AI-First User Experience: We envision a next-generation UX where AI acts as a personal guide throughout the learning journey, helping users track progress, recommend content, and reinforce learning. No more having to learn which menus go where or which buttons do what. In our vision, Professor Prompt simply guides you through everything dynamically.
A Possible Real-World Example: Automotive Training
Consider a car manufacturer rolling out a new vehicle model. Dealership employees need comprehensive training on the new features, specifications, and selling points. Professor Prompt could deliver this training through engaging video modules, interactive Q&A, quizzes, and more.
An essential part of automotive training is the in-person test drive. Through our tools employees could register for a session directly within the platform. The instructor, armed with insights into each employee's progress, can then tailor the test drive experience to address individual needs and questions, ensuring maximum impact.
Want to See It For Yourself?
One of the core philosophies at Polar Labs is building for reusability. Thanks to the Snowflake Terraform Provider, we have a starting place for any new project allowing us to quickly generate custom demos for potential partners in various industries and contexts. This enables us to easily show what Professor Prompt could look like with their training content.
We're eager to talk with anyone who would be interested in developing Professor Prompt into a fully-fledged AI-augmented training solution. We believe we can use this foundation to develop a novel way to provide remote training, completely customized to the company we build it for. Contact us today to schedule a demo and discuss revolutionizing training and/or education at your organization.
