This is huge news for the open-source community. Meta and thought leaders in GenAI have claimed that Llama 3.1 is comparable to other front-tier models such as GPT-4o and Claude 3.5 Sonnet. While it might not be as simple to gain access to this new open-source model, it raises a major question:
Could universities and colleges host their own GenAI models?
Well - some universities already have. I spoke with Sadie Guerrero, a key leader in AI adoption at Tecnológico de Monterrey, and found out how this top institution in Mexico is leading the charge to adopt GenAI.
Check it out here: TECgpt: The First Generative AI Model in Latin America
The digital tool, developed by Tec de Monterrey, will provide real-time access to information for teachers and collaborators.
Open-source means the source code is available for anyone on the internet to use. In GenAI’s case, this means that anyone can implement and host their own GenAI using an open-source LLM. Let’s look at what available models are out there.
Out of all open-source models, the recently released Llama 3.1 405b is comparable to frontier models such as GPT4o and Claude 3.5.
You can run these an open-source LLM on your own PC which means it isn’t out of reach for universities and colleges to host their own.
I always ask myself this above question if it’s a tool that will be used by students. I can name a few benefits:
Institutions that host their own GenAI can make this tool available to all students regardless of their financial background. ChatGPT and other AI tools cost $20 per month and that’s not something all students can afford.
Additionally, having a tool hosted by the institution and available to everyone means that it can incorporated into courses easier since every student has access. Students can then gain hands-on experience with GenAI tools and that’s an advantage going into the workforce.
With universities hosting their own GenAI, certain intellectual property and data security concerns can be put to rest.
“Will my course material be out there in ChatGPT?” is one concern I get a lot from instructors when demoing TimelyGrader, where we use both Claude 3.5 and GPT4o. While I always mention that the data we pass back via the API won’t be used, the concern is still there.
By hosting GenAI internally, universities can control data management practices and minimize external risks so students and instructors can use GenAI without compromising their privacy.
Having self-hosted GenAI also enables instructors to troubleshoot incidents where misuse of GenAI occurs.
If a student alleges that ChatGPT gave bad feedback and the student acted on it, the onus is on the student to make sure the feedback is good but isn’t always fair. Instructors will have to make a decision without all the information on what feedback was received and what the prompt was.
Self-hosting gives instructors and academic affairs staff transparency into what the AI’s chain of thought is, what the output is, and what the input is. Giving them a clearer idea of what happened.
Investing in a self-hosted GenAI can be costly, not just from an initial development cost but ongoing support can be substantial.
If you were involved with onboarding faculty and/or students, you know how big a pain it can be. Not everyone goes to the onboarding session and less listen and understand.
Self-hosting puts the onus on the institutional and administrative staff to onboard new users, provide regular training sessions such as ‘lunch-and-learns’ and support users when something bad happens (and it will).
Institutions will need to hire specialists to set up the infrastructure for the GenAI including servers and hardware since running an AI at full scale requires substantial computing power.
Universities will also need to invest in cybersecurity to protect sensitive data and build a scalable infrastructure to handle potential heavy usage.
I don’t see many universities self-hosting GenAI (yet). If there are, it will be the R1 institutions taking the first dive because they are the pockets to experiment. I believe we will begin to see service providers that aid in the implementation of self-hosted LLM specifically for high-risk and highly regulated industries such as education.
For now, it’s probably a few blocks down the road before this picks up but as edtech tools ourselves, we are looking a few steps ahead anyway.
Enhancing education with AI-powered grading and feedback.
