Videogames have always had AI. The baddies don’t just drop down one row and come back across the screen anymore.
LLM-powered NPCs will quickly fall out of fashion as people realize they’re literally just talking to chatGPT.
The either forced always-online requirement with privacy violating telemetry for server-side LLMs, or immensely high GPU memory requirements for local LLMs will also cripple their games.
immensely high GPU memory requirements for local LLMs will also cripple their games.
Not really, you can tune a llm to do what you want.
Why have a llm know about 17th century European politics or modern science when you are sticking it into a fantasy video game.
How small can you make an LLM before it starts having issues with grammar and coherency? I would argue that the bare minimum still would be rather large, and in videogames we’re already using vram for other resources. In a 3D game especially I imagine very little vram is left to utilize.
You’d be surprised how small you can go. That’s IMO pretty much the future of AI - a shit ton of small specialized models. While the heavyweights have their use, they’re way too expensive and overkill for specialized tasks.
Some small models can comfortably run on the CPU as well, games can easily detect whether you have VRAM to spare and use GPU or CPU based on that.
It’s not there, yet, but what some of the small models can do is impressive. And if you train them extensively on fantasy scripts, I can see them generating NPC lines on the fly.
Not sure, but what I am sure on is companies paying “ai engineers” (or whatever they are called) to trim them to a usable point instead of hiring a better writing team.
That’s immensely expensive though, and not guaranteed to work because much of that stuff is still research stage. You’re right that paring down the models to make them leaner and more specialized is the primary direction that current research is pursuing, but it’s far from certain at this point how to do it, how well it will work, and how small you can get them before they start to fall apart. Not something game studios are likely to gamble their budgets on, at least not yet.
We’re nowhere near the “just hire a guy to trim it down instead of hiring writers” stage, and it’s unclear yet whether or not that’s where we’ll end up. We could pull off “just hire a guy to fine-tune an existing foundation model,” but that doesn’t make them smaller.
Did any of you guys see those Skyrim AI NPC mods that were floating around YouTube around a year ago? I think they kept getting taken down for some reason.
But honestly it convinced me that that’s a good way to go. They were able to detect what the player was doing. For instance, if you dropped a sword on the ground, they would be like “hey don’t forget that iron greatsword on your way out”. You could also ask it for advice on how to complete your quest or what they knew about someone.
If the AI is somehow locked to only knowing information about the video game they’re in, it could really boost the immersion I think. But I think they should have to use a real voice actor to train the AI to mimic their voice or something like that.
Locked? Won’t somebody think of the cupcakes?!
I think the top use will actually be diffusion. A small model could add a lot of variability to palettes to start. Then move on to something like a slider puzzle where an AI is altering the initial terrain configuration in dynamic ways. It would likely create an environment where a player’s natural pacing can be compensated for in real time.
I could also picture a function calling setup that attempts to optimise player experience.
I’ve been looking at ways to generate tiles for CDDA using a small model. There is a ton of potential especially in open source. I won’t run any model unless it is open weights running on my hardware.
I know everyone hates AI being forced into stuff but I think it could be great for games, both in regards to voice acting like here and with creating loads of low-level content, like boring litter, etc. I’m playing Cyberpunk right now and I keeps thinking how it’s a shame that you can’t enter most of the buildings. I’m hoping that with AI that will somehow become possible. Imagine being able to walk into a huge office block, find any random person then have a deep conversation with them. It’ll be amazing. Right now if you walk up to anyone who doesn’t have something specific to say it’s just “Can I help you” and shit like that.
Yes it can. But we are still ways to go before it become a reality. The current AI is still missing the ‘intelligence’ and is no better than a a well designed predetermined script.
For every game that try to make good use of AI to improve on the player experience and immersion there will be hundreds, thousands more that do the exact opposite. Because that is capitalism for you. Just look at Upscaling/Frame generation.
Smart studios will keep their writers for the main NPCs and use AI for the random filler NPCs that wander around cities and the like. We’ll see how many studios are smart I guess.
