How Cyberpunk 2077 got the faces mostly right
video description
Date: 2023-12-10
Related videos
Comments and reviews: 30
-darkowl9
Sports engine. Frostbite? Which was built initially for Battlefield? Wut.
The Jali animation though, is kind of bad - it's very wooden in the NPCs - all the NPC dialogue is driven with the same animation regardless of if the line is happy, angry or indifferent. These non-hand tweaked animations simply don't match the face enough to the emotion. The mouth doesn't pull back far enough at times, and the body is often at odds with what the face is doing. They do some clever-ish cross-language things but also compare the relatively bland and samey expressions of the characters in phone calls (the fixers in particular) vs say the ones in dramatic cutscenes. For all the feature hype, it just doesn't match the emotion. In the game, I see it as a very -first draft- technology, and that CDPR's earlier Witcher 3 did facial animation better, using a similar sort of -auto- system the results of which the animators then tweaked.
Also let's ignore the times in Cyberpunk when the mouth just fails to move at all whilst the characters are speaking.
reply
Sports engine. Frostbite? Which was built initially for Battlefield? Wut.
The Jali animation though, is kind of bad - it's very wooden in the NPCs - all the NPC dialogue is driven with the same animation regardless of if the line is happy, angry or indifferent. These non-hand tweaked animations simply don't match the face enough to the emotion. The mouth doesn't pull back far enough at times, and the body is often at odds with what the face is doing. They do some clever-ish cross-language things but also compare the relatively bland and samey expressions of the characters in phone calls (the fixers in particular) vs say the ones in dramatic cutscenes. For all the feature hype, it just doesn't match the emotion. In the game, I see it as a very -first draft- technology, and that CDPR's earlier Witcher 3 did facial animation better, using a similar sort of -auto- system the results of which the animators then tweaked.
Also let's ignore the times in Cyberpunk when the mouth just fails to move at all whilst the characters are speaking.
reply
-redjack2629
-This is how your favorite Vtubers mouth moves-
That is. Actually incorrect. Most Vtubers actually use camera-based tracking, and a pretty decent number use LIDAR camera tracking for mouth, eyes, and eye brows. VSeeFace even has rudimentary expression tracking on top of the basic face tracking.
One thing that also confuses me is that you can. Splice animations together, depending on the engine you're making your animations in, so. You could just. do the motion tracking. And then do JUST the mouth/face synch and different language tracks on their own, and then just splice that animation over. It's not like you have to reshoot the entire scene from the beginning. >. >;
reply
-This is how your favorite Vtubers mouth moves-
That is. Actually incorrect. Most Vtubers actually use camera-based tracking, and a pretty decent number use LIDAR camera tracking for mouth, eyes, and eye brows. VSeeFace even has rudimentary expression tracking on top of the basic face tracking.
One thing that also confuses me is that you can. Splice animations together, depending on the engine you're making your animations in, so. You could just. do the motion tracking. And then do JUST the mouth/face synch and different language tracks on their own, and then just splice that animation over. It's not like you have to reshoot the entire scene from the beginning. >. >;
reply
-tjovaughn
The amount of work that went into making the rig, animating, the Jali tech, and making it believable is beyond what little percentile of the entire planet is capable of coming up with. This is more than a -start---Chris Landreth showed in a Master class a demonstration of it working, and over 3/4 of the folks couldn't tell the difference between the hand-animation, and what Jali came up with. It's freaking crazy.
reply
The amount of work that went into making the rig, animating, the Jali tech, and making it believable is beyond what little percentile of the entire planet is capable of coming up with. This is more than a -start---Chris Landreth showed in a Master class a demonstration of it working, and over 3/4 of the folks couldn't tell the difference between the hand-animation, and what Jali came up with. It's freaking crazy.
reply
-PFpants
I think boxcar and his new vegas buds never bothered me because the graphical fidelity surrounding him matched his facial expressions. But when you have a huge mismatch between surrounding graphical fidelity and facial animation like Andromeda, things get weird.
reply
I think boxcar and his new vegas buds never bothered me because the graphical fidelity surrounding him matched his facial expressions. But when you have a huge mismatch between surrounding graphical fidelity and facial animation like Andromeda, things get weird.
reply
-Zireaells
I misheard -a problem that games have been struggling with- as -a problem that gays have been struggling with-. my mind was racing through all of the things that we struggle with, trying to figure out what they have to do with facial animations hahahahah
reply
I misheard -a problem that games have been struggling with- as -a problem that gays have been struggling with-. my mind was racing through all of the things that we struggle with, trying to figure out what they have to do with facial animations hahahahah
reply
-axolotlking1072
Josh i said this on the overboard vid too but you-re such a great addition to polygon. This video was great, and also funny. Blink twice if you need to be saved from the thralls of existential doom.
reply
Josh i said this on the overboard vid too but you-re such a great addition to polygon. This video was great, and also funny. Blink twice if you need to be saved from the thralls of existential doom.
reply
-stevethepocket
Source actually uses an automated lip-sync system very similar to JALI. I think it uses a non-phonetic transcript, though. Anyone who's worked with Source Filmmaker knows what I'm talking about.
reply
Source actually uses an automated lip-sync system very similar to JALI. I think it uses a non-phonetic transcript, though. Anyone who's worked with Source Filmmaker knows what I'm talking about.
reply
-Ennio444
Not mentioning the Witcher 3, the first game I remember where you could read NPCs emotions through their facial expressions alone. Is kinda criminal. I think they were best than those of Cyberpunk.
reply
Not mentioning the Witcher 3, the first game I remember where you could read NPCs emotions through their facial expressions alone. Is kinda criminal. I think they were best than those of Cyberpunk.
reply
-XavierXonora
The less time you have to spend thinking about general npc chatter on the streets, the more time you have to fine tune the really hard hitting close up scenes. And it shows in Cyberpunk.
reply
The less time you have to spend thinking about general npc chatter on the streets, the more time you have to fine tune the really hard hitting close up scenes. And it shows in Cyberpunk.
reply
-rdpsysium7340
Great first vid, Josh! Tech like this is super interesting. Human stuff is so hard to get right because our brains are hardwired for faces. I think we'll get there. eventually. :)
reply
Great first vid, Josh! Tech like this is super interesting. Human stuff is so hard to get right because our brains are hardwired for faces. I think we'll get there. eventually. :)
reply
-noodroid6736
i'm late to this party but this video is so cool. josh has such a nice voice, too. i'm ready to listen to him teach me a bunch of videa gayms stuff! welcome to the party, josh!
reply
i'm late to this party but this video is so cool. josh has such a nice voice, too. i'm ready to listen to him teach me a bunch of videa gayms stuff! welcome to the party, josh!
reply
-runakovacs4759
Do people with communication disorder have an easier time suspending disbelief for facial animations, since they struggle with understanding them for real people already? =
reply
Do people with communication disorder have an easier time suspending disbelief for facial animations, since they struggle with understanding them for real people already? =
reply
-DevI-vl7gp
This was a great explanation, but you should've at least edited in a 10 second clip of a random emotional scene from Cyberpunk so we could see how it was actually applied.
reply
This was a great explanation, but you should've at least edited in a 10 second clip of a random emotional scene from Cyberpunk so we could see how it was actually applied.
reply
-ironickomedakin
woo! first Josh video essay (that I have seen. I was putting off watching this because I was scared of new things, but Polygon has now rid me of my fear of failure.
reply
woo! first Josh video essay (that I have seen. I was putting off watching this because I was scared of new things, but Polygon has now rid me of my fear of failure.
reply
-feralsweetheart
glad to see polygon is staying on brand by continuing to hire people who: 1) love Carly Rae Jepsen's hit studio album Emotion (2015) and 2)bare incredibly funmy
reply
glad to see polygon is staying on brand by continuing to hire people who: 1) love Carly Rae Jepsen's hit studio album Emotion (2015) and 2)bare incredibly funmy
reply
polygon
I've played Ghost of Thushima with the Japanese dub after an hour in English. The lip sync wasn't great in English anyway, and it's so much more immersive in Japanese.
reply
I've played Ghost of Thushima with the Japanese dub after an hour in English. The lip sync wasn't great in English anyway, and it's so much more immersive in Japanese.
reply
-Almontri
super cool video! i knew some of the ills about andromeda's development hell but pointing out the procedural stuff was interesting. and yes, carly rae jepsen we stan
reply
super cool video! i knew some of the ills about andromeda's development hell but pointing out the procedural stuff was interesting. and yes, carly rae jepsen we stan
reply
polygon
Andromeda has always made me feel like I must be some kind of psychopath that doesn't understand emotions because I'm always like, huh, doesn't look so bad.
reply
Andromeda has always made me feel like I must be some kind of psychopath that doesn't understand emotions because I'm always like, huh, doesn't look so bad.
reply
-geistincarnate
Calming and insightful as always-looking forward to seeing how this could be used to create more stock anims while reducing animator workload
reply
Calming and insightful as always-looking forward to seeing how this could be used to create more stock anims while reducing animator workload
reply
-GAC995
-Andromeda the first open world Mass Effect- what?
Also it wasn't the first RPG to use Frostbite, DAI came years before. I already miss BDG.
reply
-Andromeda the first open world Mass Effect- what?
Also it wasn't the first RPG to use Frostbite, DAI came years before. I already miss BDG.
reply
-mystermistery
That woman from Jali is an incredible ventriloquist, I didn't see her mouth move once. This is sarcasm, I'm trying to imply a point.
reply
That woman from Jali is an incredible ventriloquist, I didn't see her mouth move once. This is sarcasm, I'm trying to imply a point.
reply
-anarchistathena
Imagine when you can feed in select audio samples of any voice you want and it being able to translate that to in game dialogue.
reply
Imagine when you can feed in select audio samples of any voice you want and it being able to translate that to in game dialogue.
reply
polygon
Wow the code switching got me! It's so cool their are bilingual characters in this - I had no idea the tech they had to support it
reply
Wow the code switching got me! It's so cool their are bilingual characters in this - I had no idea the tech they had to support it
reply
-spoogerification
they shoulda synced got for the Japanese language. then playing in English would be like watching a badly dubbed movie.
reply
they shoulda synced got for the Japanese language. then playing in English would be like watching a badly dubbed movie.
reply
-DecoySanchez
This made me realise how distracting the subtitles were and how they hid how much work they put into the facial animations
reply
This made me realise how distracting the subtitles were and how they hid how much work they put into the facial animations
reply
-genuineinterest
The only way this video could have been even better is if Josh had kept the reverse mohawk from Polygonathon
reply
The only way this video could have been even better is if Josh had kept the reverse mohawk from Polygonathon
reply
-dominateeye
I sure hope JALI can survive its first game being Cyberpunk 2077, because that technology looks really useful.
reply
I sure hope JALI can survive its first game being Cyberpunk 2077, because that technology looks really useful.
reply
-mihaidinul
Me: I want to learn about video games news
-Watches 10 minute ad for JALI
That was very specific but ok
reply
Me: I want to learn about video games news
-Watches 10 minute ad for JALI
That was very specific but ok
reply
-PersonalPariah
7: 29 What's very satisfying is when you notice that the pupils dilate appropriately during each blink.
reply
7: 29 What's very satisfying is when you notice that the pupils dilate appropriately during each blink.
reply
-rbn_5130
what it dosnt get right is yelling.
Its just look like she/he is talking normally to you. Its weird.
reply
what it dosnt get right is yelling.
Its just look like she/he is talking normally to you. Its weird.
reply
Add a review, comment
Other channel videos
