What’s going on in VR, HCI and Games Research in 2022? – Notes from attending the IEEE VR 2022, CHI 2022, and DIGRA 2022 conferences
- debbie ding
Well folks, its official now. A couple months ago, I tendered from my teaching job and TODAY I’ve matriculated as a graduate student. In 24 days I will be embarking the first year of a PhD.
Since the idea behind doing a PhD is presumably to do a lot of writing and get said writing published, in order to prepare for this, I thought I should attend a few conferences to figure out where I might want to submit my writing? Here are some impressions and insights I had from (virtually) attending IEEEVR 2022, CHI 2022 and DIGRA 2022.
What does IEEE VR describe itself as?
The IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR) is the premier international event for the presentation of research results in the broad areas of virtual, augmented, and mixed reality (VR/AR/XR). Since 1993, IEEE VR has presented groundbreaking research and accomplishments by virtual reality pioneers: scientists, engineers, designers, and artists, paving the way for the future. Soon, IEEE VR expanded its scope to also include augmented, mixed, and other forms of mediated reality. Similarly, the IEEE Symposium on 3D User Interfaces (3DUI), which started as a workshop at IEEE VR in 2004, has become the premier venue for 3D user interfaces and 3D interaction in virtual environments. In 2018, VR and 3DUI were merged into the IEEE Conference on Virtual Reality and 3D User Interfaces, with the short name IEEE VR. In 2020, VR was the first major IEEE conference to be held entirely online and in a virtual environment.
I was drawn to this because IEEE VR seemed to be the place where a lot of pioneering work in VR/XR/AR has been presented. However, if I was forced to give a definition to the discipline represented at this conference – you could conceivably consider this to be a computer science conference on VR/XR/AR that has expanded to include research from psychology, neuroscience, interaction design and UI. Oh and of course, hardware advancements in display technologies.
Some of the “hot” / trending research seemed to include things like:
- Digital Twins
- Blindness Remapping
- Redirected Walking
- Haptic Retargetting
When I first signed up for the conference, they asked me to fill out a form indicating what keywords I was interested in within VR research. I knew that although some of the words seemed quite simple, I realised that that they must have another much more complicated meaning attached to them in the world of VR Research.
Here’s a table in which I tried to break down what “areas” of VR/XR/MR research were of interest to me at IEEE VR.
(Disclaimer: The following notes are entirely by Debbie whose only understanding of the contemporary state of VR research was from attending IEEE VR 2022 and thus these are very likely and most definitely NOT the textbook definitions of the keywords!)
Understanding Keywords in VR Research
|Keyword||What I understood about the term||How do people study this||Is this of interest to me?|
|presence||presence means to feel physically and spatially located in that environment. research could also be about how people do things once they’re present, such as whether one can kick accurately in vr, virtual body ownership and agency, the ability of VR avatars to influence user’s attitudes and behaviour, also includes immersive telepresence experiences and plausibility (believing your are in that environment). may also touch on digital twins which supposedly quite hot at the moment.||some studies are on conceptual and philosophical frameworks of how we perceive reality and testing through experimental studies||✅ I am interested in Presence, Plausibility, which also is tied in with embodiment. At the heart of this seems to be the big questions of reality, place, trust, and belief systems.|
|embodiment||virtual embodiment and avatars, full body illusions, body representation of hands, feet, other body parts, what about dual body representations, distortions of different race or scale||often controlled experiments which are either within- (each participant tests different levels) or between (comparison between different groups of participants)||✅ Ask the human what they think of their body. It intersects with research questions about presence.|
|immersive visualisation||immersive visualisation of data such as spatio-temporal data, 3d data, real places, historical places, indigenous culture preservation and continuation, real estate, architecture||case studies and experimental setups to see how to improve ease of use and minimise negative effects||✅ This has the potential to cross over to different fields and mapping.|
|virtual production||development or virtual production studios with led walls, game engine, mocap, 3d graphics to produce films, distributed pipeline||often surveys and case studies of uses||✅ This could potentially still encompass what I do, which is finding a medium for how to make artwork, and virtual production is about discussing the process and pipeline and workflows.|
|interaction design||techniques from interaction design for navigation, manipulation, browsing, and inspecting in VR; VR affordances like handles, sliders, widgets, personalisation||often involving studies of whether users completing tasks and tweaking the UI to see if performance could be improved by changing the UI||🤔 Within my expertise. But I am not sure if I want to spend all my time being what is essentially a UI designer in VR and user testing that.|
|advanced ui||novel tangible user interfaces to be used in VR, embodied interaction in VR through room-scale tangible props (such as a football), navigation instruction preferences such do arrows work, stuff like testing if desaturation of non-target areas||experimental studies which create fundings that could serve as guidelines for content creators||🤔 If its a novel interface with physical build, maybe too technical for me, but if its about novel UI design, then it could be within my expertise like the interaction design keyword. But I am not sure if I want to spend all my time being what is essentially a UI designer in VR and user testing that.|
|virtual humans / virtual agents||testing out different experimental setups with virtual humans such as “what happens if the virtual humans stare at you like a non-verbal cue”, or can the virtual humans help to virtually prime the real human to do something they otherwise would not do||often calculating human response to these virtual humans through fixation and dwell time||🤔 like designing a NPC and then doing a psychology study on them. some of this touches on interesting or weird questions.|
|3dui||novel 3D user interfaces such as time anchored comments on online videos, kinesthetic learning using 3d ui, 3d mid air gestures, peripheral 3d menus, gaze-only 3d menus.||often involving studies of the eye and head movement maybe even in terms of fixation duration, saccade amplitude, velocity of head rotatioin, eye-head coordination, etc||❌ Too technical for me to contribute to. I would imagine this is the kind of research that results in being able to create an engineered product like Tobii Eye tracker.|
|inclusive vr||ways to mitigate and minimise cybersickness, auditory or visual feedback to improve balancing in VR, field of view was common way to reduce cybersickness||often done through questionaires or sometimes more complex physiological measures, medical scanning and imagery||❌ Outside of my area of expertise. Interesting and important, but some parts also are physiological / neurological. Those kinds of measures and research is beyond my ability to conduct.|
|locomotion||walking, teleporting, redirected walking (user walks in different path in real world compared to perceptual path in virtual world), redirection and doors, object size, jumping on steps, jogging in vr and ar, friction feedback on shoes, natural walking, braking||often involving users walking and being tasked to navigate to a certain point, studies and analysis of paths, trajectories, user position and orientation. looking for ease of use, efficiency, and convienience.||❌ No, too technical. Discussions often become very mathematical, some also use machine learning to analyse and improve steering and redirection of walking.|
|multimodal||multimodality / multisensory refers to a combination of visual, auditory, kinesthetic, tactile for a fully immersive experience; such as simulating auditory of olfactory cocktail party effect, shape aware haptics, effects of vibrations, haptic retargeting (ie: where one prop provides passive haptics for multiple objects)||often computational evaluations of the effects of each of these perceptual effects||❌ No, too technical. Not only does one build the whole sensory prop which could be deeply complex, then often the data is collected and analysed through computational or even machine learning means. I mean, haptics! You need an engineer to build the haptics, don’t you? [Qn for self: is it so impossible to build my own haptics? Or build my own VR headset from scratch?]|
|collaboration||co-location, co-manipulation techniques on how to integrate inputs from multiple users, computer supported collaborative work in vr||often controlled experiments to figure out best practices for synchronising the data from collaboration||❌ Network connectivity in vr, latency, maths, a lot is out of my ability. most of this is applied to working and serious games and CSCW. its a field.|
|perception||ability to perceive objects, visual perception and what affects it – environmental lighting, display, shadow, position, depth perception, cues to improve target awareness, field of views, what happens if you focal switch or add blurs with context switching, stereo vs mono||measurement of reaction time to stimulus, fatigue levels, comfort levels, optical abberation like minification or magnification||❌ Some optical effects are on hardware level and not software level and too technical for myself to design for.|
|negative effects||discomfort, disorientation, interruptions of experience, what is responsible for widening the gaps between stimulus and perception, emotional impact, cybersickness, children’s long term impact||studies can be about physical negative effects and emotional experiences. questions about getting around ethical guidelines abound.||❌ some parts can be extremely technical to the point of measuring using galvanic skin test and other physiological conditions, whilst others just did questionnaires and focus groups. worth asking why one looks into this. Is it because VR is for the good of humanity? Who says it must be? What is my take on ethics?|
|displays||advancements in actual VR displays such as AR windshield displays, varifocal holograms, holographic displays, wearable displays, state-of-the-art optical see-through head-mounted displays, novel optical architectures, real time systems, computer graphics rendering||comparison tables of output, latency, technical evaluation||❌ No, WAY too technical.|
|haptics||a whole lot of sticks and simulation devices. tapping, vibration, haptic proxies, various suggestions for strategies for haptic retargetting such as using body warping (misalign the virtual object to fit the physical object) or blindness remapping (after you look away, the virtual objects becomes aligned with the physical proxy)||often a lot of mixed reality cross overs and game experience or presence questionnaires||❌ Most of this is too technical. Its like building robotics prototypes and testing the haptic effects.|
|audio in vr||how does sound and speech and auditory experience affect the perception of the world/space or the perceived personality of the virtual human or virtual agent, auditory cues, methods and strategies for audio in virtual scenes, audio quality evaluation for VR scenes||often through questionaires||❌ Not an area of my specialisation or particular interest.|
|rendering||computer graphics rendering, preservation of details, peripheral regions, computer graphic equations, computing methodologies, pixel distribution, leveraging ocular limitations, geometry buffer, gaze estimation by analysing a single picture using custom algorithm||tables which compare the different types of visual output, using different data and||❌ Completely out of my ability. The definition of technical computer science.|
|computer vision||using computer vision to do things like tracking gaze, looking at how to generate flicker pattern for time of flight, projection patterns, calibration||build a prototype to test||❌ Too technical for me.|
|machine learning||using machine learning to do things like depth estimation, GAN for generating realistic scan paths for 360 images (scanpath generation)||experimental studies to find out which method has superior performance||❌ Too technical for me.|
In terms of a virtual conference setup, IEEE VR had the best user experience that I have ever experienced. All the sessions were linked together in a Zoom Lobby and you could even “peek” in without joining the entire session. Without leaving Zoom you could already read the descriptions of the sessions and bookmark the ones you were interested in attending.
In addition to that, the Discord was like a carefully tended garden with links and discussions.
If you liked to have an “in-person” experience, there was also the Virbela campus where there were games and the opportunity to simply walk past another attendee in the conference hall and start speaking. (I was able to speak to several people this way! So amazing!)
What does CHI 2022 describe itself as?
The ACM CHI Conference on Human Factors in Computing Systems is the premier international conference of Human-Computer Interaction (HCI). CHI – pronounced ‘kai’ – annually brings together researchers and practitioners from all over the world and from diverse cultures, backgrounds, and positionalities, who have as an overarching goal to make the world a better place with interactive digital technologies.
This is actually the first HCI conference I’ve attended. This was not a fully virtual conference, so I feel like there must have been a lot that I missed out on as a virtual attendee. The confusing Room system made it quite difficult to understand where to attend the right sessions. I was thoroughly spoiled by the brilliant user experience I had at IEEE VR.
Although the title of my Masters has a name which resembles something from the HCI field (“Design Interactions”), those in the know will be aware that Dunne and Raby’s DI programme had never been synonimous with traditional HCI research, but instead with critical and speculative design. So… the approaches I saw used in the HCI research were quite formal and to be frank, less interesting to a weirdo like me who already knows they don’t want to write a HCI paper or do a PhD in HCI specifically. 😅 Sorry, I’ve decided that I have to go full weird.
On the practical side, I noticed all the conferences have a Doctorial Consortium where the presentations were of varying level of completeness but I enjoyed lurking and gathering up plenty good tips and advice for new PhD students. I did glean a lot of info from just the Discord channels too.
The one big benefit about having made myself attend CHI 2022 is that I properly signed up as an SIG-CHI student member and now have access to the ACM Digital Library (big resource!).
What does DIGRA describe itself as?
Founded in 2003, DiGRA is the premiere international association for academics and professionals who research digital games and associated phenomena. It encourages high-quality research on games, and promotes collaboration and dissemination of work by its members.
As someone who is likely to write a paper about “Art Games” (help! argh! what a waffly term!), out of the conferences I’ve attended so far, I think this might be the most relevant conference for me because a lot of the work situates itself in Game Philosophy but it also has enough of the practical design aspect to it too. The research is also often weird and left-field and shows so many different creative approaches to producing work in an interactive medium.
But the flip side to that is that I immediately found that I could not tolerate theory without practice. It makes sense that I feel this way, because I do want to do a practice-based PhD, and for me the theory and practice must go hand-in-hand.
I attended a few sessions dealing with very abstract philosophical discussions of reality but if they did not involve any visual references or technical experiments (some paper presentations didn’t even have a single picture on their decks) then I just could not deal with that or take any of that seriously. I don’t expect everyone to be technical wizards here or to have to directly engage in game design/game development to have a voice in his field, but it is so accessible these days that I think that technical literacy is a basic necessity.
Even the random sessions I accidentally attended were mind-expanding for me. A smattering of some of the interesting ideas/topics I encountered included:
- Abusive Game Design
- Intentionally designing evil game design briefs to instigate game design students to formulate their ethical position on game design (“I quit! I reject the assignment!” being the desired outcome)
- An online ethnography of mirror dwellers in VRchat
- LitRPG – you mean people read novels about grinding in casual games?? and the whole plot is centred around the narrator’s game progression??? Why… why don’t these readers just play a game? Or why doesn’t the author just design a game? [Is this interpassivity at work?]
- Freedom and Worldbuilding in tabletop gaming – the “mountain top dilemma” – your player climbs to the top of the mountain and says, “Hmm, can I go over to the other mountain top?” and the gamemaster is scrambling, furiously flipping the books.
- Uncanny design valleys. Games normalise weird, and that is not necessarily helpful for those using the medium of games from a Critical Speculative Design perspective… 👀
- Sometimes the player does not know who to “play” as when playing a speculative / art game, especially if it deals with historical / cultural / social / religious sensitivities. You may have to tell them who to “roleplay” as in order for them to know how to play it. A solution: You tell the player to roleplay as someone who has to critically analyze or reflect on the game, possible as a user tester / quality assurance tester who has to evaluate if the wider public should play it.
- What the artist wants out of making an art game may be the opposite of what the audience of the art game wants 🤯 [mind blown, I never thought of this before]
There’s no discounting it, I feel very privilleged to have the financial means to attend the conferences, as I was working full-time. I am grateful to have the virtual option so that I didn’t have to travel to attend these. The dingparents also helped with childcare so I could attend these properly – many thanks to them!
As you can see, all of the conferences were situated in the western/european world and although I see well-meaning attempts to improve diversity and represent the global south, the fact is still that the financial cost of these conferences (even though quite modest) may be out of the reach of many. Here’s an honest breakdown of the fees I paid as an attendee (cheapest routes possible):
|IEEE VR (Christchurch, New Zealand)||CHI 2022 (New Orleans, US)||DIGRA 2022 (Krakow, Poland)|
|Non-Member Early Bird NZD$120 |
Non-Member Regular NZD$220
I got the Non-Member Early Bird at NZD120 (Approx SGD 103.30)*
[Sponsored by my current employer]
|SIGCHI student membership at USD19|
Student ticket USD140
I got the student membership and ticket at USD159 (Approx SGD 222.98) [Self paid]
|Early Bird 450 PLN (about 95 EUR)|
Regular 850 PLN (about 180 EUR)
I got the annual DIGRA membership and full online ticket for 450 PLN (Approx SGD 131.38) [Self paid]