Research Focus

I am a professor of Human-Computer Interaction with a particular fascination for Ubiquitous Human-Data Interaction. More specifically, I am interested in ubiquitous computing technologies that enable individuals or teams to explore, experience, visualize, and make sense of data.

To achieve this, I am designing, protoyping, and evaluating ubiquitous computing user interfaces with expressive data visualizations or representations. Ideally, interacting with these system feels “natural” or “intuitive” and creates meaningful human-data experiences.

Consequently, my research focuses on the intersection of human-computer interaction (HCI), human-data interaction (HDI), ubiquitous computing (UbiComp), information visualization (InfoVis), and computer-supported cooperative work (CSCW).

The overall goal of my research is to understand how to best support individuals or teams during data exploration and sensemaking: 

  • What kind of interaction and visualization techniques and data representations do they perceive as “natural” and easy-to-use?
  • Which techniques are easier to learn and cognitively less demanding?
  • How can we harness novel post-WIMP (post-“Windows Icons Menus Pointer”) interaction techniques and ubiquitous computing systems, e.g., multi-touch, pen input, gestures, tangible objects, interactive tables and walls, virtual/augmented reality?
  • How can we better support collaboration and navigation in large data sets?
  • How can we avoid bias, false interpretations, and convey uncertainty or gaps in data?

On a more theoretical level, I am very interested in how to apply cognitive science and theories of embodied cognition to human-data interaction. As a result, I have proposed the conceptual framework of Blended Interaction as a cognitive model that can help us to design more “natural” interactive systems with a better user experience and fluid interaction.

Main Projects

DISMOSIM: Distributed Modelling and Simulation of Cyberphysical Systems (with School of Engineering, FH Upper Austria), starts Sep 2018

In the DISMOSIM project, my team is responsible for the user-centered design, prototyping, and evaluation of novel remote collaboration tools. For example, we will use large high-resolution touch screens with pen input to enable engineers to collaboratively manipulate 3d engineering models of vehicles and to run mechanical simulations and visualize their results.

My research activity and work package within the project have a total volume of EUR 0.38 Mio and are funded with EUR 0.26 Mio by the Austrian FFG for 3 years.

 

 

USIVIS: User-Centered Interactive Visualization of Big Data (with School of Management, FH Upper Austria), since Dec 2016

I am leading the joint project group USIVIS (User-Centered Interactive Visualization) together with Peter Hofer from our School of Management.

In this 4-year project funded by the Austrian Research Promotion Agency (FFG), we work together with our colleagues from our School of Management (Campus Steyr) and industry partners to design, prototype, and evaluate interactive data visualizations for “Big Data”. Our goal is to transfer InfoVis research results into practice and to support our industry partners with design guidelines and best practice examples for visualizing “Big Data” from finance and smart production. This will also include interaction and visualization techniques for cross-device and collaborative visualization using multiple tablets and large touch- and pen-enabled screens, e.g. Microsoft Surface Hub.

In the USIVIS team there are two research associates with a master’s degree who pursue a PhD (Lisa Falschlunger BA MA, Björn Zimmer B.Sc. M.Sc.), two student researchers with a bachelor’s degree (Markus Kern BSc, Conny Walchshofer B.Sc.), and an external doctoral candidate (Dipl.-Wirtsch.-Ing. Thomas Plank M.A.).

The project has a total volume of EUR 0.94 Mio and is funded with EUR 0.66 Mio by the Austrian FFG for 4 years.

More information: USIVIS website, CHI 2017 full paper (Honorable Mention at CHI 2017)

 

 

SCOLA: “Smart Collaboration for Industrial Applications”, since Dec 2016

I am leading the project SCOLA, in which we explore how multi-user and multi-device collaboration can be used to support distributed teams in production, logistics, or development.

The team includes research associate Ing. Thomas Neumayr BA MA who pursues a PhD and Ing. Thomas Luger MSc who supports our research as a software developer.

The project has a total volume of EUR 0.1 Mio and is funded by the FH Upper Austria for 2 years.

More information: SCOLA website and CSCW 2018 full paper (Best Paper Award).

Other Projects

HuddleLamp: Ad-hoc Cross-Device Interaction & Visualization (with University of Konstanz, University College London, Intel ICRI Cities), 2014-2018

We present HuddleLamp, a desk lamp with an integrated RGB-D camera that precisely tracks the movements and positions of mobile displays and hands on a table. This enables a new breed of spatially-aware multi-user and multi-device applications for around-the-table collaboration without an interactive tabletop. At any time, users can add or remove displays and reconfigure them in space in an ad-hoc manner without the need of installing any software or attaching markers.

More information: huddlelamp.org, blog post about HuddleLamp on cities.io, ITS 2014 full paper, CHI 2018 full paper.

Best demo award at ACM ITS 2014. Over 50,000 views on YouTube. Featured on many leading tech blogs such as gizmodo.com

Designing & Understanding Cross-Device Interaction & Visualization (with University of Konstanz, University College London, and Aarhus University), 2015-2017.

Technologies such as HuddleLamp, ZOIL or TwisterSearch enable us to work across multiple devices with very different form factors, e.g., smart phones, tablets, interactive tabletops, or walls. But how exactly should we design cross-device gestures and visualizations and how do the devices’ form factors affect cross-device collaboration? In this project, we build cross-device prototypes to evaluate different cross-device gestures and visualizations and to understand how different screen sizes and visualizations can change communication patterns among collaborators.

More information: CHI 2015 full paper, CHI 2016 full paper, CHI 2017 full paper (Honorable Mention at CHI 2017).

Data What? Urban Data Visualizations and Applications (University College London, Intel ICRI Cities), 2014 & 2015

Technologists conjure a future in which smart cities and the Internet of Things collect large amounts of data and provide them to companies and governments. But how could the citizens and their communities benefit from this? Can we build easy but powerful tools for using this data and empower citizens and their causes?

During our workshop in Somerleyton Road in Brixton, we worked together with the community-led development project Brixton Green to understand what data and services really matter to local residents, the local council, local activists, and artists. To inspire citizens, we provided them with a data visualization tool for tablets that helped them understand and compare air pollution in Brixton and London.

More information:  Two workshop papers at CHI 2015  Who Should Lead the Development of Community Technology? and HuddleLamp: Exploring Community Data with Co-located Mobile Screens

Embodied Navigation in Information Spaces (with University of Konstanz and University College London), 2012-2015

Recent findings from Embodied Cognition reveal strong effects of arm and hand movement on spatial memory. This suggests that input techniques may have a far greater influence on users’ cognition and users’ ability to master a system than we typically believe – especially for navigating in information spaces. We conducted different studies of mouse and multi-touch input and also embodied/egocentric peephole navigation to understand how body movement and gesturing affects spatial navigation and memory performance.

More information: Full paper at AVI 2012, full paper at ITS 2013, full paper at CHI 2014, note at CHI 2015.

Blended Interaction (with University of Konstanz), 2009-2013

We introduce Blended Interaction, a new conceptual framework that helps to explain when users perceive user interfaces as “natural” or not. Based on recent findings from embodied cognition and cognitive linguistics, Blended Interaction provides a novel and more accurate description of the nature of human-computer interaction (HCI). In particular, it introduces the notion of conceptual blends to explain how users rely on familiar and real-world concepts whenever they learn to use new digital technologies.

More information: Journal article on Blended Interaction in Pers. and Ubiq. Computing, June 2014, Volume 18, Issue 5, pp 1139-1158.

SenCity (University College London, Intel ICRI Cities), 2013

In SenCity we explored how the physical design of urban sensors (i.e. shape, appearance) can change city dwellers’ attitudes and perceptions towards being sensed. We organized a workshop at UbiComp 2013 and let participants try out different physical designs in the city to gather first reactions. For example, we found that anthropomorphic and zoomorphic designs resulted in greater engagement and trust while neutral or less visible designs created rejection and anxiety.

More information: Urb-IoT 2014 paper Suspicious Boxes and Friendly Aliens: Exploring the Physical Design of Urban Sensing Technology 

ZOIL: Zoomable Object-Oriented Information Landscape (University of Konstanz, Microsoft Research Cambridge), 2007-2013

The ZOIL paradigm is the main topic of my PhD thesis on designing and implementing multi-user and multi-device interactive spaces.  It consists of design principles, an open source software framework, and example prototypes.

More information: PhD thesis.

Awarded with the Airbus Defence and Space Research Award Claude Dornier 2014  and the Prize of the Alumni Organization of the University of Konstanz (VEUK) for Outstanding Doctoral Dissertation.

NAVI – Navigational Aids for the Visually Impaired (University of Konstanz), 2011

We present a proof-of-concept of a mobile navigational aid that uses the Microsoft Kinect and optical marker tracking to help visually impaired people find their way inside buildings. The system is the result of a student project and is entirely based on low-cost hard- and software. It provides continuous vibrotactile feedback on the person’s waist, to give an impression of the environment and to warn about obstacles.

Best poster award at INTERACT 2011. Over 160,000 hits on YouTube. Featured in Opening Keynote of Microsoft Mix Conference 2011.

Facet-Streams: Fluid interaction for InfoVis (University of Konstanz, Microsoft Research Cambridge), 2010/2011

We introduce “Facet-Streams”, a hybrid interactive surface for co-located collaborative product search on a tabletop. Facet-Streams combines techniques of information visualization with tangible and multi-touch interaction to materialize collaborative search on a tabletop. It harnesses the expressive power of facets and Boolean logic without exposing users to complex formal notations.

The Facet-Streams UI with tangbile and multi-touch interaction serves as a best-in-class example for fluid interaction for information visualization.

More information: Facet-Streams paper and video (Honorable Mention Paper Award at CHI 2011) and journal article on Fluid Interaction for Information Visualization.

DeskPiles (University of Konstanz, Microsoft Research Cambridge), 2009/2010

The DeskPiles prototype was developed as a part of a cooperation project between the HCI Group of the University of Konstanz, the Integrated Systems team at Microsoft Research Cambridge, and the NanoPhotonics Centre at the University of Cambridge. It supported NanoPhotonics researchers during sharing and discussing lab results by closely integrating tablet PCs with an interactive tabletop.

More information in my PhD Thesis and in the DeskPiles video on YouTube.

Digital In-Car Manual – Digitale Betriebsanleitung (University of Konstanz, Daimler AG / Mercedes Benz), 2009

In this industry project we explored novel ways to integrate digital and interactive car manuals into passenger vehicles. To achieve this, we used existing infotainment systems and combined them with smart phones.

A Browser for Rear-Seat Entertainment (University of Konstanz, Volkswagen AG/Audi AG), 2009

In this industry project we explored novel ways to integrate Web browsers as rear seat entertainment into luxury vehicles.

More information: Paper on optimized mouse cursor movement for automotive scenarios.

MedioVis / Blockbuster (University of Konstanz, Library of the University of Konstanz, Deutsche Forschungsgemeinschaft DFG), 2004-2007

MedioVis is concerned with the user-centred development of next generation visual information seeking systems for novice and non-expert users. A central design goal of the MedioVis interface is to offer interaction focussed on supporting realistic human search behaviour: Not only analytical queries but also browsing-oriented strategies shall be supported, for example by allowing a quick overview, filtering, zooming into details, and “surfing” within the search results.

More information: MedioVis page and Blockbuster page.

MedioVis was deployed at the Library of the University of Konstanz and in everyday use by students from 2004 to 2014. MedioVis/Blockbuster also made the 3rd place of IEEE InfoVis Contest 2007.