To achieve this, I am designing, protoyping, and evaluating ubiquitous computing user interfaces with expressive data visualizations or representations. Ideally, interacting with these system feels “natural” or “intuitive” and creates meaningful human-data experiences. Increasingly, this requires integrating intelligible and explainable forms of data science and artificial intelligence into interactive systems.

Consequently, my research focuses on the intersection of human-data interaction (HDI), human-computer interaction (HCI), interaction design (IxD), information visualization (InfoVis), computer-supported cooperative work (CSCW) and ubiquitous computing (UbiComp).

The overall goal of my research is to understand how to best support individuals or teams during data exploration and sensemaking: 

• What kind of interaction and visualization techniques and data representations do users perceive as “natural” and easy-to-use?
• Which techniques are easier to learn and cognitively less demanding?
• How can we harness novel post-WIMP (post-“Windows Icons Menus Pointer”) interaction techniques and ubiquitous computing systems, e.g., multi-touch, pen input, gestures, tangible objects, interactive tables and walls, virtual/augmented reality?
• How can we better support collaboration and navigation in large data sets?
• How can we avoid bias, false interpretations, and convey uncertainty or gaps in data?
• How to use thoughtful interaction designs to map algorithmic and statistical methods to intelligible and explainable representations or metaphors?

On a more theoretical level, I am very interested in how to apply cognitive science and theories of embodied cognition to human-data interaction. As a result, I have proposed the conceptual framework of Blended Interaction as a cognitive model that can help us to design more “natural” interactive systems with a better user experience and fluid interaction.

X-PRO: User-Centered Methods for Cross-Virtuality Analytics of Production Data (at FH Upper Austria), Jan 2020 until Dec 2024

Cross-virtuality analytics (XVA) enables seamless integration and transition between conventional 2D visualization, augmented reality, and virtual reality in order to provide users with optimal visual and algorithmic support depending on their current tasks and needs in the analysis process.

The X-PRO project proposal was successfully developed and submitted by me as principal investigator together with Christoph Anthes, Manuel Brunner, Matthias Dehmer, Bernhard Fröhler, and Christoph Heinzl. It started only a few months before I left FH Upper Austria to the University of Lübeck but I will be associated with the project as a consultant and external collaborator until the end of the project in 2025.

X-PRO is funded as an FTI Project by the State of Upper Austria with 2.7 Mio Euro for 5 years.

More information: Journal Article in Computer Graphics Forum on Cross-Virtuality Analytics, CHI 2020 workshop paper on AR-VR-transitions and ISS 2020 workshop paper on Cross-Virtuality Visualization, Interaction, and Collaboration.

DISMOSIM: Distributed Modelling and Simulation of Cyberphysical Systems (with School of Engineering, FH Upper Austria), Aug 2018 – July 2020 (until joining University of Lübeck)

In the DISMOSIM project, my team is responsible for the user-centered design, prototyping, and evaluation of novel remote collaboration tools. For example, we will use large high-resolution touch screens with pen input to enable engineers to collaboratively manipulate 3d engineering models of vehicles and to run mechanical simulations and visualize their results.

My research activity and work package within the project have a total volume of EUR 0.39 Mio and are funded with EUR 0.28 Mio by the Austrian FFG for 3 years.

More information: Article presented at ISM 2020

USIVIS: User-Centered Interactive Visualization of Big Data (with School of Management, FH Upper Austria), Dec 2016 – July 2020 (until joining University of Lübeck)

I am PI of the USIVIS (User-Centered Interactive Visualization) project in which I closely work together with Peter Hofer from our School of Management.

In this 4-year project funded by the Austrian Research Promotion Agency (FFG), we work together with our colleagues from our School of Management (Campus Steyr) and industry partners to design, prototype, and evaluate interactive data visualizations for “Big Data”. Our goal is to transfer InfoVis research results into practice and to support our industry partners with design guidelines and best practice examples for visualizing “Big Data” from finance and smart production. This will also include interaction and visualization techniques for cross-device and collaborative visualization using multiple tablets and large touch- and pen-enabled screens, e.g. Microsoft Surface Hub.

The project has a total volume of EUR 0.94 Mio and is funded with EUR 0.66 Mio by the Austrian FFG for 4 years.

More information: USIVIS website, CHI 2017 full paper (Honorable Mention at CHI 2017)

SCOLA: “Smart Collaboration for Industrial Applications”, Dec 2016 – Dec 2018

I am leading the project SCOLA, in which we explore how multi-user and multi-device collaboration can be used to support distributed teams in production, logistics, or development.

The team includes research associate Ing. Thomas Neumayr BA MA who pursues a PhD and Ing. Thomas Luger MSc who supports our research as a software developer.

The project has a total volume of EUR 0.1 Mio and is funded by the FH Upper Austria for 2 years.

More information: SCOLA website and CSCW 2018 full paper (Best Paper Award).

Data What? Urban Data Visualizations and Applications (University College London, Intel ICRI Cities), 2014 & 2015

Technologists conjure a future in which smart cities and the Internet of Things collect large amounts of data and provide them to companies and governments. But how could the citizens and their communities benefit from this? Can we build easy but powerful tools for using this data and empower citizens and their causes?

During our workshop in Somerleyton Road in Brixton, we worked together with the community-led development project Brixton Green to understand what data and services really matter to local residents, the local council, local activists, and artists. To inspire citizens, we provided them with a data visualization tool for tablets that helped them understand and compare air pollution in Brixton and London.

More information:  Two workshop papers at CHI 2015  Who Should Lead the Development of Community Technology? and HuddleLamp: Exploring Community Data with Co-located Mobile Screens

Embodied Navigation in Information Spaces (with University of Konstanz and University College London), 2012-2015

Recent findings from Embodied Cognition reveal strong effects of arm and hand movement on spatial memory. This suggests that input techniques may have a far greater influence on users’ cognition and users’ ability to master a system than we typically believe – especially for navigating in information spaces. We conducted different studies of mouse and multi-touch input and also embodied/egocentric peephole navigation to understand how body movement and gesturing affects spatial navigation and memory performance.

More information: Full paper at AVI 2012, full paper at ITS 2013, full paper at CHI 2014, note at CHI 2015.

Blended Interaction (with University of Konstanz), 2009-2013

We introduce Blended Interaction, a new conceptual framework that helps to explain when users perceive user interfaces as “natural” or not. Based on recent findings from embodied cognition and cognitive linguistics, Blended Interaction provides a novel and more accurate description of the nature of human-computer interaction (HCI). In particular, it introduces the notion of conceptual blends to explain how users rely on familiar and real-world concepts whenever they learn to use new digital technologies.

More information: Journal article on Blended Interaction in Pers. and Ubiq. Computing, June 2014, Volume 18, Issue 5, pp 1139-1158.

SenCity (University College London, Intel ICRI Cities), 2013

In SenCity we explored how the physical design of urban sensors (i.e. shape, appearance) can change city dwellers’ attitudes and perceptions towards being sensed. We organized a workshop at UbiComp 2013 and let participants try out different physical designs in the city to gather first reactions. For example, we found that anthropomorphic and zoomorphic designs resulted in greater engagement and trust while neutral or less visible designs created rejection and anxiety.

More information: Urb-IoT 2014 paper Suspicious Boxes and Friendly Aliens: Exploring the Physical Design of Urban Sensing Technology 

ZOIL: Zoomable Object-Oriented Information Landscape (University of Konstanz, Microsoft Research Cambridge), 2007-2013

The ZOIL paradigm is the main topic of my PhD thesis on designing and implementing multi-user and multi-device interactive spaces.  It consists of design principles, an open source software framework, and example prototypes.

More information: PhD thesis.

Awarded with the Airbus Defence and Space Research Award Claude Dornier 2014  and the Prize of the Alumni Organization of the University of Konstanz (VEUK) for Outstanding Doctoral Dissertation.

NAVI – Navigational Aids for the Visually Impaired (University of Konstanz), 2011

We present a proof-of-concept of a mobile navigational aid that uses the Microsoft Kinect and optical marker tracking to help visually impaired people find their way inside buildings. The system is the result of a student project and is entirely based on low-cost hard- and software. It provides continuous vibrotactile feedback on the person’s waist, to give an impression of the environment and to warn about obstacles.

Best poster award at INTERACT 2011. Over 160,000 hits on YouTube. Featured in Opening Keynote of Microsoft Mix Conference 2011.

Facet-Streams: Fluid interaction for InfoVis (University of Konstanz, Microsoft Research Cambridge), 2010/2011

We introduce “Facet-Streams”, a hybrid interactive surface for co-located collaborative product search on a tabletop. Facet-Streams combines techniques of information visualization with tangible and multi-touch interaction to materialize collaborative search on a tabletop. It harnesses the expressive power of facets and Boolean logic without exposing users to complex formal notations.

The Facet-Streams UI with tangbile and multi-touch interaction serves as a best-in-class example for fluid interaction for information visualization.

More information: Facet-Streams paper and video (Honorable Mention Paper Award at CHI 2011) and journal article on Fluid Interaction for Information Visualization.

DeskPiles (University of Konstanz, Microsoft Research Cambridge), 2009/2010

The DeskPiles prototype was developed as a part of a cooperation project between the HCI Group of the University of Konstanz, the Integrated Systems team at Microsoft Research Cambridge, and the NanoPhotonics Centre at the University of Cambridge. It supported NanoPhotonics researchers during sharing and discussing lab results by closely integrating tablet PCs with an interactive tabletop.

More information in my PhD Thesis and in the DeskPiles video on YouTube.

Digital In-Car Manual – Digitale Betriebsanleitung (University of Konstanz, Daimler AG / Mercedes Benz), 2009

In this industry project we explored novel ways to integrate digital and interactive car manuals into passenger vehicles. To achieve this, we used existing infotainment systems and combined them with smart phones.

A Browser for Rear-Seat Entertainment (University of Konstanz, Volkswagen AG/Audi AG), 2009

In this industry project we explored novel ways to integrate Web browsers as rear seat entertainment into luxury vehicles.

More information: Paper on optimized mouse cursor movement for automotive scenarios.

MedioVis / Blockbuster (University of Konstanz, Library of the University of Konstanz, Deutsche Forschungsgemeinschaft DFG), 2004-2007

MedioVis is concerned with the user-centred development of next generation visual information seeking systems for novice and non-expert users. A central design goal of the MedioVis interface is to offer interaction focussed on supporting realistic human search behaviour: Not only analytical queries but also browsing-oriented strategies shall be supported, for example by allowing a quick overview, filtering, zooming into details, and “surfing” within the search results.

More information: MedioVis page and Blockbuster page.

MedioVis was deployed at the Library of the University of Konstanz and in everyday use by students from 2004 to 2014. MedioVis/Blockbuster also made the 3rd place of IEEE InfoVis Contest 2007.