Projects
Open Parliament TV
Search Engine & Interactive Video Platform for Parlamentary Debates2020-present
View Project
logible - Sequence Analysis
Detect and visualize behavior sequences in video learning2021
View Project
Open Parliament TV2020-present
Search Engine & Interactive Video Platform for Parlamentary Debates
Almost every parliament publishes video recordings and text proceedings of sessions. But despite comparable structures and similar workflows, parliamentary proceedings are published in various, incompatible formats and parliament tv contents are only accessible via proprietary platforms.
With Open Parliament TV we have developed a parliament independent open source solution which makes the video recordings searchable, sharable and citable via a fully automatic synchronisation of video recordings and text proceedings. Open Parliament TV significantly simplifies finding, sharing, embedding and citing specific video segments of political speeches and thus makes parliamentary processes more transparent and accessible.
Resources
openparliament.tv
de.openparliament.tv
GitHub Repository
Responsibilities
Founder and Co-Director
FrameTrail2015-present
Create, Annotate & Remix Interactive Videos
FrameTrail is open-source software for creating, annotating, and remixing interactive videos directly in the browser. Its unified player and editor lets anyone layer text, images, web pages, and maps as time-synchronized overlays, connect videos through clickable hotspots to build hyperlinked or branching narratives, and arrange material on video- or canvas-based timelines. Documents are stored as portable JSON files, so projects move freely between servers and local setups. Use cases range from annotated lectures and investigative journalism to web documentaries, scholarly publishing with supplementary materials, and interactive storytelling formats.
Resources
frametrail.org
GitHub Repository
Examples / Use Cases
Responsibilities
Idea, Design, Technical Architecture,
Continuous Development since 2016
logible - Sequence Analysis2021
Detect and visualize behavior sequences in video learning
logible (“making log files legible”) is a web application for studying learning through sequences of learner actions rather than isolated clicks. It reads Excel exports from sessions in the FrameTrail environment, where playback, seeks, and annotation edits are logged automatically. A rule-based detector finds predefined behaviour patterns; priority, sequence length, and heterogeneity select the strongest non-overlapping interpretation for each segment. Timelines, sequence lanes, transition heatmaps, and Markov-style graphs let researchers compare individual versus collaborative work and annotation versus hyperlink tasks in one place. Developed with FHNW (Institute of Psychology); described in CSCL and ISLS 2021 proceedings; open source under the MIT License.
Resources
logible web application
Source code (GitHub, MIT License)
Responsibilities
Software architecture, interface design,
visualization, and implementation
Audio-Visual Rhetorics of Affect2019
Film Studies: Explore and Visualize Annotations
The AdA project (German abbreviation for "Affektrhetoriken des Audiovisuellen" - "Audio-Visual Rhetorics of Affect") was a cooperation of the film studies department of FU Berlin and the Semantic Technologies Group of Hasso Plattner Institute for Digital Engineering (HPI) in Potsdam. The aim of this interdisciplinary research group was to support empirical film studies through tool-based semantic video annotation and automated video analytics. The AdA Annotation Explorer is an extension of the FrameTrail hypervideo environment focused on visualizing and comparing large numbers of annotations in multiple movies and scenes.
Resources
ada.cinepoetics.fu-berlin.de
Github Repository
Responsibilities
Concept, Interactive Visualization and Implementation of the Annotation Explorer and FrameTrail extensions
HIRMEOS Hypervideo2018
Explanatory Hypervideo: HIRMEOS Horizon 2020 Project
HIRMEOS Hypervideo turns a three-minute explainer animation on the conceptual, economic, and technical challenges of Open Access monograph publishing into an interactive hypervideo. Produced for the EU Horizon 2020 project HIRMEOS (High Integration of Research Monographs in the European Open Science infrastructure), the video is enriched with FrameTrail: an interactive transcript runs alongside the player while four curated collections — Partners & Platforms, Services & Tools, Open Access Basics, and Scientific Publishing — surface hyperlinks, encyclopedia entries, and PDF monographs exactly when the narration touches the topic.
Resources
Announcement on HIRMEOS Project Page
Responsibilities
Consulting, FrameTrail adjustments, and set-up support
demokratie.io2018
abgeordnetenwatch.de goes Video
In this project we made debates in the German parliament searchable and interactive. Using the Bundestag's Open Data plenary protocols together with the official video archive, an automated pipeline detects speeches, agenda items, and referenced official documents, and synchronizes transcripts sentence-by-sentence with the matching video. A FrameTrail-based player shows the speech alongside an interactive transcript, persons, laws, and documents; clicking a sentence jumps straight to that moment in the video. A byproduct is a simple full-text search engine over 3,000+ Bundestag speeches. Developed in partnership with abgeordnetenwatch.de; funded by demokratie.io (betterplace lab / BMFSFJ / Robert Bosch Stiftung).
Resources
Responsibilities
Concept, Prototypical development of the video transcript generation and audio / text alignment, Implementation of the interactive video player
Field Trip2017-2019
Open Source Documentary
Field Trip is an open source documentary about the Tempelhof airfield in the heart of Berlin. Field Trip tells the story of what, for some, is a 300 hectare free urban paradise, for others a place of forced labour, and for yet more a safe haven. By openly licensing all materials created in the production process and intense on-site and online community development, Field Trip is becoming a living documentary. In a similar fashion as the web works with hypertext links, Field Trip works with hypervideo links, which allow the user to interactively “browse” through a web of interweaved filmic storylines. All components and materials can be found, used and remixed on https://github.com/OpenHypervideo/FieldTrip
Resources
fieldtrip.berlin
Article on BBC R&D Blog
GitHub Repository
Responsibilities
Technical Concept & Architecture, Implementation
FrameTrail: Prototypefund2017
Annotating Political Debates
In 2017, FrameTrail received public funding by the German Federal Ministry of Education and Research (BMBF) via the Prototype Fund to develop new video annotation technology and interfaces. The specific focus of the 6-month project phase was on annotating video recordings of political debates with additional documents at certain points of time (official records, standpoints of different parties, media coverage and background information), as well as presenting these documents in a flexible, reconfigurable user interface.
Resources
Live Demo
Project Page @ Prototypefund
Github Repository
Responsibilities
Project Lead, Concept, Interface Design, Implementation
IIIF-AV Canvas Clock2017
Proof of concept for British Library
Implementation of several test fixtures and reference players for the British Library (lead by digirati), to investigate technical challenges of the IIIF A/V Technical Specification Group. The goal was to seamlessly synchronize multimedia assets described in the IIIF Presentation API v3 format (JSON-LD / Web Annotation Data Model), using a generic "canvas clock" concept. The original proof of concept implementation has now been further implemented as a re-usable Universal Viewer component.
Resources
Live Demo
Evaluation of Existing Approaches for Media Synchronisation
Responsibilities
Concept, Evaluation of existing approaches and Implementation
dwerft2013-2017
Linked Open Data Film & TV Services
dwerft was a 3-year research cluster for new film and tv technologies based on Linked Open Data, residing in the Babelsberg Studios.
The project vision is to connect production, archival and distribution of audiovisual contents from script writing to broadcasting in a holistic workflow based on open, interoperable standards (see "Linked Production Data Cloud").
The project received funding from the German Federal Ministry of Education and Research (BMBF) within the "Entrepreneurial Regions" framework.
Resources
Responsibilities
Research on User Interfaces, Interaction Paradigms and Technology Standards for Semantic Video Annotation, Paper Prototypes, Wireframes, Proof-of-concept Implementations, Interactive Visualizations and User Studies for yovisto
after.video2016
Video Book stored on a Raspberry Pi packaged in a VHS case
after.video is a paperback book and video stored on a Raspberry Pi computer packaged in a VHS case:
"after.video realizes the world through moving images and reassembles theory after video. [...] This is an edited collection of assembled and annotated video essays living in two instantiations: an online version – located on the web at http://after.video/assemblages, and an offline version – stored on a server inside a VHS (Video Home System) case. This is both a digital and analog object: manifested, in a scholarly gesture, as a 'video book'." It is the first use case of "Open Hypervideo" before the FrameTrail environment was released. after.video is published by Open Humanities Press under the terms of the MIT and GPLv3 licenses (ISBN: 978-1-906496-23-4).
Resources
Project Page
after.video @ OPENMUTE Press
Responsibilities
Open Hypervideo Format Development, Producer
refer2016
Semantic Online-Recommendation System
refer is an online-recommendation system based on Linked Open Data and Semantic Web Technologies. It aims to improve the user's and author's experience while curating and navigating in blogs, multimedia platforms, and archives. It thus helps broadcasters, journalists, archives and companies to combine traditional image, audio and video data automatically. refer was created by Tabea Tietz, Jörg Waitelonis, Joscha Jäger and Christian Hentschel in a one year innovation project funded by Media Innovation Center (MIZ) Babelsberg.
VHB Online Course2014
Interactive eLearning Environment
eLearning environment on the subject of 'state liability law' (Staatshaftungsrecht) for the Virtual University of Bavaria (VHB), in cooperation with Bayreuth University.
Fully custom concept, design and implementation. Includes flashcard-based learning and progress tracking, navigation on an endless canvas and content management via a custom XML format.
Responsibilities
Concept, Interface Design, Frontend Development
Superglue2013-2014
Make and host your own webpages at home
Superglue provides you with an independent, end-to-end solution for creating and hosting your webpages at home. With a personal mini-server that you plug right into your wall socket and an in-browser authoring tool it promotes the original Do-It-Yourself ethos of the Internet.
The project has been funded by: European Commission, LGRU, Creative Industries Fund NL and WORM Rotterdam.
Resources
Superglue Project
GitLab Repository
Responsibilities
Concept, Interface Design, Frontend Development
TIB AV-Portal2014-2016
Semantic Video Search Engine
The TIB AV-Portal facilitates finding, browsing & licensing of video fragments, based on an automated recognition and semantic analysis of scenes, text, speech and visual concepts. Various filters enable faceted search based on detected concepts and metadata (ie. licensing model). Found segments & semantic entities are visualized dynamically in the video player and allow an easy navigation through the video contents.
This project has been carried out by the German National Library of Science and Technology in cooperation with Hasso Plattner Institute for Software Systems Engineering (HPI).
Resources
TIB AV-Portal
German National Library of Science and Technology
Responsibilities
Concept, Interface Design, Frontend Development (HTML, JS, CSS, Apache Wicket Framework)
Open Hypervideo2014
Use Case: Flipped Classroom
Video Annotation Prototype for a Flipped Classroom Scenario presented at OCWC Global Conference 2014. Joint work with Olivier Aubert from the COCo Project at Université de Nantes.
The web application is dedicated to supporting video annotation activities. It aims at improving learner engagement, by having students be more active when watching videos by offering a progressive annotation process, first by only synchronizing predefined resources (provided by the teacher), then more freely, to accompany users in the practice of annotating videos. The synchronized resources, constituting annotations, produced by each student can then be centralized by the teacher for discussion and defining a group consensus of the best way to annotate the video.
Resources
Responsibilities
Concept, Interface Design, Frontend Development (HTML, JS, CSS)
WSICC2013-2016
International Workshop on Interactive Content Consumption (2013–2016)
Co-organization of the Workshop on Interactive Content Consumption (WSICC) series, held over four editions from 2013 to 2016. The first workshop took place in conjunction with the last European Conference on Interactive TV (EuroITV) in 2013; the subsequent editions were held with the ACM International Conference on Interactive Experiences for TV and Online Video (TVX) in Newcastle (2014), Brussels (2015), and Chicago (2016).
WSICC offered a forum for combining interdisciplinary, comprehensive views on interactive multimedia, bringing together researchers and industry practitioners to discuss recent technological advances and new forms of content interaction. The scope spanned four dimensions — enabling technologies, interactive content, user experience, and user interaction — covering topics such as hypermedia, second-screen interaction, immersive devices, content personalization, Quality of Experience, audience feedback, and live performance experiences.
Resources
WSICC Website (archived)
Reflecting on the WSICC Series (IEEE MultiMedia, 2017)
Special Issue: Interactive Media: Technology and Experience (Multimedia Tools and Applications, 2017)
Responsibilities
Workshop Co-organization, Program Committee, Reviewer
Technical Demonstrators2013
CeBIT 2013
Demo-Application of the Semantic Media Explorer (semex) technologies developed at Hasso Plattner Institute (HPI). The web application enables live-analysis of text and images by means of Named Entity Recognition (NER), Visual Concept Detection (VCD) and Optical Character Recognition (OCR).
The interface aims at visualizing all required steps of each analysis process in a comprehensible way, by allowing the user to trigger each step manually. This application was exhibited at CeBIT 2013.
Resources
Research Group 'Semantic Technologies' at HPI
About the Semantic Media Explorer
Responsibilities
Concept, Interface Design, Frontend Development (HTML, JS, CSS, Apache Wicket Framework)
Open Hypervideo2012-2013
Use Case: Science Communication
The "Knowledge Space - Elementary Particle Physics" has been developed in 2012 within the framework of the project "Public Science and Digital Knowledge Spaces". The project was a cooperation between the ZAK | Centre for Cultural and General Studies at Karlsruhe Institute of Technology (KIT) and Merz Akademie Stuttgart, with funds from "Kreativitäts- und Innovationsring Baden-Württemberg". For the project, the existing Open Hypervideo Player has been improved both in terms of usability and project-specific functional requirements.
All 21 Videos have been shot in 2011 as part of the project "InsideScience – Public Science in Collaborative Research Projects", supported by the "German Research Foundation" (DFG). The goal of "InsideScience" is to make complex scientific research accessible and comprehensible to a broad public.
Resources
Use Case Description
Open Hypervideo Project
InsideScience Project (archived)
InsideScience Knowledge Spaces (archived)
ZAK | Centre for Cultural and General Studies
Merz Akademie Stuttgart
Responsibilities
Concept, Redesign of the Open Hypervideo Prototype for the means of Science Communication, Implementation
Open Hypervideo2012
Use Case: Document Access
Open Hypervideo is about interpreting the organizational concept of web-based hypertext for a film environment. The theoretical background, as well as the general concept, are described in my Bachelor's Thesis. This prototype was designed and developed as a general prototype for the idea of open hypervideo. The annotated documents, as well as the film scenes stem from the premises of the Archives of the City of Bruxelles.
The project aims to re-visualize the document context within the archive, as well as the invisible connections in between archival records (the archival methodology). The overall idea is to use film fragments as organizational structure for the archive of annotated documents. First ideas for this project have emerged within several semesters of research and experiments within the fields of interactive film and archives under the supervision of Prof. Mario Doulis (Merz Akademie). The prototype is continously being improved and developed further in different follow-up projects (more info can be found at the Open Hypervideo Project Homepage).
Resources
Use Case Description
Bachelor's Thesis
Open Hypervideo Project
Archives de la Ville de Bruxelles
Reverse Remediation Project (german)
Responsibilities
Student Research on Hypervideo Interfaces & Document Management, Design of a general Information Architecture for Open Hypervideo, Interface Design, Implementation
Semantic Media Explorer2011
Project mediaglobe - the digital archive
The cultural memory is storing an increasing amount of information and data. Yet, at present, only a negligible portion of these contents is searchable or available via digital channels. Taking media archives into the digital future, the mediaglobe project aims at providing digital availability and usability for the growing stock of audiovisual documents concerning German contemporary history. The Semantic Media Explorer combines the latest media analysis processes, such as automatic scene segmentation, intelligent character recognition, and the ability to recognize genres and faces in videos. The information extracted via the media analysis is correlated using semantic information so that users can identify new links and "rummage around" in the multimedia content. mediaglobe has been promoted from 2009 - 2012 by the Federal Ministry of Economics and Technology within the framework of the research programme »THESEUS – New Technologies for the Internet of Services«.
Resources
About the Semantic Media Explorer
mediaglobe - Project Homepage (german)
About THESEUS
Responsibilities
Research on User Interfaces & Interaction Paradigms for Semantic Video Search Engines, Conception, Prototyping, Interface Design, Frontend-Development
Video-based Archive2010
"remediate" Research Project
The video-based archive was created within a course by Prof. Mario Doulis. During the course I developed the first ideas for follow-up projects that dealt with annotated film. The basic concept is to use interactive videos as an interface to a document archive. In the Webinterface, documents are visualized in something one could describe as a timebased "virtual bookshelf". Technically the project was implemented with PHP and MySQL, with an Adobe Flex Application as the frontend.
Resources
Responsibilities
Student Research on filmic forms of archive access, Conception, Interface Design, Implementation
