e-Dance project

Research Questions

A key problem in the academic field of dance is how to capture and document the incremental development of ideas and their material manifestation in the creative process within practice-led research. In improvisational, embodied investigation, the mode of engagement is generative, pre-verbal, intuitive, experiential and fluid. This militates against types of cognitive engagement necessary for analysis, critique and reflection. The problem is most acute in the context of dance: however it is pertinent to all arts-based disciplines. This project is predicated on dialogic processes between dance and e-Science and the fluidity of concepts as they transverse the two domains, making use of recent advances in the visualisation and representation of spatio-temporal structures and discourse.

Two intersecting key research questions are central to the project:

• What unique opportunities does the distributed Access Grid environment provide for developing new approaches to choreographic composition and process and within this context how can we find new, appropriate and meaningful methodologies for capturing and modelling practice-led research?
• How can choreographic knowledge and sensibility help to shape e-Science practice to make its applications more usable within the field of performance arts practice-led research as well as the broader Arts and Humanities context?

The first key question addresses new creative possibilities provided by this distributed, collaborative environment, with multiple sites of performance and spectatorship. How might this redefine choreographic understandings of embodied spatio-temporal relationships? The Access Grid (AG) problematizes existing discreet understandings, blurring the boundaries between: live dance, pre-recorded screen-based dance, telematics, and hybrid integrations of live and projected dance material. Significantly, dance applies an aesthetic frame to the Grid, therefore the interface that literally frames the content and exchange between participants needs further consideration and design in order to offer flexible usage for arts practice.

We will explore notions of interactivity in the distributed context, in particular, whether existing tracking software can be incorporated into the AG environment to mediate live streamed video material. The multi-perspective nature of the environment throws into question the position of the choreographer’s authorial voice as arbiter. When working in a distributed yet collaborative environment how can participants take account of the paradoxical subjective position of being alone/separate, yet together?

The project examines how Grid-based hypermedia and semantic annotation tools might support, enhance and document creative processes. It is envisaged that these tools will be deployed as a means of capturing and rendering visual, the discursive/dialogic practices inherent in the choreographic process, whether that be participants’ internal dialogue or discursive practices between choreographer and performer. The visual nature of the interface employed by these e-Science documentation tools enables exploration of possibilities for pre-linguistic, multi-layered, non-linear representations of process, more aligned with the characteristics of the creative process itself. This reiteration of process will be considered not just as a static archival document of the process, but as a dynamic source of material that can be redeployed, for example, as a site for forensic, archaeological investigation; a score for further developmental commentaries generated through practice; or pre-recorded audio/visual content for re-use in hybrid distributed performance.

The second key question focuses on how choreographic knowledge can shape e-Science thinking and practice. In practice-led research, experiential knowledge is emphasised as a central epistemological frame. The embodied nature of the activity implies a pre-requisite level of physical skill with attendant features such as an enhanced sense of proprioception and heightened awareness of corporeality. These attributes have considerable significance for recent developments towards the notion of “experience design” as a means of improving the interaction design of e-Science technologies. The nexus of embodied, compositional understandings of spatio-temporal structure enables further consideration of concept-mapping practices and the visualization of spatio-temporal distributions of information and ideas, challenges currently concerning the project’s e-Scientists. Common to both disciplines is an interest in the making and marking of spaces: conceptual, narrative, virtual, embodied. The reciprocal impact of the two disciplines in developing new understandings of space and the subjective experience of space are fundamental to this proposed investigation. Can the project build new trans-disciplinary understandings and applications of spatio-temporal concepts such as narrative, trace, and memory?

Software Deliverables

This project proposes software developments to facilitate multiple stages of the practice-led dance research lifecycle. It proposes deliverables that meet artists’ needs for performance in a cinematic Access Grid context; documentation of the performance process; and dissemination of research results. These software developments are: support for enhanced video features within the Access Grid; flexible and powerful video annotation for performers; enhancements to support the planning of dance pieces in virtual spaces, and the capture of rich, multi-level annotations and session metadata; tools to enable performances to feature rehearsal and past performance records; editing and conversion software to allow Access Grid recordings to be presented in common media formats; and an interface to data Grid technology to host the potentially huge amounts of recorded material generated by the performance and documentation process in an Access Grid context.

Enhancements to Access Grid Video

The needs of performance artists using Access Grid as a medium are not well served by the current inflexibility of the video tool. Therefore, we will ensure that the Mbone tool VIC (used for video within the open source Access Grid toolkit) supports arbitrary video resizing, full screen video, chromakey-like functionality and other similar enhancements. Note that some of these developments have already been implemented by other groups [e.g. Darren Edmundson of The Australia National University] and some are on the SUMOVER project’s wish-list for integration into VIC’s common code base. Therefore they may already be in place at the start of the project. Our role will be to perform any outstanding integration tasks, to ensure usability by this community, and to further develop the tool along similar lines.

Enhanced ScreenStreamer Software

ScreenStreamer was developed within Memetic and is a tool for streaming a user’s desktop to other participants in an Access Grid session that is recorded along with other video and audio in the session. Although the tool was designed (like the rest of Memetic) to expedite the meeting process, it has potential application in this domain, such as in the streaming of pre-recorded material into a performance. We propose to extend this tool and improve it to offer better performance and usability, especially for its anticipated usage scenarios in this domain. Specific developments are improving performance by the integration of more efficient codecs (e.g. H.261, H.264), the exploration of platforms other than Java Media Framework, and user interface enhancements such as the selection of particular windows to stream (as opposed to the whole desktop). We will ensure interoperability by making streams accessible to both the open source and proprietary Access Grid clients (VIC and inSORS).

Choreographic Planning Tool

Any representation of choreography must be capable of expressing the three spatial dimensions plus a fourth for time. Further complexity is added by networked physical spaces, with all the possible interactions between them of performers and media. We anticipate, therefore, that the cognitive demands of planning and rehearsing such pieces will be considerable, and could be relieved by interactive representations that help choreographers and performers visualise event lines. We will therefore test ways of organising icons (representing performers and multimedia fragments) temporally and spatially within Compendium’s interface. 3-dimensional, zoomable user interfaces (ZUIs) are one design option that will be considered, if ways can be found to exploit effectively the principles of spatial perception and cognition, with acceptable end-user learnability, and responsivenes when the software is run on standard laptops.

Portable Video Mapping Studio

Artists working in locations other than Access Grid nodes would benefit from a better tool to record, replay, annotate, argue about, and disseminate video. We will integrate a digital video player into Compendium to create a self-contained ‘Video Mapping Studio’ which can be used on a non-networked laptop in which: users can import movies in standard digital video formats (e.g. AVI; QuickTime) from their cameras; Compendium can be used to annotate video as the material is being performed, and subsequently; these private resources can then be synchronised with a Memetic server, to enable remote access, annotation and replay capabilities.

User Interface to Support the Performance Process

Documenting the process of rehearsals and performance will be greatly served by current capabilities of Memetic and Compendium; however, these tools were designed for more formal meeting-like contexts. We will enhance the user interface of these tools to be more appropriate for the needs of the practice-led dance community. We will develop user interfaces that can capture performance and rehearsal metadata and rich, multi-level annotations that meet the needs of the performance art domain.

Support for Memory Input into Performances

Memetic and Compendium will be used to allow performances to feature rehearsal and past performance records, so allowing research into the blurring of the boundaries between performance and documentation. There are user interface and other areas of development required for both Memetic and Compendium to enable this functionality.

Memetic Support for Common Media Formats

The dissemination of performance work is an important part of the research lifecycle. Whilst the RTP format of Access Grid media streams that is stored by Memetic is appropriate for real time collaboration systems (the format is also used for H.323 traditional videoconferencing), it is not readily suitable for, say, showing as part of a PowerPoint presentation, or to dump onto DVD. The conversion of RTP to common media formats (such as AVI) is not simply a technical development, as there is an editing process required (Access Grid sessions have multiple – often many tens – of video and audio streams that must be condensed to a single feed to be played by Windows Media Player, or QuickTime, etc.). We will develop editing functionality as part of the Memetic software to convert the RTP streams to a format playable by common media players. A priority will be the development of a highly usable interface to enable this.

Interface to Large Data Stores (SRB)

A typical Access Grid session generates large amounts of media data. For example, a session involving 3-4 sites each with three or four video streams, a single audio stream and a desktop sharing session may generate about 4GB per hour. As Memetic records all of this data, a large hard disk capacity is required for the server. In this project, Memetic will be used to record whole days of performance rehearsals between multiple sites that may also include the transmission of other media streams, such as pre-recorded video. These types of scenarios potentially entail massive storage requirements beyond what is reasonable from a single Memetic server installation. We propose to allow Memetic to take advantage of heterogeneous data resources available on a data Grid and thereby overcome scalability issues, as well as implicitly offering a solution to problems of backup. We will use Storage Resource Broker technology implemented on resources provided by the National Grid Service. There are interesting technological issues arising from both the storage and subsequent playback of recorded media streams that are not stored locally. For example, because timings for data retrieval over a network are not guaranteed, there are issues for the smooth playback of a remotely-stored recording that may require solutions such as the caching of data on a local disk.