Creative Education
Vol.07 No.16(2016), Article ID:71581,9 pages
10.4236/ce.2016.716238

Sensemaking: A Proposal for a Real-Time on the Fly Video Streaming Platform

Jane de Almeida1, Maria Amelia Eliseo2, Cicero Inacio da Silva3, Helena Prates1, Vic Von Poser1, Brett Stalbaum4, Nilton Gomes Furtado3

1Laboratory of Cinematic Arts and Visualization (LabCine), Mackenzie Presbyterian University, Sao Paulo, Brazil

2Department of Computing and Informatics (FCI), Mackenzie Presbyterian University, Sao Paulo, Brazil

3Telehealth Networks Program, Sao Paulo Federal University (UNIFESP), Sao Paulo, Brazil

4Walkingtools Laboratory, University of California, San Diego (UCSD), La Jolla, CA, USA

Copyright © 2016 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0).

http://creativecommons.org/licenses/by/4.0/

Received: September 25, 2016; Accepted: October 24 2016; Published: October 27, 2016

ABSTRACT

The massive amount of data produced by the streaming and storage of videos generated from classes, medical procedures and other science and educational films using UHD technologies is creating huge databases that are not fully used. For example, the preparation of a surgery and the setup of a class which are streamed in real time are parts of videos that could be extracted from the final version before being saved in the system. Usually online video platforms only permit users to edit streamed videos after the end of the streaming. This article describes a project of a video editor called Sensemaking that can be applied to the educational field, allowing users to edit on the fly videos that are being streamed in real time by creating an online editor feature to manage in real time the video input, providing the users with a tool to produce an educational or scientific movie. By using the Sensemaking system, it will be possible to produce a movie which will be a record and at the same time an educational product with audio, images, logos and animations at the end of the streaming session. This editor will be initially integrated to the Video@RNP Platform, a database of videos dedicated to educational and research institutions in Brazil. The Video@RNP Platform is operated and developed by the National Education and Research Network (RNP) with financial support from the Education and Science, Technology, Innovations and Communications (MCTIC) Ministries in Brazil.

Keywords:

Online Education, Video, Video Editor, On the Fly, Streaming, Visualization, Big Data

1. Introduction

Video streaming has become an important element for contemporary research because it allows that an activity can be shared, discussed and evaluated by several researchers at the same and in real-time (Lima, 2012; Yoshitaka & Deguchi, 2009; de Almeida et al., 2014) . However, the streaming not only creates an intense volume of data but also generates the need to increase hardware resources for storing the recordings, and also the streaming generates extensive “raw” materials without any viewable narrative. The real-time transmission of moving images with ultra-high resolutions (4K, 8K) results in excess of images and data that make storage and retrieval of data (content) stored very complex and expensive (de Almeida et al., 2011; Weekley & de Laat, 2016; Liu et al., 2011) . Usually, a significant amount of videos are daily generated for research and teaching activities, such as surgeries, advanced visualizations, lectures and experiments records and the edition of a raw video demands hours, often invalidating its use due to the time spent on viewing or researching a desired content. The goal of this article is to demonstrate the development of a video editor that can edit on the fly videos that are being streamed live in real-time. To our knowledge, this is the first known live streaming video editor.

2. Making Sense of Big Data

With the advent of digital cinema, inventive procedures such as those discovered in previous cinemas now stem from a computational universe involving a rather extensive and complex production workflow which includes also advanced scientific data visualization that can be streamed in real-time: a sort of return to the photograph (discrete media), due to its typically static interface (as in, viewers seated in front of a projection), although it is capable of interactivity. In view of this large databases and also of the necessity of producing moving images that can explain in a meaningful way tons of detailed data that are created and streamed online by the scientific visualization (Stalbaum & da Silva, 2016) researchers, this research aims to study and to develop a real time (on the fly) video edition platform for streaming videos with the purpose of providing them with means to create educational videos that makes sense to a broad audience (Stringhini, Eliseo, de Almeida, & da Silva, 2015) . The platform is called Sensemaking and will work as an educational online feature to be connected and implemented into an existent video platform. The main goal is to create a video editor that can be made available at the Video@RNP repository. This video platform is provided by the National Education and Research Network and has been used by public universities and academic institutions in Brazil as data storage for videos and also as a video repository for the virtual classes recorded by Brazilian universities. The aim of this article is to describe a production tool with which the responsible people for the teaching or research activities that uses real time video streaming tools can, at the same time, transmit the content by using streaming features and also to be able to edit and select parts of the material, generating a final product with coherent narrative for future use. In the available streaming services the user needs to store the entire content after finishing the streaming to start using the editing features provided by some companies. The Sensemaking platform provides a differential ability to perform online editing during the streaming, or consecutively to the stream, optimizing the post-production time. As a development strategy, the project will produce an adequate editing methodology to stream, analyze and prepare an editor interface to simulate an editing software. This article describes the expected impact of this proposal in the research and educational field, the necessary requirements for its implementation and presents the preliminary results. The Sensemaking project has been developed by an interdisciplinary team of researchers, such as computer scientists, software developers, filmmakers and educators specialized in online classes. The inspiration of this project is based on the certification of the fact that producing science, cinema and technology are connected and all participants have important equal roles in the process. Just like “filmmaking has moved from the position of being a technology serving science to being an equal partner and reflexive maker of science.” (Gouyon, 2015) , the necessities of the new media for education can be thought as a displacement from the position of receiving technology and making a movie to make technology software in order to make a certain type of movie. Meaning that ‘the filmmaking apparatus’, and not only films, are also composite objects, whose epistemology is as much determined by their subject matter as it is by the medium ( Van Dijck, 2006 according to Gouyon, 2015 ).

3. Motivating Scenarios

In order to analyze the need to implement the Sensemaking project, we are going to demonstrate three scenarios in which the post-production of online streaming classes videos take a long time after the recording of a video streaming. The examples show normal video users capturing and streaming a raw footage to be post-processed. The production of the final video usually demands a review of the footage―cropping or addition of relevant information―before public disclosure in online educational classes.

The purpose of these scenarios is to show that these situations can be used to demonstrate some issues related to the fact that it is necessary for some educators the use videos for their classes. Normally the teachers or researchers do not have the necessary software to edit the video database stored after a class, nor computational resources to open the files and edit them. On the other hand, some features are also needed for professional use, in order to finalize an audiovisual narrative with the content obtained during the shooting or the streaming, i.e. the Sensemaking platform can realize, during the recording or the streaming, the post-production process for online virtual classes.

3.1. Scenario One: Educational Content Manager

Mateus is a professor of Neurosurgery and will watch a surgery in the Surgical Center. He wants to shoot the entire surgical procedure to discuss with his students in the classroom. Jussara, his assistant and a resident student, install the camera and starts recording from the preparations of the patient until the end of surgery. The surgery takes five hours of video recording.

For teaching purposes, Jussara chooses some relevant scenes and assemble the final video, which should last no more than 15 mins, making cuts, adding captions and producing an opening with logo, description and credits. Jussara has basic knowledge of video editing and uses a professional video editor. She watches again all 5h shooting to choose the most relevant scenes. She probably spends 6 to 10 hours in the process. If Jussara doesn’t do this work, the video result will be rough and hardly any researcher or professor will be interested in watching it. It is believed that the teacher rarely uses the raw video in the classroom because probably he loses some important amount of time looking for the scenes of interest to his audience formed by medical neurosurgeons residents.

3.2. Scenario Two: Educational Content Curator

Silvana, a data structure researcher, coordinates a roundtable at a Computer Seminar in a University Campus. Six experts will attend the table and present their ideas in 15 minutes for each presentation. After the presentations the participants are invited to discuss their views with the audience and the participation will be through a Q&A. As this venue is important for the university, Silvana asked the event organizers to broadcast the seminar online, getting to the end of a video with 160-minute record. Silvana would like to use it as a source of knowledge along with its researchers and make it available online to people who would not be able to attend the event, even by streaming.

Usually, researchers or students are interested in only in parts of the lecture, for example: where only one expert related to his research appears. To facilitate this process of searching for the desired presentation, Silvana asked Guilherme, who is a video editor, to split the film, separating the talks of each expert, generating other 6 films related to the topics presented at the roundtable. For this edition, Guilherme will watch the entire movie, not only separating what was requested, but also removing unnecessary information that a live streaming is subject. To obtain a searchable material in the future, Guilherme must enter subtitles on the lecture and the name of each guest.

3.3. Scenario Three: Educational Content Publication

Roberto is a film technician and is filming a neurosurgeon perform a brain surgery for a class of medical residents. This video material will be used not only to analyze the movements of each detail, but also to identify them afterword. At the end of filming, he will inserting subtitles in selected scenes identifying the name of each movement. This material will be used for classes at a Medical School.

To produce this film, he will have to review all the movie and enter these texts with a digital video editor, what will take a long time for him to post-produce this film. In the example scenarios above, a video editor platform such as the Sensemaking can optimize the video editing time, integrating the recordings along with the post-production and delivering a material with narrative after the shooting or streaming. The on the fly editing and post-production will allow users to insert subtitles and images, make cuts or improve the image, all in favor of better conditions to produce videos that can be view-

able and understandable just after the end of a stream or shooting. In the first scenario, using the Sensemaking platform, Jussara can edit the material a few minutes after the shooting. At the end of the procedure, she will have the video already edited. In the second scenario, Guilherme can make appropriate cuts and inserts in the streaming and a few minutes after the round table, the material may already be available to viewers. The same happens in the third scenario where Roberto can perform metadata inserts during the streaming with the Sensemaking editor, creating the final video also a few minutes after shooting and publishing it at a medical database for the residents to have access to the videos.

4. The Best Known Platform for Streaming

The contemporary surge in the demand for digital and online classes has created a new scenario for educators. First, it is necessary that the educational realm can get used to the use of new media features to provide to students the necessary means for them to have access to the content that the teacher wants to transmit. With the use of interactive online videos, these results can be achieved in some ways. The other feature that professors and teachers can use, if the infrastructure is available, is the streaming feature. This because a class that is streamed in real-time can provide to students a contact with the teacher, almost a face-to-face class, and also in some classes, depending on the system, students can perform questions in real-time to the teacher. Systems that allow students and teachers interact in real time are among the existing tools with streaming possibilities on the web, the best known are: a) Youtube; b) Kaltura; c) XSplit Broadcaster. Some of the cited tools only transmit video, others are able to stream and storage the content for later viewing and editing, but only if the user wants to edit the material in post-production. Those platforms with video editors allow users to insert clips, mark the beginning and end of a video and crop unwanted materials. You can also add a soundtrack. Below will be a brief description of the most known streaming, some with editing properties:

4.1. Youtube

The most widely used platform for streaming, it allows the edition of the broadcasted material after the end of the streaming. However, YouTube does not allow consecutive edition of the broadcast content, i.e. you cannot insert audio, cut or alter any content consecutively to the streaming. To stream content on YouTube it is necessary to have specific tools that make the connection between the camera, the microphone and the compression system that Youtube uses. For exemplification purposes, the XSplit Broadcaster tool provides screen capture features, streaming transmission and the addition of plugins that help in post-production.

It is also more intuitive and already adapted for use with YouTube. Another option is to use the Quik (G + Hangout on Air), which performs the streaming content through the use of the Hangout tool and, when transmission via Hangout is completed, the content can be made available on YouTube. For those who want a quick editor, YouTube is focused on online broadcasts. It has art features such as CG (Character Generator), opening effects and basic transitions, basic image enhancement, cutting tools and audio libraries available for use. Its interface shows a database of the clips (which requires a previous upload to YouTube videos storage site) and also a non-linear timeline of audio and video.

4.2. Kaltura

The Kaltura platform has both a commercial and an open source version with a choice of plug-ins for post-production and web modules apps, such as integration with CMS (Custom Management System) such as WordPress (VideoPress), CMS Joomla, among others. It also has a system that generates the display protocol videos, and usually each tool generates its own viewer, which facilitates the display of the video stream in both web browsers as the screens of mobile applications. This view allows you to view the real-time transmission of any content transmitted through their applications. According to its website, the Kaltura system (kaltura.com) is widely used by universities and colleges around the world.

4.3. XSplit Broadcaster

Streaming video classes system through which you can send multiple streams from a variety of sources (cameras, screen capture, etc.) The compression and player system does not include reading many formats and, in that sense, you will need a software that makes the “reading” of the content and present it in a specific output, for example, via YouTube. However, this software does not allow users to view a timeline with frames being accumulated one after another (the acquisition of frames) of the content in real-time. This streaming tool usually also does not generate audio files separately, only allowing the edition, a posteriori, in some other tools available. Another deficiency of this software is the fact that there is no buffering, allowing the user to decide what you would like to save and “publish” in real time from the stream that is conducted via YouTube, for example (Table 1).

5. The Sensemaking Project

The system platform hardware and software architecture for education consists of four servers. The server roles are described below (Figure 1):

a) A database server for storing videos and information from the CMS (Content Management System);

b) A web CMS (WordPress and Joomla);

c) A streaming video CMS (Red5 and Kaltura);

d) A video CMS (Vimp).

WordPress and Joomla are CMS for web, written in PHP with MySQL database, targeted mainly for creating web pages. Both are flexible in production and development of websites. Red5 is an open source media server application used for streaming and includes Flash, HLS (HTTP Live Streaming), WebSockets, and RTSP (Real Time Streaming Protocol). It has a flexible architecture that allows the customization of audio and

Table 1. Streaming software comparison.

Figure 1. Sensemaking platform architecture.

video. The Kaltura system provides an open video platform with hosting services, enabling publication, management and analysis of videos and other media content. It is compatible with the CMS WordPress and Joomla.

VIMP is a video CMS with support for HTML5 and allows the creation and management of video web portals.

Differently from other video platforms, as already described in the article, the Sen- semaking system (Figure 2) is the only one that can provide to researchers, educators, teachers and professors, a tool to stream live classes and experiments and, during the streaming, provide a tool to edit the video on the fly in real-time and, during the streaming session, provide features to create a video, with description, titles, subtitles, images, audio, allowing non-professional educators and researchers the creation of video classes. This said, right after the end of a real-time streaming the educators will have a video object that will be ready to go online, eliminating post-production costs and time spending in editing the raw materials. The Sensemaking architecture can be applied to on-line video systems, such as the platform video@RNP. It works like an app, through which the user can start click and start the editing of the streaming, cutting parts of the class that are not interesting for the final video by using the tools provided

Figure 2. Sensemaking streaming editor interface.

in a regular video. The system is very simple to use: when a video streaming starts, the user can see in real-time the frames of the video in two screens. The first screen is the on-line video that the users can see online. The second screen is the video that can be edited on the fly by the users. In this second screen, the users sees a list of frames (24 image files per second) and he can decide in real-time what he wants to keep and what he wants to remove. The user can also add a title before the end of the streaming, along with the metadata related to the content of the video. The main idea is to create meaningful videos at the end of a streaming session. At the end of a surgery that took 6 hours, the researchers can just cut the parts that are not interesting and, minutes after the end, they can have in hands a finalized film ready to be distributed to the students (off or online).

6. Conclusion

The process of assembling and editing an educational video is basically divided into select, order, and adjust plans or time in order to achieve the expected result, i.e. a result that makes sense for a student. Before the digital age, we can compare the editing process to writing a text on a typewriter. Before typed, the text should be well outlined, drafted and revised; any subsequent changes to the text on paper was complicated, time consuming and costly. This metaphor can also be applied to an educational film edition. Usually educators, professor and teachers, before starting working on an edition of a film, they should know in advance all the steps that they should follow in order to achieve the expected final result in post-production. Early in the film industry, these were physical processes such as cutting and pasting, i.e. not available to a regular teacher in a class. With the evolution of technology, recordings began using magnetic tape (BETAMAX, VHS), to select and re-record, which caused loss of quality material. With these processes there were no opening for mistakes or chance, everything had to be thought out and planned in advance. With the Sensemaking educational editor, we would like to introduce a way to produce educational films applied for science, technology and culture from streaming contents that usually will be lost or just stored in a database where no one would have time to spend looking into to find a specific topic for research.

Acknowledgements

This work has been supported by a grant from The Brazilian National Education and Research Network (RNP), a Social Organization (OS) bonded to the Brazilian Ministry of Science, Technology, Innovations and Communications (MCTIC) and maintained thereby together with the Ministries of Education (MEC), Culture (MinC), Health (MS) and Defense (MD).

Cite this paper

de Almeida, J., Eliseo, M. A., da Silva, C. I., Prates, H., Von Poser, V., Stalbaum, B., & Furtado, N. G. (2016) Sensemaking: A Proposal for a Real- Time on the Fly Video Streaming Platform. Creative Education, 7, 2515-2523. http://dx.doi.org/10.4236/ce.2016.716238

References

  1. 1. de Almeida, J. et al. (2011). Tri-Continental Premiere of 4K Feature Movie via Network Streaming at FILE 2009. Future Generation Computer Systems, 27, 924-934. [Paper reference 1]

  2. 2. de Almeida, J. et al. (2014). Integration of Fogo Player and SAGE (Scalable Adaptive Graphics Environment) for 8K UHD Video Exhibition. Journal of Computer and Communications, 2, 50-55.
    http://dx.doi.org/10.4236/jcc.2014.212006 [Paper reference 1]

  3. 3. Gouyon, J. B. (2015). Science and Film-Making. Public Understanding of Science, 25, 17-30.
    http://dx.doi.org/10.1177/0963662515593841 [Paper reference 2]

  4. 4. Lima, B. E. S. d. et al. (2012). Automatic Video Editing for Video-Based Interactive Storytelling. 2012 IEEE International Conference on Multimedia and Expo (ICME), Melbourne, 806-811. [Paper reference 1]

  5. 5. Liu, S. et al. (2011). CineGrid Exchange: A Workflow-Based Peta-Scale Distributed Storage Platform on a High-Speed Network. Future Generation Computer Systems, 27, 966-976.
    http://dx.doi.org/10.1016/j.future.2010.11.017 [Paper reference 1]

  6. 6. Stalbaum, B., & da Silva, C. I. (2016). The Sensor Network Is the Computer. The Internet Freedom Forum Conference, Valencia. [Paper reference 1]

  7. 7. Stringhini, D., Eliseo, M. A., de Almeida, J., & da Silva, C. I. (2015). Big Data, No Data. A Proposal to Manage Digital Scientific Films Repositories. Proceedings of the Webmedia Conference 2015. [Paper reference 1]

  8. 8. Van Dijck, J. (2006). Picturizing Science: The Science Documentary as Multimedia Spectacle. International Journal of Cultural Studies, 9, 5-24. [Paper reference 1]

  9. 9. Weekley, J. D., & de Laat, C. (2016). CineGrid, High Quality Media Streaming and Processing on Advanced Photonic Networks. Future Generation Computer Systems, 54, 292-295.
    http://dx.doi.org/10.1016/j.future.2015.07.011 [Paper reference 1]

  10. 10. Yoshitaka, A., & Deguchi, Y. (2009). Rendition-Based Video Editing for Public Contents Authoring. 2009 16th IEEE International Conference on Image Processing (ICIP), Cairo, 1825- 1828. [Paper reference 1]