OVRAS | an application service relay for ERT
|about protocol gallery previous downloads team ERT|
|ERT CONCIERGE @ VIRTUAL WORLDS BEST PRACTICES IN EDUCATION|
[right] external services
The gaming industry has catalyzed the fast-paced advancement of immersive Virtual Reality (VR) technology. However, there exists somewhat untapped educational benefits in the realm of VR.
A VR classroom can create a shared virtual space that mimics in-person teacher guidance and student collaboration without the obstacle of physical distance. Additionally, the VR classroom creates an opportunity for a new brand of "lab" exercises only feasible in a virtual, immersive 3D space.
ERT WEB +
ERT CONCIERGE +
ERT WEB +
ERT CONCIERGE +
Using low-level networking interfaces, I have also created a server-client protocol in order to incorporate external application functionality into the VR space and create a lab exercise for visualizing multivariable mathematical functions. Through the generic plugin architecture of the server's design, this project allows intuitive integration of new labs with extensions in Python, MATLAB or other third-party services, facilitating future collaboration with educators.
Quad Mesh (surface) generated with vertices and normals from external Matlab service, and sent back to client alcove.
The ERT (Enhanced Reality Teaching) classroom was originally built on the Unity game engine, and written in C#, using the packages SteamVR and Photon Unity Networking (PUN). In 2019, it was updated to work with the latest version of Unity, SteamVR, and PUN in order to ensure future functionality. In 2020 the ERT Concierge that allowed it to communicate with outside services was updated, and a new range of clients were added to the ecosystem.
In fact, ERT has been extended to allow users to easily integrate their own 'labs' (as applications or simulations) directly into the classroom space. Two example labs have been tested: a planetary simulation and a MATLAB calculator. These labs use socket communication in order to sync information with ERT through a CONCIERGE which handles both continuous updates (in the planetary simulation) and large packets of information (MATLAB calculator).
|"The ability to have a flexible collaborative open-source Virtual Reality application that can also easily communicate with any outside application and exchange any input or output (text, geometry, or raw data) is a game-changer!"|
From our research we were able to reveal and then focus on the following common educational issues:
Developing for Virtual Reality, as well as most Mixed Reality platforms, entails creating game like applications, that are highly dependent on specific harware architectures. Platforms like Steam, have generalized the interface between an application developement space (such as Unity) and a general approach to managing a VR/MR/AR headset and the inputs.
However, applications are still highly dependent on creating a monolothic tool/executable. We can adjust to this reality, by allowing ecosystems that could benefit from a VR display, by having them act as a service. With the creating of a middle service manger --a service concierge-- we can easily incorporate applications into an evironment where input and output is relayed via standard protocols (JSON based).
|# actionable insights|
|Interacting with an external application can be done through specific client designed user interface elements, or by interacting with objects in the scene that are equipped with sensors/colliders. ERT Concierge is designed to send small continous updates (such as in the case of the Python based Planetary Simulation) as we as large comprensive data (as in the case of results from the Matlab Calculator).|
input from the front end client is wrapped in generic JSON, and concierge
transmits it to the registered service. understanding how to apply the information
is not at the service side through call to provided/supported APIs.
[currently supported] python and matlab
[left] student, in alcove, changes value of mass for center sun
[right] synchronized simulation results for all participants
_GENERAL CLIENT PROTOCOL
In 2020 we updated ERT Concierge to efficiently tie in clients (of different types) with a range of services. Both clients and services connect to ERT CONCIERGE with a common socket and simmilar protocols, but depending on the registration and context will have different protocols/payloads available. SEE PAYLOAD PROTOCOL
Furthermore, in order to support larger packet/data transfer, we created a file system which stores data with disticint indentifiers and is accessible to registered clients via standard http protocol. SEE FILESYSTEM USAGE
_ project lead : santiago v lombeyda
_ surf 2020 | efficient network services & new web base client : an tran
_ surf 2019 | network services & python realtime app synchronization : alison noyes
_ surf 2019 | network services & matlab large data app communication : sasha fomina
_ center director : george djorgovski
_ george djorgovski
_ jim barry
_ mathieu desbrun
PROJECT BORN from CALTECH's OVRAS LAB
_ with support from the CENTER for DATA DRIVEN DISCOVERY
_ with support from Caltech's SUMMER FELLOWSHIP PROGRAM
_ with donations from MICROSOFT
_ with donations from LOGITECH
_ with donations from NVIDIA
_ with donations from HTC
|ovras | OVS | art by jimBARRY | CD3 | caltech|