Augmented Reality Authoring

Location

This project was done as a research internship at Big Data Experience Lab, Adobe Systems, Bangalore during May ‘18 - Jul ‘18, the summer of my third year.

Details

During the internship, I worked as a Research Intern, among a team of 3 members and in close coordination with multiple research members of technical staff at the company, to accomplish the above-mentioned project. A general area of interest was assigned, and a specific problem statement was chosen in that area after an extensive period of brainstorming. Having decided on a problem statement, methods were devised to come up with a feasible solution for the problem. Multiple NLP techniques such as scene graph parsing, coreference resolution, and others were utilised to understand text, along with learning multiple MLP networks for certain tasks involved in the solution pipeline. The augmented reality aspect of the project was realized by using the ARCore library provided by Google, in the Unity3D framework.

The final solution pipeline included multiple methods leveraged from existing works and improved upon, as well as novel methods for scene augmentation and a MLP prediction model that were introduced for the first time.

Using these multiple techniques intelligently in a pipelined manner, a first-of-its-kind novel end-to-end interface was developed that allows easy authoring of augmented reality experiences from natural language input. An internal survey was conducted, which temporarily validated the genuinity of the interface, and conducting the final crowdsourced survey is work in progress.