Interact/ND --

A Distributed, Multi-User, 3D-Immersive Development Platform and Design Environment


Michael Shilman


Goal and Objective

The goal of the Interact/ND Browser project is to provide a set of tools to facilitate 3D-immersive, multi-user environments across the Internet. Such environments will serve a variety of purposes, ranging from effective and intuitive net browsing to communications to powerful collaborative design applications and top-notch entertainment interfaces.

We are interested in creating not just "another browser", but rather an environment by which people can more completely interface with both their own computers and with data from the network. This includes supporting a variety of user peripherals for high interactivity and providing a system by which client applications can utilize all the graphics capabilities and interactivity that the browser provides.

Our particular interest is in electronic system design as we believe that networked, collaborative, immersive CAD software is an unexplored area of research with great potential to enhance the productivity of design engineers.


Overview


Figure 1: High level diagram of Interact/ND

At the user level, Interact/ND allows the browsing and retrieving of VRML files, objects and associated links located across the Network. Within the user interface (UI), users can explore and view 3D models using immersive VR devices as well as perform a variety of scene manipulation and editing functions. A Multi-User environment is also supported where collaborative work can be performed.

To extend the multi-user, immersive environment of Interact/ND to a distributed design system, more system-level support and plug-in applications are to be developed. The system will include a Simulation Engine which allows objects to have properties, behaviors and animation. there will also be a Decision Support system which helps the user to intelligently find information and components across the network and represents data in an intuitive and useful manner. There will be provisions for Customer Tracking which allows the management of operations and transactions by a central server. Finally there will be CAD Software and an associated Component Library (which will be developed by a number of parties) which takes advantage of all the seamless underlying capabilities (3D, immersiveness, multi-user interaction) which Interact/ND provides.


Functional Specification

The Interact/ND Environment combines some of the key elements in providing an infrastructure for the development of a distributed design system which includes 3D rendering, browser interface, immersive hardware support, multi-user environment and multiple platform support.


VRML Overview

The vehicle by which we hope to accomplish our goal is VRML, the Virtual Reality Modeling Language. We chose this because it is an open standard and that it provides a very general syntax for scene description. However, the current version of VRML (VRML v1.1) is a minimal specification -- it provides a basic language for describing hyperlinked, static, 3D scenes. We are working to expand this specification to give network objects properties and behaviors which will allow them to interact with each other for simulations, and will also allow them to interface with existing CAD programs running on local machines. We are actively participating in various forums (SIGGRAPH 95, VRML95, VRML Consortium), newsgroups (alt.lang.vrml, comp.graphics.api.inventor) and mailing lists (vrml-behaviors@sdsc.edu, vrml-modeling@sdsc.edu, www-vrml@wired.com) and watching closely the academic and industrial efforts to develop and extend the VRML specification.

Using SGI as our main development platform, we have utilized Inventor, SGI's object oriented 3D graphics and visualization toolkit, into the Interact/ND browser as it provides sophisticated 3D graphics primitives and a structure for adding our own elements to its simple database. Its simulation characteristics also make it a desirable software to use. At the same time, OpenInventor, a development of Inventor which is available on a wider range of platforms, is emerging as a de facto standard for the management and interchange of 3D scene information and has been used as the basis for the VRML extension to the World Wide Web. These are encouraging signs of increasing support of VRML as the future 3D modeling language.

Browser Organization


Figure: Organization of the Interact/ND modules

This figure shows a high level overview of the software organization. The scene is the abstract body of data structures and programs which represent the state of the world at any given time. The inputs modify the scene, and the scene managers are responsible for arbitrating these modifications. As shown in the diagram, the inputs can be roughly classified into two types: real-time peripherals and World Wide Web. The WWW will replace the entire scene when a user traverses a hyperlink. In contrast, the peripherals are constantly changing, so performance is a much bigger issue for these modules.



Figure 3: A snapshot of Interact/ND


Interact/ND provides the following features and capabilities:

Feature Capabilities
File Operations Open, Import, Save, Print, etc.
Scene Viewing Allows user to view objects using multiple modes (e.g. Fly, Walk, Plane) and different lighting and transparency levels
Scene Editing Copy, delete, cut and paste; object resize, re-orient, reposition, and shear through many different modes of manipulation (trackball, handle box, jack, centerball, transformbox, etc.)
Communication with existing HTML browser Interfaces with Netscape for display of non-VRML data
Hyperlink Data and object retrieval from remote sites
History Stack-style history of sites visited during the session


Currently, the user interface is inherited from a scene viewing demonstration software which we used as part of our code base. For future development and usage purposes, we find that it is not well-suited to our needs and plan to re-write a tailored interface using more sophisticated tools (Java) in the coming quarter.

Immersive Hardware Support

The support of immersive hardware devices is essential to an effective 3D design environment. We support several commercial and internally developed immersive devices, and have used our software to evaluate the usefulness of these devices. We will continue to broaden our device support as new we hear of or develop new devices which will enhance interaction with 3D environments.

Devices Supported
Device (with description and developer) Current Status/Capability
Spaceball:
A spherical input device which offers 6DOF relative position and orientation input, as well as several buttons. The Spaceball is produced by Spacetec Inc. 		Updates camera position/orientation
TRK300 Tracking System:
This device was developed within our group to track object orientation.		 Updates camera orientation. We attach this device to the top of an HMD to orient the scene camera within the scene.
TRK300 Joystick:
This device uses the same underlying technology as the TRK300 Tracking System.		 Updates camera position
VR4 HMD:
A high resolution (742x230) head-mounted display device. Early this quarter we will be getting a (640x480) upgrade called the VR5. These units are produced by Virtual Technologies. 		Displays the scene to the user in an immersive fashion. We can enhance this effect with the use of "heads-up controls" -- scene manipulation controls which reside in the scene or overlay the scene, so that the scene itself can take up the entire viewing area.
Fastrak:
A 6DOF device which uses absolute coordinates. They are produced by Polhemus Incorporated. 		We currently use this to update the camera's position and orientation because it was the easiest thing to start with. However we believe that this device is much better suited to direct scene manipulation and plan to implement this feature over the course of the next quarter.
Dataglove.
Please see more detailed information about our Dataglove research here.		 Will provide support coming quarter


Multi-User Environment

Unique Event Model

The Interact/ND environment possesses a unique event model which handles the network like any other peripheral, meaning that messages which are passed from other Interact/ND clients are treated in a similar manner to events from input devices. This allows us a great deal of code reuse, and enables easy incorporation of new input devices as necessary, with minimal modifications to the core code. It also has the inherent capability of supporting input devices across the network.

Multi-User Collaborative Virtual Environments on the Internet

On top of the event handling mechanism, Interact/ND has provisions for multi-user interaction. At this time, there is only support for motion updates through rudimentary UDP client-to-client communication, but a more sophisticated messaging system using MBONE multicast is under development. In the long run, we hope to add full voice communications so that multiple engineers can easily collaborate on design despite geographic separation.

Multiple Platform Support

Interact/ND was developed entirely on an SGI running IRIX5.3. It currently runs on SGI and is in the process of being ported to Windows NT and Windows 95. Once we have released a prototype (running on IRIX, Windows NT and Windows 95), and if there is sufficient demand, we may port the software to other popular platforms (SunOS, Solaris, DEC Alpha).

The current user interface is implemented using Xt; we plan to rewrite it in Java for true platform-independence over the coming quarter.


Ongoing and Future Possible Extensions

Decision Support

In the ever-increasing complexity of electronic design, especially when tools, components and services are distributed throughout the world, the need for an understanding of the nature of the relationships between components and their use in particular designs, related services, and various users or user groups, is essential. Therefore, we plan to develop a decision support application using the browser, with the capabilities of identifying and classifying various forms of dynamic and static information about the design and its environment, as well as display the information in a way of most use to the engineer.

On the system (database) level, issues and products regarding data browsing, data mining, "network makefile" will be investigated. On the user (graphics) level, data representation, or scientific visualization approaches will be explored to find an effective and intuitive way of presenting data and dependencies.

Customer Tracking

Traditionally, the "purchase-maintenance" model has been used in the EDA business. In this model, EDA firms sell licenses for proprietary design automation software to their customers and then charge annual maintenance fees. With the emergence of the Internet and a new mode of distributed design system, new charging methods should be adopted. This may be on a per-use basis (i.e. a customer gets charged every time he downloads an executable), per-transaction or time-usage basis, etc.. and thus the need for customer tracking arises. This will entail developing a client-server architecture where a central server logs clients transactions and performs necessary data processing and calculations. It will also require the development of a subsystem to enforce the security of this model.

Simulation Software

In order to be useful to CAD, Interact/ND must have provisions for simulation software. Initially we hoped to adopt some large simulation engine and place this beneath our browser to drive its simulations. With the incorporation of a Java interpreter into our core we can now afford to be more flexible.

We define two types of simulation software: heavyweight and lightweight engines. Heavyweight engines are computationally intensive or anything that must be native-code compiled to be effective. This software will be compiled into Java extensions and dynamically loaded by the Java interpreter when a lightweight engine calls for it. Lightweight engines are anything else, ranging from small code that acts as a wrapper for data (as in the case of a CAD component) to a simulation engine that is small enough that it can be written entirely in Java. Both types of simulation engines can co-inhabit the browser and are expected to work together.

There are numerous significant reasons why we choose Java over other languages. First of all, Java supports programming for the Internet in the form of platform-independent applets. Its simplicity makes the development of user interfaces an easy task, thus we anticipate its increasing gain in popularity and support of developers. It has a secure architecture and structured support for simulation which also make it fit nicely into the model of distributed design systems.

CAD Software and Component Library

Our target prototype application is a tool for MEMS CAD. We see this as a very positive first step for several reasons:

  • Visualization for MEMS is very suited to a 3D environment. The same cannot necessarily be said for other areas of the EDA industry, though we will expand our research to traditional CAD tools upon completion of the MEMS software.

  • MEMS is still an emerging technology. By investing in it now we avoid competition for standards with industry heavyweights, and can potentially develop a niche market for ourselves.

  • The sensors section of our group has a lot of experience with MEMS and has successfully leveraged MEMS technology in several applications. By developing tools for MEMS we can assist in their research efforts.

    In the long run we plan to develop a suite of CAD programs that take advantage of 3D capability of our environment and will heavily leverage existing CAD systems. If necessary, we will initially develop a component library for our CAD tools. These components/objects will have dynamic properties and behaviors and will respond graphically and propagate back to the simulation environment.

    We look to set open standards in cooperation with other academic and commercial CAD organizations. By defining open standards, we hope to gain popularity, adoption and support by these organizations in not only using Interact/ND as a design platform, but also in developing proprietary or shared components to be retrieved across the net.

    User Interface

    Device Support

    We will continue to investigate and perform research in new popular hardware devices to provide a more immersive and intuitive environment for engineers, in terms of better scene manipulation and more efficient data entry. We will also expand our device support to new and popular devices as we see the demand.

    Graphics

    There will be on-going work with UC Berkeley computer graphics groups to improve the user interface and graphics capabilities and to incorporate more sophisticated presentation techniques including collision detection in the scene.

    Plans and Projected Milestones

    		•
    Task Resources Required/Obstacles Development Time Release Date
    Prepare for release, making code robust Re-thinking and re-designing the modules and overall architecture 3 months Next Quarter
    Incorporation of Java into browser Need Java wrapper for Inventor. Solve potential Xt, Java problems 3 months Next Quarter
    Simulation Engine Need a protocol for simulation engine to follow 3-6 months Mid 96
    More Device Support Coordinate with group research. Pay attention to market and consumer support On-going On-going
    Multi-User Define architecture. Solve problems in protocol, concurrency, etc.. 6-9 months Late 1996
    Decision Support Application In-depth look at existing protocols, research (database search engines, internet search engines) and products (VOV)
    Research in visualization methodology. Link/develop software that represents data in 3D format and link it into Interact/ND    		 1 - 1.5 years Late 1996
    Develop New CAD Software Need infusion of (graduate) students working full-time in defining and architecting the software and also behaviors and properties of components. Require a great amount of software design and development, rapid prototyping, testing and evaluation 1 - 1.5 years for prototype
    On-going development and improvement    		 On-going
    Customer Tracking Need to design architecture and perform necessary code change. Will perform only if there's a necessity/demand or if time and resources permit 6 months To be determined


    Future Outlook

    Although significant development efforts and progress have been made in the fields of 3D rendering, immersive devices, network browsers and multi-user environment separately, we believe that Interact/ND provides a unique environment for the development of 3D, immersive, multi-user applications. These range from network games, to sophisticated design tools. With the security and platform-independency of Java, we will be ready to provide and develop an environment for a distributed, immersive design system that we see as the mode of design for the future.

    $Date: 1996/03/10 01:47:33 $
    ©1996 Michael Shilman <michaels@eecs.berkeley.edu>
    Contact 
    ©2002-2018 U.C. Regents