Interactive 3D Machine Showcase with Threepipe

To get started with Threepipe locally, you need to have Node.js installed on your machine. Vite Projects require Node.js version 18+, so upgrade if your package manager warns about it.

1. A new project can be quickly created using the npm create command. Open your terminal and run the following command:

npm create threepipe

After creating a basic project, open the file src/main.ts.

This is a basic setup for a 3D scene using Threepipe that loads a sample 3D model of a helmet and an environment map(for lighting). The scene is rendered on a canvas element with the ID threepipe-canvas(which is added to the file index.html).

The ThreeViewer class is used to create a new 3D viewer instance. The viewer has several components including a Scene, Camera(with controls), Renderer, RenderManager, AssetManager, and some default plugins. It is set up to provide a quickstart to create a three.js app with all the required components. Additionally plugins like LoadingScreenPlugin, ProgressivePlugin, SSAAPlugin, and ContactShadowGroundPlugin are added to extend the functionality of the viewer. We will add more plugins to the viewer for different use cases as we progress through the tutorial.

Check the comments in the code to understand what each part does.

import {
  ContactShadowGroundPlugin,
  IObject3D,
  LoadingScreenPlugin,
  ProgressivePlugin,
  SSAAPlugin,
  ThreeViewer
} from 'threepipe';
import {TweakpaneUiPlugin} from '@threepipe/plugin-tweakpane';

async function init() {

  const viewer = new ThreeViewer({
    // The canvas element where the scene will be rendered
    canvas: document.getElementById('threepipe-canvas') as HTMLCanvasElement,
    // Enable/Disable MSAA
    msaa: false,
    // Set the render scale automatically based on the device pixel ratio
    renderScale: "auto",
    // Enable/Disable tone mapping
    tonemap: true,
    // Add some plugins
    plugins: [
        // Show a loading screen while the model is downloading
        LoadingScreenPlugin,
        // Enable progressive rendering and SSAA
        ProgressivePlugin, SSAAPlugin,
        // Add a ground with contact shadows
        ContactShadowGroundPlugin
    ]
  });

  // Add a plugin with a debug UI for tweaking parameters
  const ui = viewer.addPluginSync(new TweakpaneUiPlugin(true));

  // Load an environment map
  await viewer.setEnvironmentMap('https://threejs.org/examples/textures/equirectangular/venice_sunset_1k.hdr', {
    // The environment map can also be used as the scene background
    setBackground: false,
  });

  // Load a 3D model with auto-center and auto-scale options
  const result = await viewer.load<IObject3D>('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf', {
    autoCenter: true,
    autoScale: true,
  });

  // Add some debug UI elements for tweaking parameters
  ui.setupPlugins(SSAAPlugin)
  ui.appendChild(viewer.scene)
  ui.appendChild(viewer.scene.mainCamera.uiConfig)

  // Every object, material, etc has a UI config that can be added to the UI to configure it.
  const model = result?.getObjectByName('node_damagedHelmet_-6514');
  if (model) ui.appendChild(model.uiConfig, {expanded: false});

}

init();

For this showcase, we’ll use 3D models of a MacBook and an iPhone. You can find free 3D models online or create your own using software like Blender.

These are two amazing models from Sketchfab that we will use in this tutorial:

Using the models, we’ll create a scene with a MacBook and an iPhone placed on a table. The user can interact with the scene by rotating and zooming in/out.

To load the 3D model in the project, we can either load the file directly from the URL or download the file to the public folder in the project and load it from there.

Since this model includes all the settings, including the environment map, we can remove the environment map loading code from the starter code and load the file directly.

const viewer = new ThreeViewer({
  canvas: document.getElementById('threepipe-canvas') as HTMLCanvasElement,
  msaa: true,
  renderScale: "auto",
  plugins: [
    LoadingScreenPlugin, ProgressivePlugin, SSAAPlugin, ContactShadowGroundPlugin,
  ]
});

const ui = viewer.addPluginSync(new TweakpaneUiPlugin(true));

// Note - We dont need autoscale and center, since that is done in the editor already.
const devices = await viewer.load<IObject3D>('https://asset-samples.threepipe.org/demos/tabletop_macbook_iphone.glb')!;
// or if the model is in the public directory
// const devices = await viewer.load<IObject3D>('./models/tabletop_macbook_iphone.glb')!;

// Find the objects roots by name
const macbook = devices.getObjectByName('macbook')!
const iphone = devices.getObjectByName('iphone')!

const macbookScreen = macbook.getObjectByName('Bevels_2')! // the name of the object in the file
macbookScreen.name = 'Macbook Screen' // setting the name for easy identification in the UI.

console.log(macbook, iphone, macbookScreen);

// Add the object to the debug UI. The stored Transform objects can be seen and edited in the UI.
ui.appendChild(macbookScreen.uiConfig, {expanded: false})
ui.appendChild(iphone.uiConfig, {expanded: false})
// Add the Camera View UI to the debug UI. The stored Camera Views can be seen and edited in the UI.
ui.setupPluginUi(CameraViewPlugin, {expanded: false})
ui.appendChild(viewer.scene.mainCamera.uiConfig)

This code will load the 3D model in the scene and add the objects to the debug UI for tweaking parameters.

The file has been configured in the editor with several camera views(states) and object transform(position, rotation) states. This is done using the plugins CameraViewPlugin and TransformAnimationPlugin. To see the stored camera views and object transforms and interact with them, we need to add them to the viewer and the debug UI.

First, add the plugins to the viewer constructor

const viewer = new ThreeViewer({
   canvas: document.getElementById('threepipe-canvas') as HTMLCanvasElement,
   msaa: true,
   renderScale: "auto",
   plugins: [
      LoadingScreenPlugin, ProgressivePlugin, SSAAPlugin, ContactShadowGroundPlugin,
      CameraViewPlugin, TransformAnimationPlugin
   ]
});

Then at the end, add the CameraViewPlugin to the debug UI

ui.setupPluginUi(CameraViewPlugin)

We don’t need to add the TransformAnimationPlugin to the UI since the states are mapped to objects and can be seen in the UI when the object is added.

We can now interact with the UI to play the animations and animate to different camera views.

Transform states are added to two objects in the file, the MacBook Screen and the iPhone. 

The camera views are stored in the plugin and not with any object in the scene. We can view and animate to different camera views using the plugin UI. Here, we have two sets of camera views, one for the desktop and one for the mobile (with different FoV/Position)

Now that we have the scene set with the models and animations, we can add user interaction to the scene. The idea is to slightly tilt the model when the user hovers over it and fully open it when clicked, along with animating the camera views. Let’s do it step by step.

For the interaction, we can use the PickingPlugin which provides events to handle hover and click interactions with 3D objects in the scene.

First, add PickingPlugin to the viewer plugins

plugins: [
   LoadingScreenPlugin, ProgressivePlugin, SSAAPlugin, ContactShadowGroundPlugin,
   CameraViewPlugin, TransformAnimationPlugin, PickingPlugin
]

With this, we can now click on any object in the scene and it will be highlighted with a bounding box.

Now, we can configure the plugin to hide this box and subscribe to the events provided by the plugin to handle the interactions.

// get the plugin instance from the viewer
const picking = viewer.getPlugin(PickingPlugin)!
const transformAnim = viewer.getPlugin(TransformAnimationPlugin)!

// disable the widget(3D bounding box) that is shown when an object is clicked
picking.widgetEnabled = false

// subscribe to the hitObject event. This is fired when the user clicks on the canvas.
picking.addEventListener('hitObject', async(e) => {
   const object = e.intersects.selectedObject as IObject3D
   // selectedObject is null when the user clicks the empty space
   if (!object) {
       // close the macbook screen and face down the iphone
      await transformAnim.animateTransform(macbookScreen, 'closed', 500)?.promise
      await transformAnim.animateTransform(iphone, 'facedown', 500)?.promise
      return
   }
   // get the device name from the object
   const device = deviceFromHitObject(object)
   // Change the selected object to the root of the device models. This is used by the widget or other plugins like TransformControlsPlugin to allow editing.
   e.intersects.selectedObject = device === 'macbook' ? macbook : iphone

   // Animate the transform state of the object based on the device name that is clicked
   if(device === 'macbook')
      await transformAnim.animateTransform(macbookScreen, 'open', 500)?.promise
   else if(device === 'iphone')
      await transformAnim.animateTransform(iphone, 'floating', 500)?.promise
})

Here, the animateTransform function is used to animate the transform state of the object. The function takes the object, the state name, and the duration as arguments. The promise returned by the function can be used to wait for the animation to complete.

The deviceFromHitObject function is used to get the device name from the object clicked. This function traverses the parents of the object to find the device model.

function deviceFromHitObject(object: IObject3D) {
   let device = ''
   object.traverseAncestors(o => {
      if (o === macbook) device = 'macbook'
      if (o === iphone) device = 'iphone'
   })
   return device
}

With this code, we can now interact with the scene by clicking on the models to open/close the MacBook screen and face down/floating the iPhone.

Now, we can add camera animations as well to animate to different camera views when the user interacts with the scene.

Get the plugin instance

const cameraView = viewer.getPlugin(CameraViewPlugin)!

Update the listener to animate the views using the animateToView function. The views are named ‘start’, ‘macbook’, and ‘iphone’ in the plugin.

const object = e.intersects.selectedObject as IObject3D
if (!object) {
   await Promise.all([
      transformAnim.animateTransform(macbookScreen, 'closed', 500)?.promise,
      transformAnim.animateTransform(iphone, 'facedown', 500)?.promise,
      cameraView.animateToView('start', 500),
   ])
   return
}
const device = deviceFromHitObject(object)
if(device === 'macbook') {
   await Promise.all([
     cameraView.animateToView('macbook', 500),
     await transformAnim.animateTransform(macbookScreen, 'open', 500)?.promise
   ])
}else if(device === 'iphone') {
   await Promise.all([
     cameraView.animateToView('iphone', 500),
     await transformAnim.animateTransform(iphone, 'floating', 500)?.promise
   ])
}

This would now also animate the camera to the respective views when the user clicks on the models.

In the same way, PickingPlugin provides an event hoverObjectChanged that can be used to handle hover interactions with the objects.

This is pretty much the same code, but we are animating to different states(with different durations) when the user hovers over the objects. We don’t need to animate the camera here since the user is not clicking on the objects.

// We need to first enable hover events in the Picking Plugin (disabled by default)
picking.hoverEnabled = true

picking.addEventListener('hoverObjectChanged', async(e) => {
   const object = e.object as IObject3D
   if (!object) {
      await Promise.all([
         transformAnim.animateTransform(macbookScreen, 'closed', 250)?.promise,
         transformAnim.animateTransform(iphone, 'facedown', 250)?.promise,
      ])
      return
   }
   const device = deviceFromHitObject(object)
   if(device === 'macbook') {
      await transformAnim.animateTransform(macbookScreen, 'hover', 250)?.promise
   }else if(device === 'iphone') {
      await transformAnim.animateTransform(iphone, 'tilted', 250)?.promise
   }
})

On running this, the MacBook screen will slightly open when hovered over and the iPhone will slightly tilt.

To allow users to drop images to display on the devices, we can use the DropzonePlugin provided by Threepipe. This plugin allows users to drag and drop files onto the canvas and handle the files in the code.

The plugins can be set up by simply passing dropzone property in the ThreeViewer constructor. The plugin is added and set up automatically.

Let’s set some options to handle the images dropped on the canvas.

const viewer = new ThreeViewer({
  canvas: document.getElementById('threepipe-canvas') as HTMLCanvasElement,
  // ...,
  dropzone: {
    allowedExtensions: ['png', 'jpeg', 'jpg', 'webp', 'svg', 'hdr', 'exr'],
    autoImport: true,
    addOptions: {
      disposeSceneObjects: false,
      autoSetBackground: false,
      autoSetEnvironment: true, // when hdr, exr is dropped
    },
  },
  // ...,
});

We are setting autoSetEnvironment to true here, which will automatically set the environment map of the scene when an HDR or EXR file is dropped on the canvas. This way a user can drop their own environment map and it will be used for lighting.

Now, to set the dropped image to the devices, we can listen to the loadAsset event of the AssetManager and set the image to the material of the device screen. This event is called since the DropzonePlugin also automatically imports as a three.js Texture object and loads the file in the asset manager. To get more control, you can also subscribe to the events in the DropzonePlugin and handle the files yourself.

// Listen to when a file is dropped
viewer.assetManager.addEventListener('loadAsset', (e)=> !iPhoneScreen) return
  mbpScreen.color.set(0,0,0)
  mbpScreen.emissive.set(1,1,1)
  mbpScreen.roughness = 0.2
  mbpScreen.metalness = 0.8
  mbpScreen.map = null
  mbpScreen.emissiveMap = texture
  iPhoneScreen.emissiveMap = texture
  mbpScreen.setDirty()
  iPhoneScreen.setDirty()
)

This code listens to the loadAsset event and checks if the loaded asset is a texture. If it is, it sets the texture to the material of the MacBook and iPhone screens. The texture is set as the emissive map of the material to make it glow. The emissive color is set to white to make the texture visible. The changes in the material need to be done only in the Macbook screen material and not the iPhone, since iPhone material setup was done in the editor directly.

While interacting with the project, you might notice that the animations are not properly synced. This is because the animations are running asynchronously and not waiting for the previous animation to complete.

To fix this, we need to maintain a state properly and wait for any animations to finish before changing it.

This tutorial covers the basics of creating an interactive 3D device mockup showcase using Threepipe. You can further enhance the project by adding more models, animations, and interactions.