So you want to integrate your Babylon.js experiments with your Gatsby blog, huh?

So, you've decided that you want to play around with Babylon.js. And you want to show off some experimental projects on your Gatsby powered blog. But, you want to do it your way. You don't want to clone the BabylonJS Playground. You don't want a playground embed code. You want to illustrate the cool stuff you're working on right in the middle of your article. How else is anyone ever going to discover your WebXR genius?

In this post, I'm going to walk you through an option using the documentation from the Babylon.js site, plus a little bit extra so you can change things in your BabylongJS scene from the React part of your app. I may go through some other 'more advanced' topics in another post.

tl;dr; Read the docs. Make a page. Make a custom Babylon.js component. Add some controls. Profit. Or, see the demo of Babylon.js inserted by hand into a stand-alone Gatsby page.

Official Babylon.js React documentation

Go check out the Babylon.js in React documentation if you haven't already. That's what we're going to build on.

We're going to focus on the first two snippets of code. The first snippet creates a Babylon.js scene component, and the second snippet implements the component. The main difference is that I'm going to combine them into a single Gatsby page.

First, make sure you've got @babylonjs/core installed.

npm install @babylonjs/core

Gatsby Page

In your pages directory, create a new page and import Engine and Scene from Babylon, and we're going to make use of a couple of hooks.

import { Engine, Scene } from "@babylonjs/core";
import React, { useEffect, useRef, useCallback, useState } from "react";

And we'll set up a simple page with a couple of controls.

export default function BabylonPage(props) {
  const [rotationSpeed, setRotationSpeed] = useState(10);
  return (
    <>
      <h1>Babylon.js in a Gatsby Page</h1>

      <button
        type="button"
        onClick={() => setRotationSpeed((prevSpeed) => prevSpeed - 1)}
      >
        -
      </button>
      {rotationSpeed}
      <button
        type="button"
        onClick={() => setRotationSpeed((prevSpeed) => prevSpeed + 1)}
      >
        +
      </button>
    </>
  );
}

As you might guess from the code, we're going to use the rotationSpeed state-variable to store the rotation speed.

Custom Babylonjs React Scene component

Next, we'll add the code for the scene component almost exactly as it appears in the documentation, with one tiny functionality change. The change will allow us to update the Babylon.js engine render function based on the rotationSpeed state variable from our page component.

export function SceneComponent({
  antialias,
  engineOptions,
  adaptToDeviceRatio,
  sceneOptions,
  onRender,
  onSceneReady,
  ...rest
}) {
  const reactCanvas = useRef(null);

  useEffect(() => {
    if (reactCanvas.current) {
      const engine = new Engine(
        reactCanvas.current,
        antialias,
        engineOptions,
        adaptToDeviceRatio
      );
      const scene = new Scene(engine, sceneOptions);

      if (scene.isReady()) {
        onSceneReady(scene);
      } else {
        scene.onReadyObservable.addOnce((scene) => onSceneReady(scene));
      }

      engine.runRenderLoop(() => {
        if (typeof onRender === "function") {
          onRender(scene);
        }
        scene.render();
      });
      const resize = () => {
        scene.getEngine().resize();
      };
      if (window) {
        window.addEventListener("resize", resize);
      }
      return () => {
        scene.getEngine().dispose();
        if (window) {
          window.removeEventListener("resize", resize);
        }
      };
    }
  }, [
    adaptToDeviceRatio,
    antialias,
    engineOptions,
    onRender,
    onSceneReady,
    sceneOptions,  ]);
  // eslint-disable-next-line react/jsx-props-no-spreading
  return <canvas ref={reactCanvas} {...rest} />;
}

Besides destructuring the props in the function signature, I updated the useEffect hook to only run if the props have changed. In the example provided in the documentation, the useEffect was only going to run if the ref to the canvas changed. But I want this effect to change if onRender and maybe onSceneReady changes - you'll see why in a moment. (The reason the other props are there is that ESLint for React insists this exhaustive dependency lists is a better practice. I agree since we probably would want this effect to run if those values change, but I won't be covering that in this example.)

Adding the component to the page.

Next, we'll add this component to the page and add our onRender and onSceneReady callbacks.

Once again, I'm using almost the same code as in the documentation, with a couple of minor modifications.

let box;
const onSceneReady = useCallback(
  (scene) => {
    // This creates and positions a free camera (non-mesh)
    const camera = new FreeCamera("camera1", new Vector3(0, 5, -10), scene);
    // This targets the camera to scene origin
    camera.setTarget(Vector3.Zero());
    const canvas = scene.getEngine().getRenderingCanvas();
    // This attaches the camera to the canvas
    camera.attachControl(canvas, true);
    // This creates a light, aiming 0,1,0 - to the sky (non-mesh)
    const light = new HemisphericLight("light", new Vector3(0, 1, 0), scene);
    // Default intensity is 1. Let's dim the light a small amount
    light.intensity = 0.7;
    // Our built-in 'box' shape.
    box = MeshBuilder.CreateBox("box", { size: 2 }, scene);
    // Move the box upward 1/2 its height
    box.position.y = 1;
    // Our built-in 'ground' shape.
    MeshBuilder.CreateGround("ground", { width: 6, height: 6 }, scene);
  },
  [box]
);
/**
 * Will run on every frame render.  We are spinning the box on the y-axis.
 */
const onRender = useCallback(
  (scene) => {
    if (box !== undefined) {
      const deltaTimeInMillis = scene.getEngine().getDeltaTime();
      const rpm = rotationSpeed;
      box.rotation.y += (rpm / 60) * Math.PI * 2 * (deltaTimeInMillis / 1000);
    }
  },
  [box, rotationSpeed]
);

You'll notice that onRender now defines rpm as the rotationSpeed state variable. You'll also notice that useCallback hooks wrap the onRender and onSceneReady functions. useCallback will return the exact same function object every time unless the values in the second argument array change. If we don't do this, onRender and onSceneReady will be different functions every React render cycle, and the useEffect hook in the SceneComponent will get triggered every time as well.

Lastly, we actually add the component to the page.

<SceneComponent
  antialias
  onSceneReady={onSceneReady}
  onRender={onRender}
  id="my-canvas"
/>

Understanding the changes to the code

If we test out this page, we'll see that the cube will change its rotation speed as you click the buttons. How does this work?

When you click the buttons, the state of rotationSpeed changes. The state change causes React rendering to kick in, which causes the <SceneComponent> to re-render. The re-render causes the useCallback hooks to fire. Since rotationSpeed has changed, onRender will return a different function object. <SceneComponent>'s useEffect hook sees that onRender is different and runs its code, ultimately resulting in the scene reset and with the new Babylon.js render loop calling the new function with the new rotation speed.

Phew! That's a lot to take in. See a demo of all this code.

Conclusion, objections, alternatives and next steps

Is this way of updating a scene a good idea? For the most part, I'm guessing this isn't the best way to update your scene if you're approaching this from the Babylon.js side of things. I'm very, very, very new to Babylon.js, but I know that it has its own way of binding UI elements to the scene from right within the scene. If you're going to make any WebXR experiences, I think that is likely the only way you're going to be able to wire up a UI that makes sense in those XR contexts.

On the other hand, this kind of approach might be a neat light-weight way to trigger data updates or new graphQL queries that load new elements into a scene depending on the results. Or, if you're running a commerce site and are loading product models into your web page, this simple solution may be the way to go.

You may also notice that on the Babylon.js documentation page there is mention of the babylonjs-hook npm module written by Brian Zinn. From what I can tell, that module encapsulates the default example code for a Scene Component that is provided in the documentation. And, from what I can tell, there is no way to communicate between React and a scene using that module.

As a next step, I'd love to figure out a way to maintain the scene state. Right now the engine and the scene are re-intialized and re-stared whenever the render function changes. It would be better if I was just able to pass information into the scene without restarting it.

I took a quick look at Brian Zinn's React Babylon.js project, and it looks promising. It provides a declarative approach to building scenes, where scenes are composed with React components, and 'escape hatches' are provided to let you do things procedurally if you'd like. I'm curious about digging a little bit deeper into that solution if I ever need to build something a little more robust than my current needs today.

A final next step for me, will be to change the way I'm building this site so that I can make use of the component in-line between paragraphs. MDX will likely be the solution. If I ever make that change, I'll update this post to include the demo in-line. Oh, and I'll be implementing a commenting system so that this isn't a one-way conversation.