swift - Dynamically change text of RealityKit entity

Question

I have created a very simple scene ("SpeechScene") using Reality Composer, with a single speech callout object ("Speech Bubble") anchored to a Face anchor.

I have loaded this scene into code via the following:

let speechAnchor = try! Experience.loadSpeechScene()
arView.scene.anchors.append(speechAnchor)

let bubble = (arView.scene as? Experience.SpeechScene)?.speechBubble

It renders as expected. However, I would like to dynamically change the text of this existing entity.

I found a similar question here, but it's unclear to me how to refer to the meshResource property of a vanilla RealityKit.Entity object.

Is this possible? Thank you!

Answer 1

First Approach

At first you need to find out what's an hierarchy in Reality Composer's scene containing Bubble Speech object. For that I used simple print() command:

print(textAnchor.swift!.children[0].components.self)   /* Bubble Plate */

print(textAnchor.swift!.children[1].components.self)   /* Text Object */

Now I can extract a text entity object:

let textEntity: Entity = textAnchor.swift!.children[1].children[0].children[0]

And bubble plate entity object:

let bubbleEntity: Entity = textAnchor.swift!.children[0]

Here's a final code version that you can adapt for your needs:

import RealityKit

class GameViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let textAnchor = try! SomeText.loadTextScene()
        
        let textEntity: Entity = textAnchor.swift!.children[1].children[0].children[0]

        textAnchor.swift!.parent!.scale = [4,4,4]  // Scale for both objects
        
        var textModelComp: ModelComponent = (textEntity.components[ModelComponent])!
                
        var material = SimpleMaterial()
        material.baseColor = .color(.red)
        textModelComp.materials[0] = material

        textModelComp.mesh = .generateText("Obj-C",
                            extrusionDepth: 0.01,
                                      font: .systemFont(ofSize: 0.08),
                            containerFrame: CGRect(),
                                 alignment: .left,
                             lineBreakMode: .byCharWrapping)

        textEntity.position = [-0.1,-0.05, 0.01]

        textAnchor.swift!.children[1].children[0].children[0].components.set(textModelComp)
        arView.scene.anchors.append(textAnchor)
    }
}

Second Approach

And you can always use a simpler approach for this case – to create several scenes in Reality Composer, each one must contain different speech-object.

Consider, this code isn't for tracking, it's just a test for dynamically switching two objects using Tap Gesture. Then you need to adapt this code for tracking faces.

import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    var counter = 0
    var bonjourObject: FaceExperience.Bonjour? = nil
    var holaObject: FaceExperience.Hola? = nil
     
    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
        
        // Reality Composer Scene named "Bonjour"
        // Model name – "french"
        bonjourObject = try! FaceExperience.loadBonjour()
        bonjourObject?.french?.scale = SIMD3(x: 2, y: 2, z: 2)
        bonjourObject?.french?.position.y = 0.25
        
        // Reality Composer Scene named "Hola"
        // Model name – "spanish"
        holaObject = try! FaceExperience.loadHola()
        holaObject?.spanish?.scale = SIMD3(x: 2, y: 2, z: 2)
        holaObject?.spanish?.position.z = 0.3
    }
    @IBAction func tapped(_ sender: UITapGestureRecognizer) {            
        if (counter % 2) == 0 {
            arView.scene.anchors.removeAll()
            arView.scene.anchors.append(holaObject!)
        } else {
            arView.scene.anchors.removeAll()
            arView.scene.anchors.append(bonjourObject!)
        }
        counter += 1
    }
}

If you want a text portion to be on the same place – just copy-paste object from one scene to another.