Montage Studio

Introduction

Imagine if you could easily create interactive experiences using scenarios as a building block:

Choose a command 🡆 Choose a target 🡆 Behold the magic!

This simple concept can be leveraged in presentations, educational apps, even adventure games. Montage Studio will be the platform for building all these with incredible ease. In a matter of minutes you'll be able to create a scene, add visuals, and define interactions.

Development

This is where I'll document the development journey, from wireframes all the way to launch.

September 5, 2023: Drag-and-drop, Studio/Player proxy, & more

A major hurdle is cleared: figuring out how to have two separate packages — "Studio" and "Player" — both operate on visual elements while correctly maintaining state and without violating React/Redux principles. The GIF below shows a visual being added through Studio, then it's dragged across the scene within the Player. This is made possible via "proxy" modules which allow Studio to update certain parts of state, and also enables both packages to report/react on changes in that state.

I chose dnd kit for drag-and-drop functionality. It's lightweight and extensible with accessibility build right in. I'm a big fan!

Note: this GIF contains mostly placeholders, so hold your judgment! The finished UX will look infinitely better.

April 23, 2023: Targets, scaling, and pixel-level detection

This was a big weekend on the Montage front, starting with the inaugural vlog:

And the cherry on top: pixel-level targeting is now implemented. This will be especially useful in situations where multiple visuals occupy the same area.

March 5, 2023: Coordinates, z-indexes, and transitions

I'm really enjoying this deep-dive into React, Redux Toolkit, and Framer Motion. Most people won't fully appreciate the challenges involved with pathing, scaling, scene transitions, etc. — and that's ok — but those challenges are magnified in the context of a React app.

Until this weekend I was dreading a major hurdle: how can I move visuals around the screen while tracking their coordinates in real time and keeping the animation reasonably smooth? Using React state was a non-starter; constant re-renders would result in a "jittery" animation and grind things to a halt.

Then it dawned on me: move the coordinate tracking to a separate module and let Framer Motion animate the CSS, then have visuals regularly report their coordinates independent of state/render cycles. The clip below doesn't really capture how smooth the movement is but suffice to say I'm over-the-moon happy with the results. A long-overdue win!

February 24, 2023: The path is clear

With Montage you'll be able to move visuals from Point A to B, but with each scene you'll have to make a decision: do they go straight from A to B or is it more complicated?

If you're making an adventure game for example, there may be obstacles between A and B. Luckily Montage will support two pathing types: Direct and Grid. The latter leverages the A* pathfinding algorithm to determine the most efficient path between two points.

When a scene with a pathType of Grid is loaded it's essentially divided into a grid of cells:

When you click anywhere on the scene, the algorithm accounts for obstacles and finds the most efficient path:

The best part: setting up the grid in Montage won't be complicated. You'll essentially "paint" the grid cells that are walkable (or not walkable, whichever's easier) and that's it. The runtime engine will take it from there.

February 5, 2023: Glimpses of a new frontier

The last couple weeks have been interesting with mass layoffs in Big Tech and an explosion of AI. The AI impact on Montage remains to be seen but I spent some time with ChatGPT and Midjourney. Mind-blowing stuff to say the least and my imagination is now in overdrive. To inject more fun into this early development stage I used Midjourney to create background images for a couple test scenes:

The above gif might seem rough with its plain buttons, yellow rectangles, and abrupt transitions, but there's a lot going on there. The rectangles represent interactive areas and are for illustrative purposes. I spent most of yesterday figuring out how to intelligently scale a scene — including its foreground visuals, characters, etc. — when the browser window is resized. Very happy with the result so far!

The best part: I'm becoming very confident with the approach I've taken with the engine. This has a very real chance to become the platform I've always wanted to build.

December 17, 2022: The hole in the ground

In the early stages of a building's construction, it doesn't look like much more than a hole in the ground. It stays that way for a while, making folks wonder if work halted altogether.

Montage Studio is in this stage and will be for a while. It's tempting to take shortcuts right to the eye candy, but patience is key. I'm building the infrastructure, experimenting with caching and state management, piecing together the "engine" bit by bit.

Below is a gif of what the client currently does:

Like I said — a hole in the ground! But a lot's going on under the hood. A "player" component is initializing a manifest with two test scenes, each of which have a number of visuals. When I click a command + a target it creates a "scenario" which is then processed by the engine. One scenario moves a visual to a new location while another changes the scene.

Thanks to Redux toolkit I'm able to manage slices of state and easily consume them in various components. I'm close to an optimal approach to processing scenarios without relying on spaghetti code. Once I'm fully confident in the approach I'll start adding "action processors" one-by-one, and that's where the fun begins!

December 10, 2022: Humble beginnings

It's easy to get "paralysis by analysis" in the early stages of a project. Wireframes are always a good place to start, a way to convert abstract thoughts into something a bit more concrete.

The main challenge/question so far won't surprise any developer: how do I make Montage extremely easy-to-use but also powerful and flexible? If someone wants to make a quick educational app, they shouldn't be overwhelmed with options and/or jargon. They should click a few buttons, upload a few images, and voilà. At the same time, authors should be able to create complex experiences like point-and-click adventure games.

October 14, 2022: Unofficial kickoff

My initial technical goals were clear: build on lessons learned from my previous Tenterhook platform and choose a stack that's built to last. I spent a long weekend in Ocean Shores, WA to set aside distractions and dive into a Udemy course on the MERN stack. The course was phenomenal and convinced me to commit to MERN, and as a bonus I secured the project name/domain.

Bonus: I brought my beloved Murphy and we enjoyed playtime on the beach!