2018: Vortex Retrospective
Despite what it might look like from reviewing this blog’s history, a lot of work went into Vortex through the course of 2018. So much so, in fact, that just like I did a retrospective for 2016 with lessons learned then, I wanted to take some time to reflect on the state of Vortex at the end of 2018.
Codebase after 2.5 years
April 11, 2016 marks the date where we decided to kick off Vortex V3 by means of building it a visual editor from scratch.
Back then progress was crazy fast, with every coding session adding lots of new features worth covering in full.
2.5 years later, feature work continues, but at a more steady pace. Making changes to systems requires careful engineering and testing to make sure existing features are not broken, and in some cases, a small fix might end up in a big refactor of a less-than-polished system.
This is normal and expected, but it has been a factor in the cadence of new material in the blog. Sorry about that :)
On the flip side, building atop a more complete system enables focusing on newer features and on more polish, since we are not constantly bootstrapping all the systems in the engine anymore. This leads us to…
Vertical Slice Complete!
As work on persistence and Vortex Archives became more complete, having expanded the renderer, having introduced support for Lua scripting, and a complete Editor-to-Device workflow, we have reached a point where I think we could consider the Vertical Slice of Vortex V3 complete!
This is a big milestone. We set off to building an Editor that could be used for visually assembling playgrounds, scripting them, serializing and deploying them in a wide array of platforms. This year we have finally achieved this goal.
There is something truly special about the possibility of opening up the engine for users to script via Lua.
As of now, the limit is no longer whatever was pre-built into the engine. We are at a point where we can power the user’s imagination with (many) limitations removed.
On that topic,
Lua Scripting
Lua has been an absolute joy to work with. The code is neat, self contained, very rich and portable. In the Engine we we able to easily wrap the VM and run it both in Editor and on iOS devices.
Exposing the engine’s API opens up tones of possibilities where we are no longer just building internal systems, but rather we’re publishing a “service” that scripts can leverage to simulate and change the 3D world.
Not many people know this, but Vortex has had very comprehensive terrain generation and spline evaluation logic for a number of years now. These features are completely buried, as they are more playground-level than engine-specific code. With Lua, we can now surface these to the user as pre-packaged, efficient, native facilities.
Editor Run and Edit Modes
As soon as we opened up the possibility of having scripts running in editor changing the world we knew we would need to have a way to revert the changes made by scripts to an unsaved scene. By leveraging the now complete persistence system, this task was very easy to achieve.
In editor, when you start running your Lua scripts, the Editor will behind the scenes serialize the scene into a temporary manifest that is persisted somewhere. You can pause or continue running your scripts as normal, with them altering the world.
As soon as the user decides to stop script execution, the scene will be restored from the persisted manifest and the VM reset.
There is more work that can be performed here to ensure better performance (shorter deserialization cycles), but the core concept is a very powerful one, as it allows freely testing scripts before packaging the playground for distribution.
This of course leads to,
Vortex Runtimes
This concept came out of left field, considering we set out to “only” building an editor and revamping the engine. Once the use case was identified however, it all but made sense.
Vortex runtimes are lightweight apps built on the engine that allow loading a Vortex Archive (generated via the editor) and running it on the target platform.
We started off by implementing an iOS runtime. It consists of a simple UI that allows selecting which archive to run (coming from a list of predefined playgrounds), choosing a rendering backend and loading and running it.
Runtimes allow us to bring playgrounds to any platform without having to port the entire editor over, nor requiring the user to build her custom C++ app that hosts the engine and without having to worry about how to draw it all.
What that means is,
All Things Rendering
On the topic of rendering, we have now reached a point where the renderer is starting to look acceptable enough to be able to produce richer visuals than ever before in the history of the engine (so expect to see more screenshots and videos on this blog!).
The plan for next year is to continue pushing in this direction, building on a solid foundation and adding more high-fidelity visual techniques.
In addition to this work, this year saw the addition of a new native Metal renderer to the engine. The renderer is simple, but 100% compatible with the inputs taken by the rendering system used on other platforms. Metal was also a joy to work with. Very modern API with very good design decisions.
In Closing
A big year for Vortex V3, with many efforts that started about 2 years ago starting to take shape.
We now have 3 major verticals including editor, scripting API, and graphics (with OpenGL and Metal). Two years ago we talked about how 2 engineers could work on this project full time. I think that, after this year, we could easily keep 3 engineers busy.
This was also a year where we started to see some tech debt rear its head, but we’ve been able to keep it under control. There are some major refactors that are going to be required for enhanced performance, but we are not at a point yet where I would prioritize that work over adding more features that directly improve a user’s experience.
We want to thank you for joining us throughout this year. We’re looking forward to what’s to come in 2019 and, as usual, stay tuned for more ;)