Dear Esther Unity: How..?
Originally Posted February 17, 2014
So I thought I’d do a follow-up to my previous post on the Dear Esther Unity port with a little more detail for those of you who are curious about exactly how I managed it and some of the tools I used:
Content
Overall, I’d estimate Dear Esther to be 10% gameplay code and 90% content, so starting out, my number 1 priority was to get the content into Unity.
The biggest obstacle I faced was how to get all of the content out of the Source Engine and over to Unity. I had the source files for models and textures, but the biggest problem was extracting content created within the Source Engine itself: stuff like the brushes and displacements which the environments are heavily made up of and which are only stored in Source’s VMF and BSP formats.
Wall Worm Model Tools
So I started searching to see if there was any way to convert BSPs into some kind of readable format for 3DS Max. Hammer has a internal DXF exporter but it is horrid, and externally, all I could find were a couple of really old tools, the best of which was Nem’s Crafty but unfortunately it didn’t export full displacement data such as vertex alpha, which is vital to get blend textures working. It was only after searching for “VMF tools” that I stumbled across something very promising: A free set of scripts for 3DS Max called the Wall Worm Model Tools by Shawn Olson.
He’d made a bunch of nifty tools for designing maps and compiling models directly in 3DS Max, which could be exported straight into hammer (Man how I wish I’d had these 5 years ago!). Included in the toolset was a early VMF importer which looked very promising. It was, however, mostly unfinished and had a lot of issues that needed addressing before it could be useful. I decided to contact Shawn and see if he’d be interested in finishing up the importer if I commissioned him for his time, and fortunately he was more than happy to help, actually taking it far beyond my original expectations: The Final VMF Importer not only had the ability to import brushes and displacements, but absolutely everything, from the models and textures, to triggers and entities. The final result was an exact replica of my levels inside Max, all ready for exporting!
After that I had only one roadblock remaining: Shaders.
Shader Forge
Although most of Source’s shaders aren’t the most sophisticated, they have a very distinct look to them which isn’t reproducible with the basic set of shaders that come with Unity. I had a strong idea of how they could be recreated, but being a useless coder I was kind of at an impasse. Initially I tried Strumpy’s Shader Editor, but it was very unfamiliar and horribly broken for the most part. Later, whilst venting my frustrations over twitter, a very kind person directed me to a recent post on Polycount by a guy called Joachim Holmér. He was working on a very early version of node-based shader editor for Unity called Shader Forge.
The design will be instantly familiar to anyone like myself who has tinkered with UDK’s shader editor, and seemed like the last piece of the puzzle I needed to get started on bringing in content to Unity. After generously letting me into the alpha testing team, I got stuck in.
What was involved
It wasn’t a completely straight forward process of just importing assets though, and I still had to do the following:
Create lightmap UV’s for all of my models, since Unity doesn’t support vertex lighting
Convert everything between Source’s units of measurement and Unity’s (Source=inches, Unity=meters, although Source seems to have some crazy voodoo measurements going on some of the time).
Recreate lighting (again stuff like light strength, falloff, etc. was more guesswork, and I still haven’t found an exact formula for it).
Solving a lack of 3D Skybox in Unity (I ended up scaling the skybox models up to match the scale set in Source – not the most efficient way to do it I know, but I’m working on that!)
Workaround for lack of max fog opacity control (I always had fog_maxdensity set to 0.9 in Source so it never becomes a solid wall of colour – still trying to work this one out for Unity though)
Recreate the Soundscape system in Source (Done using the excellent Sound Manager Pro Plugin)
Voice-overs with subtitles (+Multi-language support)
FPS Movement and interaction: Using UFPS at the moment but am still ironing out movement issues, namely with Step Offset jittering.
Swimming – sounds easy but it really wasn’t, It seems UFPS isn’t really set up for it…)
Create a custom water shader + Script (Almost there but still ironing out some bugs on that: Right now I have just modified the basic water 4 script to output render textures to my SF shader)
Recreate underwater fog (See this nifty plugin by David Miranda)
Recreate all particle systems
Reproduce colour correction
Foliage Wind: now much improved (hopefully!)
Finding a workaround for the lack of a foliage system that can be applied directly to meshes (Unity only allows you to paint foliage onto terrain, and since I’m using meshes, it’s kind of useless)
That last one has been an ongoing point of contention for me since the foliage in Dear Esther is such an important part of the game’s visuals. Early on, I asked Shawn if he could extend the Wall Worm Importer to include detail sprites which he managed to implement rather quickly, but the problem then was: how do I get them over into Unity as efficiently as possible? Unfortunately I’m still trying to figure that one out. If anyone has any ideas, I’d love to hear them!
Shrubbery
Fortunately a second option for foliage I’m looking into is a tool called Shrubbery – it’s still in beta and I’m thoroughly testing it right now; basically, it’s a promising tool that allows you to paint foliage onto any surface in Unity. Obviously the downside to this approach is the time sink with having to repaint all of the foliage in-game, and how closely it will look and perform against the original. This might have to be a sacrifice I have to make though…
Finally I still have a lot of optimisation to do, getting performance in line with the original game is absolutely paramount for me, and I’m still figuring out how to best do this in Unity. Things are looking promising though: on some levels, without any optimisation at all, FPS in on a par with Source Engine version!
It’s probably also worth mentioning that I still haven’t written a single line of code yet, all of the functionality so far has been done via Playmaker, as mentioned in my previous post.
So what’s left?
I still need to finish up the UI, menus etc. get scripting in for the last three levels, and as well as the foliage, particles and optimisations, so there’s still a ways to go yet, but for now a lot of the legwork is done.
I think that covers the most important bits! If anyone has any questions please let me know either on here or on twitter and I’ll be glad to answer them.