The Name of the Engine
03.07.2008
Alpha Blending
Well – only three levels of transparency just wasn’t good
enough for me so I had to research a little further and found out an even
simpler solution to get a linear transparency with better resolution than the
previous method (2 samples instead of 4 samples).
The method will be explained in details in the next ShaderX series
book.
And here is a screenshot.

I’ve also implemented a double sided lighting (green circle)
and transparency shadows (red circle).
03.06.2008
Alpha to Coverage
Alpha blending is one of the drawbacks of deferred
rendering. You can’t use alpha to coverage since it requires multi sampling
which is not possible under DirectX 9 and tricky event on DirectX 10.
Nevertheless I tried to use alpha-to-coverage and do a
manual half pixel offset blending to get 2x2 sampling for alpha. The method is
very simple:
- Project alpha-to-coverage texture onto alpha objects in
screen-space to get one-to-one screen mapping.
- Map and quantize texture lookup to get homogenous texture
piece.
- Write original alpha value into color channel.
- In color and light combine pass check the color alpha
value and offset texture offset for half texture pixel and do texture
lookup again to get multi-sampled texture (4 pixels multi-sample).
Drawbacks are:
- Only three levels of transparency (25, 50 and 75 percent).
- A little visible blur on alpha blended objects caused by a
half pixel offset sampling.
- Only the last transparent object is visible.
And the good things are:
- Same quality of lighting for opaque and transparent
objects.
- Very simple and robust method.
- No need to sort alpha objects.
- It’s working with deferred rendering.
01.13.2008
iMaya plug-in robustness
That’s now basically all that needs to be done before a new
public release.
01.10.2008
DoxyGen
I fixed a DoxyGen a little so that the documentation is in
more programmer friendly way.
Screen shot.

73 classes to go.
01.06.2008
Documentation
It was time to document the stuff that I’ve done. And that’s
never an easy task. So in the name of my sanity I decided to document at least one
class per day (of 79 classes) plus some general documentation.
And here it is my first part of general documentation: render
architecture.

Click for larger image.
01.03.2008
Update
I decided to stop working on the engine… until I update the diary.
12.26.2007
6th month universary
Additional script optimization and design unexpectedly took
a lot more time than expected. And of course here was a happy December – do I
need to say anything more?
Status and roadmap so far (+ marks new features):
- +Player
- +Simple player controller
- +Render view set
- +Props
- Animation automat
- Player controller
- Sound (FMOD)
- Network
- Blend shapes
- Post process rendering effects (DOF, HDR, Fog)
- Detail textures
- Transparency
- Visibility (portals)
- Effects (particle system, etc)
- Decals
- Instancing
- Engine SDK release
- Multi thread rendering
- PlayStation3 (driver)
- Xbox360 (driver)
- DirectX 10 (driver)
- Resource editor (UI)
And finally the list IS SHRINKING…
12.24.2007
Maya manipulators
Unfortunately Maya doesn’t have a separate manipulators
overlay or some other way to redraw only manipulators and other design stuff.
So (again) I had to do it the hard way. I did grid, move, rotate and scale
manipulators (almost the same as Maya’s) and lights visualization.

Maya renderer and Insomnia renderer side by size.
12.19.2007
Prop script
It took some doing to elegantly link prop with script since
each props data needs its own script instance and each prop needs its own data
storage (e.g. health, destruction state).
For that I implemented two things:
- Event function - basically an ordinary function with a compound
name (e.g. objectname::eventname) and import capability without a formal definition.
In short: event function can be called in every script without previous
declaration. This solved the problem of communication between several
props.
- Property variable - this is a special kind of global
variable (defined with a keyword ‘property’) that can be accessed from
within the engine.
12.12.2007
Bencmarking
And it was time to do a little benchmarking to see how fast
we got.
The result numbers represent how many times the script is
slower than the same VS C++ code compiled with full optimization.
Synthetics tests
Practical tests
Recursion
|
66
|
Fibonacci and Ackerman
|
Math
|
23
|
Mandelbrot
|
Actual usage
12.10.2007
JIT (Just In-time Compilation)
The only way to make script really fast is to do JIT
compiler. I don’t want to do real JIT (jet) so I tried to do a simple JIT by tabling
all possible script functions instead of doing a switch; and there are many
function, actually (parameter_type * parameter_address_mode) ^ number_of_parameters
to be exact. In my case that is: (5*8)^3. Fortunately not all combinations are
used so I had to do only ~19000 functions.
Anyways the speed improvement was huge, around 3 times
better.
Some facts:
- Down to 17 instructions of redundant code per instruction
(from ~75).
- Average of ~10 instructions per script instruction (from
~25).
The next step is real JIT. I’ve already invested some time
in it and it is just not time right now. I have to move on.
12.09.2007
Script Array
I tried to port a simple menu script from VS code and found
out that script without arrays is useless.
So I had to implement arrays.
The design is set to support unlimited array dimensions
however for game engine two dimensions will suffice since I found a very nice
way to optimize it (I hope I will not regret it).
And since I started working on script again I also
implemented basic typecasting and variable scopes.
Typecasting is a little different than in C/C++. I tried
making it the same as in C/C++:
(typename) expression
but I got too many reduce/reduce conflicts so I had to find another
pragmatic solution. And that is:
<typename> expression
I know it is a little bizarre and it breaks the copy paste
from C/C++ code but it works. I will try and fix that later.
12.07.2007
Joystick
I took an opportunity and borrowed a couple of joysticks
from a friend to do a joystick support. And I completely underestimated the
task (wow – that’s a new one). All I wanted to do is a multiple joystick
support, device port recognition and… well that’s it. And a presumably 1 hour
task prolonged into a 5 hour suffering…
Nevertheless I did it: a multi joystick support with USB
port recognition (joystick input index and USB ports can be linked).
12.06.2007
Viewport
Working without any decant viewport system has become a real
annoyance (I sense the visibility will become similar annoyance very soon). So
I had to do a simple and elegant viewport design.
I wanted to support a multi viewport multi scene with custom
lights (e.g. menu over a scene).
And here is the design:

I also implemented light sets. Light set is a set of lights
that light chosen scene. Without light sets all active lights influence all visible
objects.
New:
- TViewportManager
- TViewport
- TViewportSet
- TRenderScene
- TLightSet
12.05.2007
Animation node linking
Insomnia has one skeleton animation. However you can use it
in two ways. You can bind the mesh on skeleton (soft bind) or you can drop the
mesh in joint (rigid bind).
12.03.2007
Node Optimization
Even in a static design some things can’t be static. One of
them is a TProp node inheritance. When it is activated it must link a TPropData
and this TPropData can be different every time activation is made. This means
that it must change its node child; which is prohibited in a static design.
To solve this problem I created a reserved node (or node
reservation). It is a single dormant node owned by TNodeManager and used for
one purpose only – to be linked on nodes that need a node reservation.
What you actually do is that you link a reservation node on
the Prop on creation time. Then when the Prop is activated you replace
reservation node with TPropData node and swap it back again on TProp
deactivate.
Few more node optimizations:
- Static node –is an un movable/un rotatable node, it can
only be hidden or shown.
- Socket node – only socket nodes can be search and named.
12.03.2007
Prop
Prop is a dynamic, scripted, compound, engine element with a
preset number of instances. You have limited instances of TPropData and
unlimited number of TProps. The TPropDataPool is responsible for prop data
assignment.
Example: Trees in a forest. TPropData is a tree and you set
number of instances to 50. Then you plant 5000 trees (TProp). The TPropDataPool
will take case that only the closest 50 trees (TProp) will have actual
TPropData.
New:
- TPropManager
- TPropData
- TProp
- TPropDataPool
12.02.2007
Visual Studio Plugin
The main source of my frustrations in the last couple of
month was inability to resolve my (beautiful) TName class string value (TName class
is only a single integer value that must be looked up in the name manager to
get a related string value and VS can’t do that) in debugger.
And today I draw a line (I have image to prove it) – I mean;
I wrote a VS debugger plug-in to resolve that issue. And I hope I will live
happily ever after.
12.01.2007
Collision groups
Collision must be able to filter the collision objects you
want to collide with. For that reason I implemented collision groups. Collision
groups are defined on per mesh basis.
Defined collision groups:
- Collision – used for general user collision query.
- Dynamics – used by dynamics.
- Player – used by player.
- Ballistics – used by projectile ballistics weapons.
- Melee – used by melee weapons.
- Decals – used for decals.
- Effects – used for effects.
AGEIA has an efficient custom collision filtering.
11.29.2007
Player and Controller
Today I did a player and controller design. Player is an
abstract class that links player input and information with the controller
which is an actual thing that moves in the environment.
For example controller can be either a first person or third
person human or a vehicle that player operates with.
AGEIA has a nice controller class that I used to make a
simple player.
11.28.2007
Cursor
Today I decided that I need a cursor. So I did an input cursor
class a multiple mice input system.
New:
11.26.2007
5th month anniversary
It’s been 5 months since I started this little pet project
of mine. And as for all anniversaries till now (except the 4th) I’ll
spend some time and compile the past (design) notes into more or less coherent
text.
Status and roadmap so far (+ marks new features):
- +Animation
- +Script
- +Threads
- +Collision (AGEIA)
- +Triggers
- +Resource editor (base)
- Render view set
- Player
- Props
- Network
- Animation automat
- Sound (FMOD)
- Blend shapes
- Post process rendering effects (DOF, HDR, Fog)
- Detail textures
- Transparency
- Visibility (portals)
- Effects (particle system, etc)
- Decals
- Instancing
- Engine SDK release
- Multi thread rendering
- PlayStation3 (driver)
- Xbox360 (driver)
- Resource editor (UI)
Hmmm… as you did (or did not) notice the list is not
shrinking!?
11.24.2007
Custom Maya classes
And it was time to write a custom Maya user interface so
that I can put other non-native objects in engine (e.g. triggers, dynamics, sounds,
player, etc.).
For that I wrote a simple property class that describes the
object properties and can create a new object from properties or modify an
existing one.
This property can be used for Maya objects description and
for real-time monitoring and tuning.
Here is the Maya property screen shot.

New:
Thoughts:
11.17.2007
Cooking with AGEIA
… or collision preparing for the rest of the word.
The collision detection is an essential part of the engine
and AGEIA have a really nice collection of collision detection functions.
On top of that these guys know how to do interfaces and how
things are done in games.
New:
- TCollisionManager
- TMeshCollision
- TMeshCollisionData
11.15.2007
Triggers
This simple thing was actually written now since it is
simple and I needed an interactive way to test script events. However simple
I’ve written some very optimized routines for line segment vs. box and sphere
routines (faster than AGEIA).
New:
- TTriggerManager
- TTriggerActor
- TTriggerGeometry
11.13.2007
Script Save Load Update
One very important thing in script is the ability to save
state. Without it you have to have separate code that saves the state and
script thread functions just loose appeal. Since the design of the script VM is
static the load/save only has to save call stack and registers.
The other thing that actually happen by chance is that script
can be dynamically rebuild while it is running. Well it will crash if you
change persistent thread functions while they are running but otherwise it
works great and it’s really useful.
I’ve also written a command functions (lots of them).
11.09.2007
Script Threads
Script threading is a really nice concept. It allows you to
write linear script which will run asynchronously with the rest engine. For
that goal you have persistent functions – these are functions that can pause
script execution and resume it on next script update.
There was also a major overall memory and speed optimization.
11.08.2007
Precompiled Headers
Compiling the auto-generated engine include file “system.h”
took almost 2 seconds so I decided and save the precompiled header for this
file. It takes only 2ms to load the precompiled headers.
11.07.2007
Engine Script Framework
The mail beauty of the script is that it is fast, simple and
it has the same framework as the engine. All engine public functions and
managers are accessible from the script (1:1 - engine:script).
I had to create static plug-ins to optimize script
initialization (plug-ins that have only one static instance).
11.05.2007
Virtual Machine
Virtual machine is designed to be simple and as fast as
possible. It is register based (no stack) and has only 30 instructions. It has
no dynamic allocations.
Speed comparison with VS 2005 SP1 C++ compiler (full
optimizations):
- recursion ~115 times slower
- iteration ~95 times slower
There is so much room for optimization (but enough time
spend on this for now – move on!).
11.01.2007
C+ Script
C+ is a simplified script language based on C with some
features of C++.
Script can be written with C++ source assistance programs
like VS InteliSense and VisualAssistX.
The error codes and formatting is the same as in VS so that
error tracing and error help can be used from VS.
Main differences with C:
- build-in types are int, float, vector and string
- function class parameters are used as pointers (void
Foo(TClass bar) { bar->DoIt(); } )
- global variables can have only a single constant assignment
(int a=1)
- no pre compiler (only #include statement)
- no structures and typedefs
- variables can be defined anywhere (C++)
- class can have only method definitions (C++)
Not implemented yet:
- arrays (already implemented)
- scopes (already implemented)
- typecasting (already implemented)
- function parameter of built-in type as a reference
- vector component access (a.x)
And here is the C+ syntax (updated):
program :
root_statement*
root_statement :
‘include’ ‘”’ file_name ‘”’
|
function
|
class_definition
|
global_variable_definition
function :
(‘thread’)? type name ‘(‘parameters_definition‘)’ ‘{‘ ( statement ‘;’ )* ‘}’
|
‘event’ (‘thread’)? type name ‘::’ name ‘(‘parameters_definition‘)’ ‘{‘ ( statement
‘;’ )* ‘}’
class_definition :
‘class’ name ‘{‘ (type name ‘(‘ parameters_definition ‘)’ ‘;’ )* ‘}’ ‘;’
global_variable_definition :
(‘property’)? type name (‘[‘ literal_int ‘]’ )? (‘=’ literal )? ‘;’
statement :
|
expression
|
‘if’ ’(‘ expression ‘)’ statement ( ‘else’ statement )?
|
‘for’ ‘(‘ expression ‘;’ expression ‘;’ expression ‘)’ statement
|
‘while’ ‘(‘ expression ‘)’ statement
|
‘do’ statement ‘while’ ‘(‘ expression ‘)’
|
‘switch’ ‘(‘ expression ‘)’ ‘{‘ ( case_statement ‘;’ )* ‘}’
|
‘break’
|
‘continue’
|
‘return’ ( expression )?
|
‘{‘ ( statement ‘;’ )* ‘}’
case_statement :
‘case’ expression ‘:’ statement?
|
‘default’ ‘:’ statement?
expression :
(type?) name
(‘=’|’+=’|’-=’|’*=’|’/=’|’%=’|’|=’|’$=’|^=’|’<<=’|>>=’)
expression
|
(‘-‘|’+’|’!’|’~’) expression
|
expression (‘==’|’!=’|’||’|’$$’|’^^’) expression
|
expression (’+’|’-’|’*’|’/’|’%’|’|’|’$’|^’|’<<’|>>’) expression
|
‘(‘ expression ‘)’
|
‘<’ type ‘>’ expression
|
expression ‘[‘ expression ‘]’
|
name ( ‘.’ name )+ ‘(‘ expression ( ‘,’ expression )* ‘)’
type :
‘int’ | ‘float’ | ‘vector’ | ‘string’ | name
parameters_definition :
( type name ( ‘,’ type name )* )?
10.27.2007
What happened to LUA?
It started quite nice. I had LUA script up and running in a
day. But then problems began to pile up. So many little problems:
- too simplified and inelegant syntax
- tons of dynamic allocations
- unable to do load in place
- no save/load interface
And it all led to conclusion that it will be more rational
to write my own script language (again!).
10.21.2007
Animation Function
I’ve created a mini animation virtual machine (really fast)
that is capable of performing various animation functions (set, blend and add)
on multiple animations, animation poses and skeleton poses.
I’ve also implemented constant animation motion
compensation, additive animations and
weight sets.
New:
- TAnimationFunction
- TAnimationWeightSet
- TAnimationPose
- TSkeletonPose
10.19.2007
Animations – the old way
Doing it the old fashioned way (that is saving matrix for
every frame) worked like a charm. Nothing easier – everything was already set
in place so I just had to switch curve with matrix and magic –animations are finally
working.
New:
- TAnimationMatrix
- TAnimationData
- TAnimation
- TAnimationManager
- TSkeletonMeshData
10.18.2007
Animations – the wrong way
Doing it the wrong way again! I always wanted to try and do
animations with Maya curves. You get a nice sample how to export and how to
evaluate the curves. Everyone is telling you how much better this is. And yes –
if you think about it: you have only few animation keys (much less data than
saving matrices in one way or another for every frame) and you have a perfectly
smooth evaluation. But nobody tells you that the evaluation code is not working
properly for all cases and nobody mentions that IK and limits are not in the
curves… (there is more stuff that gets even more complicated that I will not
mention… for my sanity sake).
10.3.2007
Effect State Manager
Looking at PIXWIN logs I noticed that tons of render states
are set redundantly over and over again (not to mention Begin/End –
Capture/Apply). The way to filter this redundant state changes is to write
custom effect state manager.
I’ve upgraded DX state manager with a little more filtering
(SetVertexDeclaration, SetStreamSource, SetIndices, SetRenderTarget) so that
all DX states are set only if really changed.
And here are the results of a “smart” effect state manager:
- ~1% speed improvement for ~20 batches
- ~24% speed improvement for ~150 batches
10.1.2007
commands
All commands are now linked with a buffered input (e.g.
alt+Enter, ctrl+Num/).
I’ve also added a multi monitor support (now you can assign
a key to move an application from one monitor to another).
action.toggle “ctrl+alt+Num*” device.adapter
9.29.2007
Dynamic buffers
I created a dynamic mesh data object for dynamic meshed. I
needed this class for some time to do an optimized canvas functions.
And so I did canvas functions:
- Line
- Box
- Circle
- Sphere
- Axis
- Frustum
- Arrow
New:
- TDynamicMeshManager
- TDynamicMesh
9.27.2007
Pro console and all my sins
I decided to fully finalize the console (to do both 90% of
it) and it all blow up in my face. I had to do tons of small thingies in order
to do that (old sins).
I also did a simple console design and insomnia icon
(warning programmer’s art ahead).

Parsing syntax:
Sequence
|
Action
|
^x
|
x=[a..r] will
set a system color table
|
^X
|
x=[A..) will
set a user defined color
|
^^
|
display a ^
|
^.
|
set original color
|
^:
|
set previous
color
|
^<
|
align left
|
^|
|
align center
|
^>
|
align right
|
|nnn
|
move to offset
nnn (as tab)
|
System font color table:
^Code
|
Color
|
^Code
|
Color
|
a
|
Red
|
j
|
White
|
b
|
Turquoise
|
k
|
Light Grey
|
c
|
Light Blue
|
l
|
Dark Grey
|
d
|
Dark Blue
|
m
|
Black
|
e
|
Light Purple
|
n
|
Orange
|
f
|
Dark Purple
|
o
|
Brown
|
g
|
Yellow
|
p
|
Burgundy
|
h
|
Pastel Green
|
q
|
Forest Green
|
i
|
Pink
|
r
|
Grass Green
|
9.26.2007
3rd month anniversary
And here we are at a 3rd month anniversary. The
last two months were a little bumpy. I’ve quit from my last job what gave me
more time for this project but brought other annoyance…
Anyways – it’s time to reflect on what was done and what
lies ahead (+ marks new features):
- + Render manager
- + Render To Texture
- + Deferred rendering
- + Deferred lighting
- + Shadows
- + Load in place Maya export
- + Console
- Animation
- Script (LUA + CPP)
- Threads
- Collision (AGEIA)
- Triggers
- Resource editor (base)
- Instancing
- Transparency
- Multi thread rendering
- Render view set
- Visibility
- Sound (FMOD)
- Effects (particle system, etc)
- Decals
- SDK
- PlayStation3 (driver)
- Xbox360 (driver)
- Resource editor (UI)
- Network
And today was still in the sign of old sins. I had to write
a buffered input and input focus to finish the console. Too be honest all the
console logic (text editing, auto-completion browsing, command history, etc.)
took much more time than I expected – but it is so sweet to have a fully
functional console.
New:
- TBufferedInput
- TInputFocus
- TConsoleManager
9.25.2007
Console
I noticed today that I still haven’t got a most basic hard
core thingy – the console. And so I spent a day in old sins – basic commands
(e.g. set, quit) and command auto completion.
9.24.2007
Parameter Cleanup
At the beginning I tried to use as many build-in parameters as
possible from Maya objects but it was getting more and more ugly so I added few
extra parameters – well several to be honest.
I also started creating a dedicated web page for the engine http://www.evi1m3.com/insomnia with
this dev-diary and download section and stuff.
And I made a simple installer application to ease the
installation and for better first impression.
Thoughts:
- I need an automated process to update this blog (winword
to web magic).
9.23.2007
Final data export
Today I finished a big thing. Maya plug-in can now export
final data for engine. Basically it required from engine to be capable to
restart at any point – and I already had that. What I had to do is to append
the initial resource data file with new data from Maya.
And that’s it folks.
9.19.2007
Dynamic array load in place
After some thinking I came to the conclusion that after all I
need dynamic array load in place. Resources and nodes are too limited if they
have a hard limit and it becomes a real annoyance to live with it – additionally
having a static array for resource tree bulks-up all classes derived from the TResource
class.
9.16.2007
Maya Renderer
How to put an engine into Maya window? Well not easy that’s
for sure. But there is a way –
Maya 8.5 has MViewportRenderer class which can be used to
derive a custom renderer. Unfortunately it is still a little buggy (sometimes
it start flickering or duplicate a render window in other windows) and API a
little incomplete (some functions are not implemented to the full extend).
But it is great. You can tweak everything in real time –
change parameters, move/rotate/scale objects,…

9.12.2007
Shadows
O my god… I completely forgot how frustrating the projection
matrices can be. I’ve been joggling with them for 2 days to get a decent spot
light shadow.
Well a lot of effort was also spent on a mechanism to do the
whole lighting with all shadows in a single pass.
New:
Thoughts:
- I will have to write a ps.2.0 for lighting – this will
cause lighting to be a little less efficient and slower but compatible
with previous gen.
- Directional light shadow – PSSM, CSM or what? None seems
to be good enough…
9.10.2007
…and all the lights
I implemented basic light types:
- directional light
- point light
- spot light
Thoughts:
- I need to write an effect state filter to optimize effects
begin/end state changing.
- LOD shader.
9.08.2007
Deferred rendering
The time has come to do the proper lighting. I decided to go
with deferred rendering since the results I’ve seen in past confirm that it is
at the moment best rendering method to achieve high quality real time
rendering.
In the next table are input channels and the how they are
packed in textures.
Channel
|
Texture
|
R
|
G
|
B
|
A
|
Color
|
Color
|
Cr
|
Cg
|
Cb
|
|
Alpha
|
Color
|
|
|
|
Ca
|
Normal
|
Normal
|
|
Ny
|
Nz
|
Nx
|
Parallax
|
Mask
|
|
|
|
P
|
Specular power
|
Mask
|
|
|
Sp
|
|
Specular mask
|
Mask
|
|
Sm
|
|
|
Reflection mask
|
Mask
|
Em
|
|
|
|
Self
illumination
|
Normal
|
I
|
|
|
|
I implemented deferred rendering in two steps:
- G-buffer creation
- Ligting
G-buffer is created with MRT (Multi Render Target).
Next table shows how the channels are being used.
Channel
|
R
|
G
|
B
|
A
|
Color
|
Cr
|
Cg
|
Cb
|
1
|
Position
|
Px
|
Py
|
Pz
|
Sp
|
Normal
|
Nx
|
Ny
|
Nz
|
Sm
|
Light
|
Cr*I
|
Cg*I
|
Cb*I
|
1
|
Lighting is a performed as a single full screen render with
all the lights fed into pixel shader.
New:
- TDeferredRenderingManager
Thoughts:
9.06.2007
Real rendering
I had to prepare for a real rendering. Mesh objects need to
be arrange in some order so I wrote a simple render queue manager.
I also need a Maya cameras and lights.
New:
- TRenderQueue
- TRenderQueueManager
8.24.2007
Maintenance
I added full resource management a.k.a. each resource knows
exactly his parents and children resources and knows how many times is it being
used by a parent resources and how many child resources are loaded.
Render targets were modified to coop with new resource
management.
I also found and fixed some memory leaks.
8.23.2007
Shaders rulz
Three days of shader reshaping and joggling finally paid off.
I found a simple and clean solution for shaders and shader programs. The point
is in using shared effect parameters and minimizing shader parameters settings.
I have three types of parameters:
- global parameters plugins– set only once per frame by
shader manager (eg. projection matrix, camera matrix, global shader
settings)
- local parameters plugins – set for each shader program that
uses it (eg. object matrix)
- local parameters – set for all programs on shader that
uses it
8.15.2007
Render target
Render target finalization and system architecture.
Multi resource management: system resources, game resources.
New:
- TRenderTarget
- TRenderTargetManager
8.14.2007
Windows windows
Windows window handling… holly crap… so much code for so
little functionality (resize, activate, suspend, etc.).
7.30.2007
Skeleton
Today I had to create a basic animation skeleton structure
to get all basics straight with shaders.
New:
- TSkeleton
- TSkeletonData
- TSkeletonJoint
- TSkeletonPose
- TSkeletonManager
Thoughts:
- Must use quaternions for local animation matrix.
- Try Maya animation curve.
7.29.2007
Shaders – finally
Finally I’ve gotten to the sweet part – shaders. I wrote
basic normal map and specular shader.
Implementing recursive shader compilation (if shader
compilation fails it opens a debugger and message window and after a message
window is closed it tries to compile again) and runtime shader recompilation
was a win. I also added notepad++ debug support – failed shader is opened in
notepad++ and send to error line.
Thoughts:
- Decompressing resources could be faster than reading them
from slow media
- Try Torrance-Cook specular method
7.28.2007
Maya round 2
Maya 8.5 presented a new way of acquiring a mesh already
prepared for rendering – which is unfortunately not working properly ATM. So I
had to do it myself.
Currently exported Maya objects:
- Mesh
- Skin
- Face
- Vertex
- Position
- Color
- Normal
- S and T (Tangent space)
- Textures
- Light
- Camera
And so I was able to play a little with shaders.
7.26.2007
It’s 1 month anniversary
And I think it’s time for the roadmap.
- + Load in place
- + Resource system
- + Manager system
- + DirectX (driver)
- + Input system
- + Command system
- + Dynamics (AGEIA)
- + Localization
- + Maya exporter
- Render manager
- Render To Texture
- Deferred rendering
- Shadows
- Animation
- Resource editor (base)
- Threads
- Multi thread rendering
- Render view set
- Visibility
- Sound (FMOD)
- Collision (AGEIA)
- Effects
- Decals
- Transparency
- Script (LUA + CPP)
- PlayStation3 (driver)
- Xbox360 (driver)
- Resource editor (UI)
- Network
7.25.2007
Localization
All display texts will be saved as uni-string. Yes – I’m steaming
it out.
New:
- TUniString
- TDialog
- TLocalizationManager
7.20.2007
The right way and the wrong way
“Collada research” or “what a waste of time”. In theory it
is a very compelling idea but the implementation and usage and speed and code…
OMG. I had to port the code to compile it under Maya 8.5 not to mention that
source code needed to compile it is hidden all over the net. And when you have
all up and running it’s not running because it is too slow.
A day !well spent.
7.19.2007
Memory cleanup
Clearing the resources. Font stayed in. How to find DirectX
memory leaks. Hideous task...
CRT Memory leaks.
Thoughts:
- Deferred rendering
- Could shader program be a resource so that it can be saved
as resource - precompiled?
7.18.2007
Dynamics
After no breakthroughs with Maya I had to shift my focus on
dynamics (for sanity sake). And surprisingly all wend according to plans. AGEIA
knows its ways. Interface is clean and straight to the point and more importantly
it is working. I had the whole interface up and running in not time at
all. And more you have a remote debugger and visualization. To simply put – it
is great.
Thoughts:
- Engine SDK/ license/ open source...
Notes:
7.16.2007
Maintenance
After yesterdays fiasco I decided to ease my mind and do a long
postponed maintenance. So I finalized command system: simple compiler, basic
commands, cleanup, testing... and now the engine main looks a whole lot
cleaner.
I also implemented manager locking system to control manager
internal usage. Internal usage is allowed only after manager initialize and
until manager finalize is called.
Thoughts:
- I have to write effect state manager to optimize render
state changes.
- Find an efficient way to extract effect states for non
render state aware platforms (cg based).
Note:
- All commands parameters must be validated.
- Basic commands (set, bind, etc.)
7.15.2007
Maya round 1
I find myself again before the Maya API. Not as enthusiastic
as yesterday, when I thought that this will be done in a sec. Anyways I need to
get that exporter running – writing an exporter from scratch would be too long,
suffering, experience (and I did that already).
7.14.2007
From Maya to AGEIA and back
I was very disappointed when I found out that Maya .x export
plugin was NOT in the June DirectX SDK and even more disappointed when I found out
that it is uncompilable in February version of DirectX SDK...
So to cool my steam I redirected focus on dynamics. I took a
quick look at AGEIA and Bullet Physics Library SDKs documentation and demos...
and AGEIA won.
So I need to implement it. But first I need to take care for
my ladies.
Thoughts:
- TResource needs ‘ResourceLost’ and ‘ResourceReset’ functions
to coop with system reset capability (windows DirectX only).
- String array one limited memory pool for string data and
sArray<const char*,max>.
7.13.2007
First demo 0.23
Shaders are still uncompiled (for development purpose)
however the engine is ready for some test driving, to be honest a ten second test
and you’ve seen it all because there is not much to see.
It’s what beneath the hood that actually plays the
difference:
- 8ms launch time of which 8ms is taken by DirectX device
creation.
- 7 memory allocations
- 7 disk reads
- 122 objects constructor calls
In a word LightningFast!
You can download it here. For running you will need DirectX
9.0 June (d3d9_34.dll) and msvcr80.dll and msvcp80.dll.
7.12.2007
Where are my textures?
It all started pretty badly. Something that should be done
in a sec took me nearly two hours. Hell in PIXWIN...
So I exercised my thought from yesterday and added a macro
around all DirectX calls, reporting all errors and I managed to narrow down a
bug in a mere minute.
Anyways, in a day or two I will be ready to show first
screen shot... I hope. Don’t expect too much!
Thoughts:
- Today I was thinking a lot about a resource editor; how to
easily interconnect resources into a coherent dependency list; it also
needs to be platform and localization friendly.
Lesson:
- Always check everything for errors.
7.11.2007
Let’s look around
Today was a good day. I managed to write a basic camera WSAD
+ mouse movability. It was a great test of the input architecture (it’ flawless
if I may add).
I also added DirectX .x file import to include textures and
multiple materials per mesh.
Thoughts:
- Trap all function error results.
Note:
- Save precompiled shaders.
7.10.2007
Should all manager be dynamic?
I’m not so sure anymore... after a not so good night sleep I
reevaluated the dynamic vs. static managers I got to a new conclusions. What I
got with dynamic managers is:
- Control over when managers are created (+).
- Additional new calls and management (-).
- Still no control over what can and cannot be done in
constructors (-)
After some deliberation I found a simple way to gain control
over constructors issue. Basically it is an old trick that I already practice
in the file system. That is to lock the file system until it is safe to use it.
And with that solution all managers can be static since it doesn’t matter anymore
when they are created because they can only do stuff that is allowed. And it is
also negligibly faster.
I finally written a test manager class with a simple forward
rendering.
Thoughts:
- Asynchronous resource reading must collect all resources
to be loaded and sort them by file offset.
- I need render manager.
7.09.2007
All manager must be dynamic.
The experience from the old engine thought me a lesson about
static managers. They are a pain to control the creation order (does ‘#pragma
init_seg(lib)’ ring any bells) and what is allowed and not allowed to be done
in constructors.
I also need a command interface with user commands and
variables. Commands are custom plugins to execute specific commands (e.g.
‘quit’) while variables are simple callbacks for user variables (e.g. ‘set
window.resolution 1024 768’) that can be queried from within the engine.
Shaders... the final frontier. The last several times I
designed (fixed) the system for shaders I always did something wrong (3:0 -
shaders vs. me). So based on those experiences I embarked on an epic designing
once again and the time will show the score this time.
New:
- TVariant – variant type that can hold either int, float or
vector value (used for TVariable)
- TCommand – command interface class
- TVariable – command
- TCommandManager -
- TColor – very simple unsigned int color wrapper class.
- TBox - with center and extends. I’ve been using box class
with min and max position for years and
- TCanvasManager
- TFontManager
- Basic font functionality to write text.
- TDxFontManager implementation
Note:
- Font caching must be written to reuse frame to frame text
render.
- Write command compiler.
7.08.2007
And the name is...
I’ve finally given some thought on how to name my new
engine. Well my motto for the engine is to be fast and simple, so I was considering
a name ‘simplex’ (Latin for simple). However the fact that I work on the engine
in late night hours gave me a thought to name it ‘insomnia’. So until further...
Anyways it was time to know what time it is so I added a
simple timer class with basic functionality (time query, delta time, pause and
resume, etc.)
New:
7.07.2007
Input
Today I was looking at a something in my brand new window
with an even newer camera. I know what it should be even if it doesn’t look
like it... yet.
And it snapped “I need an input!”. So I created a plugin
driver based input system and wrote driver for mouse and keyboard.
New:
- TInput
- TInputManager
- TWinInputManager
Thoughts:
- Buffered input for console.
- Windows input driver for Xbox 360 controller.
- Input action linking.
- Game friendly input focus system.
7.06.2007
Plugins
[TW] Plugin
New:
7.04.2007
Camera
I have window, I can render a mesh... now I need a camera to
look with. A simple Maya based projection system (aperture width, height and
focal distance) will suffice.
New:
7.03.2007
First window
Windows window and main DirectX initialization. I hate that
part but it needs to be done in order to get to the good part.
I’ve used D3DX utilities to load .x and .dds files and
converted them into my native format.
6.31.2007
Huston... we have a __vfptr
Well... I was certain that storage concept was working. I
did save all the arrays and load them back, update the pointers and it all work
fine... until I tried load only and change the code a bit so that something
changed. And it took me one hour to figure it out what went wrong. Today’s
topic should give you a clue.
Yes, I forgot to update the virtual function pointers (or
__vfptr as you see it in VS). It was pretty easy from here on.
I decided to put all platform dependant functions (for threads,
input output, atomic functions, system messages, kernel functions, etc.) into
single header file for easier porting.
I also written a log system with .html preview
‘system.log.html’ (sweet if you ask me) and an attribute wrapper class (not
really worth mentioning).
New:
Thoughts:
- Dynamic memory allocation is very slow – reduce usage to
minimum.
- Remove all unnecessary constructors.
- Engine must not have any linear searches in run-time.
- Simple way to mark and track slow functions (as linear
search).
6.30.2007
Testing the concept
I had to write a couple of engine classes and managers to
test the concept.
I also wrote a TName class. It is basically a single integer
that holds a CRC 32 value of a name. This class can be easily saved (as int)
and provides an extremely fast way for searching, sorting and mapping. However
a asChar() resolving function is little slower.
[TW] Template naming convention (DdSs).
New:
- TName
- DSet
- SSet
- SMap
- TDeviceManager
- TDxDeviceManager
- TMaterialManager
- TDxMaterialManager
- TTextureManager
- TDxTextureManager
- TShaderManager
- TDxShaderManager
6.29.2007
Pointers are bad... are they?
I have two types of storage pointers, known and inherited. Known
pointers are pointers of a known storage class types (e.g. TNode) while
inherited pointers are pointers that are inherited from other storage classes
(e.g. TMeshData is inherited from TResource which is storage class).
Storage system must be able to distinct between these two
types in order to fix them.
[TW] More detailed about storage system and load in place.
New:
- TManager
- TManagerManager
- TNode
- TNodeManager
- TResource
- TResourceManager
- TDxMeshManager
Thoughts:
- Name must be simple int class with CRC32 for index.
- I really need a static map template.
- Precompiled headers are a must.
- Everything that I write must be either final or mark
explicitly and noted for future update.
6.28.2007
It must be data driven
An engine must be data driven. All starts and ends with resource
data manipulation and how efficient you are with it.
Data is divided in two parts resource properties and
resource data. All resources property is loaded in main memory and connections
fixed while resource data must be prepared in such way that it can be directly
loaded into video memory without any modifications.
[TW] Storage system
[TW] Cluster system
New:
- TStorage –storage system for resource properties pointer
resolving
- TCluster – array registration and storage functionality
- TClusterManager
- TMesh
Thoughts:
- Separate tool chain engine from runtime engine.
- Separate data into two chunks – resource properties and
resource data
- Load properties as one block and fix pointers
- Protect everything that shouldn’t be accessed directly and
write access functions.
6.26.2007
New Engine
An accidental late night playing with a thought how bulky my
old engine spawned a bold question “Why not write a new one... from
scratch... brand new... shiny... with all the new stuff I cannot do in the old
one...?”
Any a new question few seconds after “Is this feasible for
one man alone?”
And so I spend the rest of the night writing random
templates and functions with an agenda to write a new engine.
Thoughts:
- Simple design
- Pragmatic/direct approach
- Speed
- Static data members
- Portable
- Data oriented
- Plugin based