Lesson 16: Resource Parsing

Now that we have a way for our program to render models easily and in multiple places in our OpenGL window, we can look at making this virtual world building process easier and more streamlined.
In this lesson, we'll look at how we can load an entire scene from a file, rather than have every model, texture, and entity hard-coded into our program.
This makes the process of rapidly prototyping different designs much easier, as once again we'll add the ability to hot-reload these scenes while our program is running. This also provides a convenient way in the future to have multiple scene objects in our program, should we wish to have more than one scenario or level that our program can run.
Designing a scene file
If we're going to parse all the objects in our world from a file, we should first take a moment to think about what the file will look like and what it needs to contain.
I'm going to concentrate on loading models, textures and entities from a file for now to keep things relatively straight-forward. Over the next few lessons we'll progressively add more things such as reading in the camera settings, but for now we'll just focus on these.
I'm also going to use a really simple text file format. While JSON or XML formatting might work quite nicely for this task and provide some level of protection against making obvious mistakes, I've opted not to use them here. There are a few reasons for this, including because we've already looked at some text file parsing before for model loading. I also don't want these lessons to push specific formats where possible, especially for things not related to graphics, and this means we avoid adding any non-core dependencies to our code. So, we'll just stick with a really trivial text file format for now.
There's absolutely no problem if you want to implement this lesson using one of those formats though, there's nothing in this lesson or future lessons which would make it problematic. I'd probably even encourage it, I just want that to be your decision!
The plan for storing all our texture, model, and entity data in a file therefore looks something like this:
1. |
|
2. |
|
3. |
|
4. |
|
5. |
|
6. |
|
7. |
|
8. |
|
Hopefully it's fairly intuitive. Each line of our scene file starts with the kind of object we're defining, followed it's name and it's parameters.
We can parse the file relatively easily by looping over each line and splitting each of them by spaces, just like we did to load Wavefront OBJ models. If the first token on the line matches "Model", "Texture" or "Entity", we'll create a new object of that type and then use the next token as the object's name. We can then attempt to parse more tokens from the line to read in the parameters of the object.
Any lines in the file where the first token isn't one of "Model", "Texture" or "Entity" we'll assume is just a comment, so we'll ignore the line entirely.
For handling entities, things are slightly more complicated.
Currently in our code these use pointers to the relevant model and texture objects which they use for drawing.
In our file format we instead reference the name of the texture or model to use for the entity.
So in the example above, both entities will use the texture named myTexture
and the model named myModel
for drawing.
To implement this, we'll need to think about a few edge cases, but it shouldn't be too difficult. The main risks are that an entity references a model or texture which isn't defined in the file, or one which is defined later on in the file, but hasn't yet been parsed/created. A solution to both of these scenarios is that we can first parse the file and create all the textures and models, and then do a second pass this time just concentrating on the entities. Again this is similar to how we built our model loader, and means that when we're creating an entity, we can easily verify that the corresponding models and textures do exist and get a pointer to them.
That's basically it for our scene file. Not the most elegant file format ever invented, but it should be easy for us to work with, and quick for us to throw together a parser for. Again, I only use this to guide you on how it can be done, feel free to play with it and modify it as you like.
Scene class
To handle loading this file and managing it's contents, let's write a new "Scene" class.
We'll declare it's structure in a new file called scene.h
:
+ 1. |
|
+ 2. |
|
+ 3. |
|
+ 4. |
|
+ 5. |
|
+ 6. |
|
+ 7. |
|
+ 8. |
|
+ 9. |
|
+ 10. |
|
+ 11. |
|
+ 12. |
|
+ 13. |
|
+ 14. |
|
+ 15. |
|
+ 16. |
|
+ 17. |
|
+ 18. |
|
+ 19. |
|
+ 20. |
|
+ 21. |
|
+ 22. |
|
+ 23. |
|
+ 24. |
|
+ 25. |
|
+ 26. |
|
+ 27. |
|
+ 28. |
|
+ 29. |
|
+ 30. |
|
+ 31. |
|
The class re-uses much of the same structure we've used before, so should be quite familiar.
The header file starts by including our texture, model and entity headers as we'll need access to those classes. We also include the map header from the standard library to give us an easy map/dictionary implementation to store each object we create from the file and to reference it by name. These are defined at the bottom of the class, the private map variables, which use a string as the key and the value is either a texture, model, or entity.
Once the scene has been loaded, the entities map will hold everything which needs to be drawn in the world, so the draw function is then simply a matter of iterating through the entire map and calling draw on each item.
Scene implementation
We can now write the code to define those functions in scene.cpp
:
+ 1. |
|
+ 2. |
|
+ 3. |
|
+ 4. |
|
+ 5. |
|
+ 6. |
|
+ 7. |
|
+ 8. |
|
+ 9. |
|
+ 10. |
|
+ 11. |
|
We start by including the class's header file, the SDL header which we'll need for making file paths absolute, and the fstream
and sstream
libraries which we'll need for manipulating files and strings.
The first function we've written is setFilename
.
Like the rest of our classes, we'll accept a relative path from the executable to the file as a string.
The function then prefixes the executable's location to make it an absolute path, and stores the result in the class's filename
variable.
We can then move on to our loadScene
function, which will do the parsing of the file:
10. | filename = SDL_GetBasePath() + newSceneFilename; |
11. | } |
12. | |
+ 13. |
|
+ 14. |
|
+ 15. |
|
+ 16. |
|
+ 17. |
|
+ 18. |
|
+ 19. |
|
+ 20. |
|
+ 21. |
|
+ 22. |
|
+ 23. |
|
+ 24. |
|
+ 25. |
|
+ 26. |
|
+ 27. |
|
+ 28. |
|
+ 29. |
|
30. | |
31. | ... |
The function returns a bool indicating success, where if we fail to parse and load the file, we'll return false and set the errorMessage
variable to give an indication as to why loading failed.
The first thing we do is to reset the error message and call deleteScene
, to make sure that if this function is called repeatedly (as happens when hot-reloading a scene), any old data is properly deleted before re-reading the file.
We also perform a check that a filename has been set before proceeding to try and open the file.
With the file open, we can start looping through it line-by-line:
27. | errorMessage = "Unable to open file"; |
28. | return false; |
29. | } |
30. | |
+ 31. |
|
+ 32. |
|
+ 33. |
|
+ 34. |
|
+ 35. |
|
+ 36. |
|
+ 37. |
|
+ 38. |
|
+ 39. |
|
+ 40. |
|
+ 41. |
|
+ 42. |
|
+ 43. |
|
+ 44. |
|
+ 45. |
|
+ 46. |
|
+ 47. |
|
48. | |
49. | ... |
As we discussed, we'll parse the whole file twice, with the first run just focusing on just textures and models.
We use the getline
function to read each line of the file into the line
variable in a loop until we hit the end of the file.
We start processing each line by first creating a stringstream from it to help us with the parsing.
The parsing starts by creating a set of string variables we'll use to hold the first three tokens of the line: the object type, it's name, and it's filename. We then use the stringstream to read the first three space-separated tokens of the line into these three variables. This is safe to do even if the line doesn't contain three tokens!
Then, we can test the value of objectType
against the literal "Texture" or "Model" to decide if this is a line we're interested in, otherwise it gets ignored and we continue to the next line.
Let's now fill in the code block for handling a line defining a texture:
39. | if(objectType == "Texture") |
40. | { |
+ 41. |
|
+ 42. |
|
+ 43. |
|
+ 44. |
|
+ 45. |
|
+ 46. |
|
+ 47. |
|
+ 48. |
|
+ 49. |
|
+ 50. |
|
+ 51. |
|
+ 52. |
|
+ 53. |
|
+ 54. |
|
55. | } |
When we find a line starting with the literal "Texture", we make a call to the setFilename
function of the texture in the map with that name.
If the map doesn't contain an entry for the key objectName
, which it won't at the start, the object will implicitly be created, and then the setting of the filename will be done.
With the texture created we make a call to the texture's loadTexture
function to actually attempt to open the image and copy it to the GPU.
If something failed while loading, we can use the texture's error message as the scene's error message.
We also need to remember to delete anything from the half-loaded scene and close the file before returning in this case!
Otherwise that's all we need to do for parsing textures!
Once this block of code has looped over each line in the file, our textures
map will now contain all textures from the scene file, successfully loaded onto the GPU.
Next, we'll do the same for models. The code is largely the same:
56. | else if(objectType == "Model") |
57. | { |
+ 58. |
|
+ 59. |
|
+ 60. |
|
+ 61. |
|
+ 62. |
|
+ 63. |
|
+ 64. |
|
+ 65. |
|
+ 66. |
|
+ 67. |
|
+ 68. |
|
+ 69. |
|
+ 70. |
|
+ 71. |
|
72. | } |
73. | } |
We again create a model in the corresponding map with the specified name, set it's filename, and then attempt to load it.
With this in place, our block of code looping through the file for the first pass is complete. When the code has finished running, we'll have all of our textures and models loaded on the GPU, ready to go.
We'll now go back to the start of the file and attempt to load any entities:
70. | return false; |
71. | } |
72. | } |
73. | } |
74. | |
+ 75. |
|
+ 76. |
|
+ 77. |
|
+ 78. |
|
+ 79. |
|
+ 80. |
|
+ 81. |
|
+ 82. |
|
+ 83. |
|
+ 84. |
|
+ 85. |
|
+ 86. |
|
+ 87. |
|
+ 88. |
|
+ 89. |
|
90. | |
91. | ... |
We reset the file to start reading it from the beginning again, loop through it line-by-line, and this time look for lines beginning with "Entity".
Entities require a bit more effort to parse from our file than with the textures or models.
Rather than the line simply being type, name, filename, they have a set of key/pair values on the line covering various parameters.
As a result, we don't create the filename
variable and parse three tokens as we did before.
Instead, we'll need something a bit more robust. Once we know we're looking at an entity definition line, we'll create a loop in our code which will continually attempt to parse two more tokens from the line, until there's nothing left on the line to read in. We can then treat of each of these pairs of tokens as a key/value pair for the entity's parameters. So if we see "x" as the first token, we'll expect a float for the second token of the pair which we can use as the entity's "x" coordinate.
To do this, we'll create a loop in our code like this:
85. | if(objectType == "Entity") |
86. | { |
+ 87. |
|
+ 88. |
|
+ 89. |
|
+ 90. |
|
+ 91. |
|
+ 92. |
|
+ 93. |
|
+ 94. |
|
+ 95. |
|
+ 96. |
|
+ 97. |
|
98. | } |
99. | } |
When we see an entity definition line in the file, we create a pair of strings and read the next two tokens from the line into them. We can then handle this key/value pair of parameters in the loop, and at the end blank the strings and attempt to read two more parameters from the line into them. This loop will keep running for each pair of tokens until we hit the end of the line.
Into this loop, we can then handle each kind of parameter we want to parse. First, textures:
90. | while(value.length()) |
91. | { |
+ 92. |
|
+ 93. |
|
+ 94. |
|
+ 95. |
|
+ 96. |
|
+ 97. |
|
+ 98. |
|
+ 99. |
|
+ 100. |
|
+ 101. |
|
+ 102. |
|
+ 103. |
|
+ 104. |
|
+ 105. |
|
106. | ... |
If the first token of the pair is they key "texture", then the second token is the name of the texture object we should use.
As all textures in the file will have already been loaded into our textures
map, configuring the texture for the new entity is quite easy.
We first perform a check that a texture with that name exists somewhere in the map.
Assuming it does, we can then set the entity's texture to the address of the texture in the map.
We can do the same for reading in the entity's model:
104. | entities[objectName].setTexture(&textures[value]); |
105. | } |
+ 106. |
|
+ 107. |
|
+ 108. |
|
+ 109. |
|
+ 110. |
|
+ 111. |
|
+ 112. |
|
+ 113. |
|
+ 114. |
|
+ 115. |
|
+ 116. |
|
+ 117. |
|
+ 118. |
|
+ 119. |
|
120. | ... |
Again checking a model with the specified name exists in the map, and setting it as the entity's model.
We can then finish the parameter parsing loop by adding some code to allow us to configure the position and orientation of the entity from the file:
118. | entities[objectName].setModel(&models[value]); |
119. | } |
+ 120. |
|
+ 121. |
|
+ 122. |
|
+ 123. |
|
+ 124. |
|
+ 125. |
|
+ 126. |
|
+ 127. |
|
+ 128. |
|
+ 129. |
|
+ 130. |
|
+ 131. |
|
+ 132. |
|
+ 133. |
|
+ 134. |
|
+ 135. |
|
+ 136. |
|
+ 137. |
|
+ 138. |
|
+ 139. |
|
+ 140. |
|
+ 141. |
|
+ 142. |
|
+ 143. |
|
144. | |
145. | key = ""; |
146. | value = ""; |
147. | iss >> key >> value; |
148. | } |
149. | } |
150. | } |
If we see a key/value pair where the key is "x", we make a call to updatePosition
on the entity.
The value is converted from a string to a float using the stof
function.
As the entity will default to a position of (0, 0, 0), updating it's position by this value will in effect set it's x coordinate to this value without changing the rest of the entity's coordinates.
The same is done if we see the keys "y" or "z", updating the entity's y or z value instead accordingly. Likewise for the orientation too, if we see a key value of "rx", we update the entity's rotation in the x axis, etc.
That completes the entity configuration loop, now giving us a way to create entities and configure their models, textures, positions and orientations from a file. This construction has the additional benefit of making us robust to the order of the parameters too, so it's perfectly fine for the file to contain the position of the entity first on the line, and then reference which model and texture it should use.
We can now add the finishing touches to our loadScene
function:
145. | key = ""; |
146. | value = ""; |
147. | iss >> key >> value; |
148. | } |
149. | } |
150. | } |
+ 151. |
|
+ 152. |
|
+ 153. |
|
+ 154. |
|
+ 155. |
|
After we've finished looping through the file looking for entities, we make sure the file gets closed before returning true to indicate success. All textures, models, and entities will now be properly configured and loaded, and stored in their corresponding maps.
With the parsing of the input file complete, we can move on to writing the rest of the class's functions.
Next, let's write the deleteScene
function to clear up the scene after we're finished with it:
154. | return true; |
155. | } |
156. | |
+ 157. |
|
+ 158. |
|
+ 159. |
|
+ 160. |
|
+ 161. |
|
+ 162. |
|
+ 163. |
|
+ 164. |
|
+ 165. |
|
+ 166. |
|
+ 167. |
|
+ 168. |
|
+ 169. |
|
+ 170. |
|
+ 171. |
|
+ 172. |
|
For each texture, we need to call it's deleteTexture
function to make sure that it's memory on the GPU is cleared up.
We loop over the map, and for each element (a pair of it's name, and the texture object itself), we call the delete function on the second item, the texture object.
We then do exactly the same on the models, which will free up all the model's buffers on the GPU. Entity objects don't have any GPU or dynamically allocated memory themselves, so we don't need to do anything explicitly for them.
With all the memory freed, it's then just a matter of clearing/emptying each of the maps to return our scene object back to a default state.
Next, we can look at the draw function for our scene:
171. | entities.clear(); |
172. | } |
173. | |
+ 174. |
|
+ 175. |
|
+ 176. |
|
+ 177. |
|
+ 178. |
|
+ 179. |
|
+ 180. |
|
Drawing every entity in the scene is then simply a matter of looping over the entities map and calling draw on each object!
We can finish up the implementation of our scene class by defining the last few getter functions:
178. | element.second.draw(); |
179. | } |
180. | } |
181. | |
+ 182. |
|
+ 183. |
|
+ 184. |
|
+ 185. |
|
+ 186. |
|
+ 187. |
|
+ 188. |
|
+ 189. |
|
+ 190. |
|
And with that, our scene object is complete!
Makefile
We've added a new source file, so let's now make sure it gets compiled:
1. | CC = g++ |
2. | |
+ 3. |
|
4. | |
5. | INCLUDE_DIRS = -IC:\SDL3\include -IC:\SDL3_image\include -IC:\glm -IC:\glew\include |
We need to make sure that scene.cpp
is added to the list of files to be compiled in our Makefile.
Loading a scene
We can now integrate our scene class into our main.cpp
.
First, we need to include it's header:
1. | #include "shader.h" |
2. | #include "texture.h" |
3. | #include "model.h" |
4. | #include "entity.h" |
+ 5. |
|
6. | |
7. | #include <SDL3/SDL.h> |
8. | #include <SDL3/SDL_main.h> |
9. | #include <SDL3_image/SDL_image.h> |
Then, we can replace all of our models, textures, and entities we were previously using with a single scene variable:
37. | Shader mainShader; |
+ 38. |
|
39. | |
40. | bool init() |
41. | { |
All of those variables are removed entirely, we won't need them at all any more.
We've replaced them instead with a scene object called crateScene
.
We can then do exactly the same in our init
function:
110. | mainShader.setFilenames("shaders/main_vertex.glsl", "shaders/main_fragment.glsl"); |
111. | if(!mainShader.loadShader()) |
112. | { |
113. | printf("Unable to create shader from files: %s\n", mainShader.getFilenames().c_str()); |
114. | printf("Error message: %s\n", mainShader.getError().c_str()); |
115. | return false; |
116. | } |
117. | |
+ 118. |
|
+ 119. |
|
+ 120. |
|
+ 121. |
|
+ 122. |
|
+ 123. |
|
+ 124. |
|
125. | |
126. | SDL_SetWindowRelativeMouseMode(window, true); |
We rip out all the old code for manually creating textures, models, and entities, and instead replace it by just setting the path to our scene and attempting to load it.
The same is then done to our closing code for when the program has finished:
137. | void close() |
138. | { |
+ 139. |
|
140. | mainShader.deleteShader(); |
141. | |
142. | SDL_GL_DestroyContext(context); |
143. | SDL_DestroyWindow(window); |
144. | SDL_Quit(); |
145. | } |
We strip out all the previous code for deleting our textures, models, and entities, and instead just call delete on the scene object.
Our draw function gets the same treatment:
277. | mainShader.bind(); |
278. | |
279. | glUniformMatrix4fv(0, 1, GL_FALSE, glm::value_ptr(pMatrix)); |
280. | glUniformMatrix4fv(1, 1, GL_FALSE, glm::value_ptr(vMatrix)); |
281. | |
+ 282. |
|
283. | |
284. | mainShader.unbind(); |
285. | |
286. | SDL_GL_SwapWindow(window); |
287. | } |
No individual draw calls are made any more. Instead, drawing involves binding the shader, setting the perspective and view matrices, and then calling draw on our scene, which will handle all the individual objects.
Finally, we'll update our handleEvents
too.
We strip out all the old variables and alter our code so that if the user presses "r" on their keyboard, we just hot-reload the scene rather than everything individually:
191. | else if(event.key.key == SDLK_R) |
192. | { |
193. | if(!mainShader.loadShader()) |
194. | { |
195. | printf("Unable to create shader from files: %s\n", mainShader.getFilenames().c_str()); |
196. | printf("Error message: %s\n", mainShader.getError().c_str()); |
197. | programRunning = false; |
198. | } |
+ 199. |
|
+ 200. |
|
+ 201. |
|
+ 202. |
|
+ 203. |
|
204. | } |
The call to loadScene
will already have the scene's filename cached, and the first thing a call to loadScene
will do is safely delete the old scene.
So this call will throw away everything we had and then attempt to reload everything while the program's running.
Bear in mind that for complex scenes, as this is all done on the main thread currently, this may cause the program to stall for a moment until the loading process has finished. For me that's fine as I see this more as a useful debugging tool. If you wanted to have dynamic scene loading as an integral part of your program though then maybe you would show a loading screen to the user while it's happening, or push the call to another thread, rather than have the program appear to freeze momentarily.
With that, our code is now complete!
Recreating our crates
To test our new scene system is working, let's use it to recreate the crates from the previous lessons. To do that, we can write a scene file in the following way:
+ 1. |
|
+ 2. |
|
+ 3. |
|
+ 4. |
|
+ 5. |
|
+ 6. |
|
+ 7. |
|
We start by defining a texture and model for the crate, and the filename where each of these resources can be found.
We've then added 3 entities using these models and textures, and position them as they were before. If we run our program with this file, we should end up with exactly what we had in the previous lesson (just without all the hard-coding global variables!).
Summary
We now have a way to create a scene with tens, hundreds or even thousands of entities in our program without having to pollute our source code with pages of variables!
We also have a way to hot-reload those scenes if we need to, and even load several scenes into our program if want to switch between multiple scenes or levels.
Our virtual world is well on it's way!
In the next lesson we'll take this a step further, and see how we can build a fully-fledged virtual world into our program. See you there!