Best way to handle large amounts of "pops" with unique stats attached to them?

Summary · qwen2.5:32b

The developer is seeking efficient methods to manage large numbers of "pops" (population units) with unique stats in a 4X game using Godot, noting that current array-based solutions become slow at scales over 10,000 pops. The article questions how games like Stellaris efficiently handle large populations by potentially clumping similar pops together to reduce computational load. One concrete detail is the use of an array structure and enum for pop attributes like health and knowledge to manage individual pop stats and operations.

Excerpt

<!-- SC_OFF --><div class="md"><p>I have taken the advice of many people on here and for the past few months I have broken up this 4X game I am making into digestible elements. Each element being its own game so I can gain some practice. My current project is handling the pops (using Godot).</p> <p>My goal is to eventually have pops that have a stat like home province, current province, health, age, innovation, ability, and courage. </p> <p>The game is turn-based, so every time the &quot;end turn&quot; button is pressed the game determines what needs to be calculated then actually goes through and calculates things. So, for example, it'll check if there are active institutes in the world, if there is, then it knows to find pops in these institutes and improve their innovation by some small value.</p> <p>So here is where things get trickier.</p> <p>Initially I tried to have each pop be an object, which I realized was a bad idea right away do to loss of performance when accessing lots of pops. So I switched to an array format because I thought it may help with performance (?).</p> <p>This is the first mock-up I came up with: </p> <p><code>enum pop {</code></p> <pre><code>`POP_HEALTH,` `POP_KNOWLEDGE` </code></pre> <p><code>}</code></p> <p><code>var pops: Array[Array] = [</code></p> <pre><code>`[],` `[]` </code></pre> <p><code>]</code></p> <p><code>var empty_pop_slots: Array = []</code></p> <p><code>func remove_pop(id: int):</code></p> <pre><code>`if id &gt;= len(pops[pop.POP_HEALTH]) or id &lt; 0:` `return` `for p in pops.size():` `pops[p][id] = null` `empty_pop_slots.append(id)` </code></pre> <p><code>func add_pop(... properties: Array):</code></p> <pre><code>`if properties.size() &lt; pops.size():` `return` `if empty_pop_slots.size() == 0:` `for p in pops.size():` `pops[p].append(properties[p])` `else:` `var trash = empty_pop_slots.pop_back()` `for p in pops.size():` `pops[p][trash] = properties[p]` </code></pre> <p>This improved performance a lot but there is still ultimately an issue. It is slow at larger pop scales. Even something like 10,000 pops can take a while to cycle through and modify properties or check for certain values.</p> <p>So I needed to think differently. I wanted to know how games like stellaris handle these things, and it seems that games with large pop amounts tend to &quot;clump&quot; their pops together, but I was struggling to find out exactly how. Any suggestions? I can feel my lack of technical knowledge and experience holding me back.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/CerealMan027"> /u/CerealMan027 </a> <br /> <span><a href="https://www.reddit.com/r/gamedev/comments/1upjqet/best_way_to_handle_large_amounts_of_pops_with/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/gamedev/comments/1upjqet/best_way_to_handle_large_amounts_of_pops_with/">[comments]</a></span>