The First World War Mod: Python Tools
The main features of my World War 1 mod for Hearts of Iron will centre on the map overhaul: a map focusing on Europe, detailed enough to feature the mile-by-mile battles of the more stagnant fronts. In my last post, I explained the concept and introduced my map. Over the recent weeks (months, maybe. I’m … not sure anymore) I’ve been populating this base map with terrain and provinces.
Hearts of Iron features provinces, or tiles, on which a unit can stand. Fighting takes places when one friendly unit tries to push an enemy unit out of a neighboring tile. The shape and adjacencies of provinces are all determined by the province map, in which each provinces is drawn in a unique RGB color (the vanilla province map of the low countries can be seen above).
Provinces are also organised into larger areas, called states. States are what a country can actually annex, build factories in, etc. They’re the more political unit of geography in the game. So in vanilla HoI, there’s a Wales state in the UK, broken into 7 or 8 provinces.
It’s perfectly possible to draw these provinces out by hand, and at first, that’s exactly what I did. But this was proving time consuming: drawing the provinces is fast enough, but then you have to fill them in with a unique color, AND remove the border pixels. After filling out a fraction of the low countries took a few days of work, I decided that suich a seemingly mundane task would need some automation.
Map-Making Tools
My first attempt to speed up province workflow was very simple. I knew that Photoshop had custom action-management to chain together several operations. You could, for example, magic-wand select an area, expand it by a few pixels, and fill it in to quickly blot out (most of) the border pixels. This had pixed results, however.
Then I tried automating the color randomisation process, saving me the irritating task of re-sliding RGB values every time I select a new province. This had to be done with actual photoshop script, which was easy enough to learn. I wish I had some gifs to demonstrate, but this all happened so long ago that I’ve since reset my photoshop and lost the scripts.
But in the end, these techniques weren’t enough (for me). Province-filling was boring, arduous, and getting in the way of more interesting tasks. If I’d stuck with it, maybe I would have had all of France and Germany populated by now. But in any case, I really enjoy tool programming and wanted to learn how to handle images in scripts. It turns out, Python has a library that does this especially well called skimage.
My plan was to create a script that did the following:
Read an input image defining border pixels.
Filled in the interior of those borders with a unique color.
Removed the border pixels with relatively acceptable guesswork: for example, coloring each remaining border pixel the most common color of its neighbors, until they’re all gone.
After some time, I got a script that did just that. And I do have gifs this time.
Filling Provinces
This is an early iteration of the Benelux region, plus northern France, being filled in automatically by my fillprovinces python script. You’ll notice that the provinces are done in blocks: these are the states I mentioned earlier. It’s not super visible (this is from an early version of my toolset that required all the borders to be shades magenta, for some reason …), but states are defined by a unique border color, so the tool is able to recognise what a state is, and color its provinces appropriately. The benefit of this is that when you’re doing map editing later, it’s really easy to see where a state is.
I won’t go into too much detail here, but for the curious, here’s how the algorithm works:
Identify a state. Focus the operation on the bounds of that state (saving us constantly searching through a huge array of pixels).
Increment the canvas we’re working on by 1 pixel’s worth of padding around each edge, so that we can:
Run a flood operation (basically a paint-bucket tool in photoshop) from the corner of the canvas. The entire outside area of the state is now selected, so now we know which pixels are inside a state and which ones aren’t. Note that this doesn’t support any torus-shaped states.
Identify each unique white area in the state, again, using the flood operation. Each of these areas is a province.
Fill the provinces in with unique colors.
Go through all the border pixels, find their most common neighbor, and color them that color, until there are no border pixels left. This basically ‘grows’ a province’s interior out over the edge borders. There are some other housekeeping things I do for neater results, like prioritising pixels which were not originally border-colored.
And that’s it. Repeat for all states and output the result. Also watch out for things like provinces that seem to be too small, or border fragments that are detached from the state. Warn the user about these things and mark them on the output map.
This dramatically sped up province creation time. As of now, my main workload is just drawing the provinces which, once I’ve planned out the states and terrain (more on that later) is quite fast. In the past few days, for example, I’ve filled out all of France and Germany, which is about 20% of the map.
So this was my first experience with Python, and I’m really impressed with the language for its accessibility and flexibility. Numpy (the plugin used to handle image arrays) can be a huge pain sometimes, but is fundamentally very elegant. I reccomend using numpy/skimage (numpy is a prerequisite of skimage) for anyone looking to build their own tools like this. The IDLE editor makes things very easy!
I wasn’t done, however. Marking issues on the output map gave me an idea.
Validation
Hearts of Iron IV can be picky with province maps. It cannot, for example, accept cases where four provinces are touching (so-called ‘x crossings’), because the adjacency of these provinces can’t be resolved - do all four provinces touch each other? It also gets fussy with provinces that are too large, or too small. This would all be understandable, if finding these mistakes was made easy. But in fact, all HoI really does is print the pixel coordinates of the discovered issues for you.
After maybe a week of modding, I got really, really sick of having to locate coordinates on my 10k x 10k photoshop image, sometimes dozens at a time, for every new stretch of land I added and every tiny edit I made. Checking for x-crossings should be easy enough, as should be finding provinces that are too big or small. What I wanted was a tool that could clearly show me where these issues were, making them so much easier to fix.
So that’s what I wrote next. validatemap
Assigning Provinces to States
That would have been it. But then a user on reddit mentioned to me that they were having trouble assigning provinces to states in HoI’s Nudge tool (this is an in-game map editor for simple database operations, but it’s quite buggy). This got me thinking. Even when Nudge works, you assign provinces to states by shift-clicking each individually, and creating the state file. This can take a really long time, and I already had thousands of provinces just for the Western Front. The system I’d build HAD all the information it needed to assign provinces to states automatically (remember, you define states with a unique border color when using fillprovinces.py). I was eager to get back to the actual mod, but was also enjoying learning a new skill, and wanted to save myself some headaches in the future, so I created a script which can:
Determine which provinces belong to a given state
Either print out sets of provinces (by their ID) for each state, ready for you to paste into your HoI files, or …
Write to the HoI files themselves.
This, too, has really sped up my workflow, to the degree that I can carve up huge areas of my map within minutes.
This entire endeavor has been very taxing. I like scripting (especially tools scripting), but it isn’t my favourite type of development work (that would be anything game-design related), and I’ve been eager to get back to actually working on the mod itself. However, I’m really not sure I’d be able to pull such an enormous mod off without tools like this.
With the toolset (which I like to call ‘Provincial’) ready to use, though, I was finally ready to start really drawing up the map.
