The cross platform code that isn't.
A couple of weeks ago, another piece of wxWidgets infrastructure in the Alter Aeon client was replaced with custom code. This time, it was the bulk of the keyboard handling code that fell in battle.
In theory, the wxWidgets keyPressEvent is where you should trap out various key combinations; things like 'a', the escape key, and 'control-c' pairs. Certainly under linux, nearly all of these worked correctly, and the ones that didn't were obscure and not really worth hunting down. When I first ported to Win32, all that changed.
In Win32 builds, a lot of keys were not handled by the key press event. You'd press the key combination, and nothing would happen. It was like the operating system was trapping out the key combination and never bothering to pass it up the stack. As it turns out, that's pretty much exactly what was happening.
I did eventually get the Win32 builds to work by intercepting a handful of keys at the keyDownEvent and keyUpEvent layers. When I finally found these, it was hugely helpful; in the end, I trapped on the order of 30 key combinations to do away with things like annoying system beeps and idiotic behavior of the default keystrokes, in addition to handling our own special control combinations.
Enter Mac OSX. Enter a whole new raft of keyboard weirdness. I hacked on this for about an hour, and quickly realized that I was on the path to madness. The code was becoming a rat's nest of ifdefs with key combinations trapped out in multiple different layers in different operating systems.
About the only thing that could be called consistent between any of the operating systems was that the keyDownEvent and keyUpEvent were always reliable. The solution then seemed obvious: move everything down into the keyDownEvent, and ignore everything else. Intercept the keyboard control -before- wxWidgets could do anything stupid with it.
Keyboard control now works consistently and reliably on all three platforms.
Showing posts with label software engineering. Show all posts
Showing posts with label software engineering. Show all posts
Monday, September 26, 2011
Saturday, July 9, 2011
wxWidgets GTK font rendering
In my previous post, I managed to fix the bugs in wxDC::DrawPoint() by using my own custom drawing class. Unfortunately, a lot of the places I where I used DrawPoint were also places where I needed to draw text, and in order to draw text I needed a wxDC. No problem, I'll just draw the text using a temporary wxMemoryDC on the bitmap, and to make sure there's no clobbering problems I'll make sure that I don't intermingle direct drawing scope with the DC object scope.
This appeared to work, except for the scrolling/updating windows. For some reason, even though there was a Clear() call before the font rendering, the windows would never clear. If I destroyed and recreated the bitmap every time, it worked fine but was very slow. If I cleared after drawing the text, the clear worked. If I cleared before drawing the text, the clear did not work.
Eventually I figured out the reason:
Under wxGTK, wxDC drawing and rendering are done to a separate bitmap/bitmap layer that is permanently attached to the real bitmap. When font data is drawn using wxDC::DrawText(), the text is drawn to the hidden layer, then the hidden layer is blitted in its entirety on top of the real bitmap. The end result is that anything you do prior to constructing the DC and rendering text with that DC is overwritten/deleted when the DC destructs.
Just how the hell does this make any sense? Nevermind, forget I asked. On the plus side, this explains why full screen font rendering is so slow in wxGTK. I never could figure that part of it out.
For roughly a day, I was utterly at a loss when faced with this problem. Switch back to the old code and lose the MacOS X platform; stay with the new code and lose wxGTK (and possibly other platforms, I only ever tested it on GTK.) I couldn't even think of a way to work around it.
Eventually, a possible solution presented itself: add font rendering to my custom bitmap class. I wasn't real clear on how to do this at first, but the key breakthrough came when I realized I could just use a cheap glyph rendering scheme. Use wxDC to draw each of the letters I needed (ascii chars 32 through 126) into tiny, clean bitmaps, then use a modified alpha channel copy routine to draw them to my custom bitmaps.
I'm proud to say that I now have proper text rendering and graphical drawing primitives on all three platforms. And to think that all I needed to do was rewrite the entire wxDC rendering system to do it!
I should have just done it in the first place, as I recommended to myself in a past life. Sometimes our older selves are no wiser than we are in youth.
This appeared to work, except for the scrolling/updating windows. For some reason, even though there was a Clear() call before the font rendering, the windows would never clear. If I destroyed and recreated the bitmap every time, it worked fine but was very slow. If I cleared after drawing the text, the clear worked. If I cleared before drawing the text, the clear did not work.
Eventually I figured out the reason:
Under wxGTK, wxDC drawing and rendering are done to a separate bitmap/bitmap layer that is permanently attached to the real bitmap. When font data is drawn using wxDC::DrawText(), the text is drawn to the hidden layer, then the hidden layer is blitted in its entirety on top of the real bitmap. The end result is that anything you do prior to constructing the DC and rendering text with that DC is overwritten/deleted when the DC destructs.
Just how the hell does this make any sense? Nevermind, forget I asked. On the plus side, this explains why full screen font rendering is so slow in wxGTK. I never could figure that part of it out.
For roughly a day, I was utterly at a loss when faced with this problem. Switch back to the old code and lose the MacOS X platform; stay with the new code and lose wxGTK (and possibly other platforms, I only ever tested it on GTK.) I couldn't even think of a way to work around it.
Eventually, a possible solution presented itself: add font rendering to my custom bitmap class. I wasn't real clear on how to do this at first, but the key breakthrough came when I realized I could just use a cheap glyph rendering scheme. Use wxDC to draw each of the letters I needed (ascii chars 32 through 126) into tiny, clean bitmaps, then use a modified alpha channel copy routine to draw them to my custom bitmaps.
I'm proud to say that I now have proper text rendering and graphical drawing primitives on all three platforms. And to think that all I needed to do was rewrite the entire wxDC rendering system to do it!
I should have just done it in the first place, as I recommended to myself in a past life. Sometimes our older selves are no wiser than we are in youth.
wxDC DrawPoint broken under wxMAC
To follow up on my previous post, I had written my own routines for handling alpha channel support under wxWidgets, and finally everything seemed to be working properly. Both Win32 and Linux/GTK builds seemed to work exactly as expected using my alpha channel bitmaps where I needed them. For most other places, I still used wxDC primitives, as wxDC direct drawing seemed to be a bit faster than going through a bitmap/blit interface, and I still needed to use wxDC to render fonts.
I figured porting to MacOS X would be a snap, and for the most part it wasn't too bad. There were some issues with Repaint() not working the same, but adding a few explicit paint calls in the right place fixed that up.
However, something was wrong with some of the graphics in the various windows. The sky bar, normally with lots of little stars in it, still had stars in it, but instead of one pixel there were two: one at offset (0, 0), and one at offset (1, 1). I also noticed 'fuzz' on the corners of various windows and buttons, and in a handful of other locations.
Long story short, wxDC::DrawPoint() draws two pixels under wxMAC. Not one, but two. I found a couple bug reports regarding this from a few years ago, where it was basically acknowledged as a "verified bug", and the ticket was never closed. Advice was "these routines are slow and shouldn't be used for high performance drawing". Thanks guys.
I have some mighty fine choice words to say to the developers regarding this, but I'll try to keep this post on topic.
If something as fundamental as DrawPoint() is malfunctioning on a major development platform, I figured it would probably be a good idea to transition away from the classes that implement it in favor of my own drawing routines. So, I started using my bitmap class, which just happens to have a DrawPixel() function that works properly.
As expected, this fixed the drawing problems on all three platforms. But alas, using my bitmap class exposed another set of very nasty wx bugs, which I'll discuss in the next post.
I figured porting to MacOS X would be a snap, and for the most part it wasn't too bad. There were some issues with Repaint() not working the same, but adding a few explicit paint calls in the right place fixed that up.
However, something was wrong with some of the graphics in the various windows. The sky bar, normally with lots of little stars in it, still had stars in it, but instead of one pixel there were two: one at offset (0, 0), and one at offset (1, 1). I also noticed 'fuzz' on the corners of various windows and buttons, and in a handful of other locations.
Long story short, wxDC::DrawPoint() draws two pixels under wxMAC. Not one, but two. I found a couple bug reports regarding this from a few years ago, where it was basically acknowledged as a "verified bug", and the ticket was never closed. Advice was "these routines are slow and shouldn't be used for high performance drawing". Thanks guys.
I have some mighty fine choice words to say to the developers regarding this, but I'll try to keep this post on topic.
If something as fundamental as DrawPoint() is malfunctioning on a major development platform, I figured it would probably be a good idea to transition away from the classes that implement it in favor of my own drawing routines. So, I started using my bitmap class, which just happens to have a DrawPixel() function that works properly.
As expected, this fixed the drawing problems on all three platforms. But alas, using my bitmap class exposed another set of very nasty wx bugs, which I'll discuss in the next post.
wxwidgets has no alpha channel
A few months after I ported the Alter Aeon client to the wxWidgets toolkit, I decided I wanted to add alpha overlay support to the automap and main window.
Encouragingly enough, this appeared to be trivially supported: wxColor included an alpha value, bitmaps could be defined as having an alpha channel, etc. Pretty much everything looked good to go, so I started experimenting with it.
Alas, it was not to be. It turns out that there are precisely three places where the alpha channel is used:
1) loading a PNG from PNG data that has an alpha channel,
2) drawing a bitmap containing alpha channel data using wxDC::DrawBitmap, and
3) wxDC::DrawText, which politely draws text on top of the underlying pixels.
For all other locations, there simply should have been this assert:
wxASSERT("Alpha channel is stubbed here and works on precisely zero platforms ever, ha ha, the joke is on you")
After upgrading to wx 2.9 and a pile of scouting around, I came across wx/rawbmp.h, which has some direct access functions that allow access to the alpha channel. With this, I managed to get alpha channel working - but there were other problems with 2.9, so I downgraded back to stable, 2.8.12.
The alpha channel code that worked under 2.9 works properly under wx 2.8.12 in wxMAC and wxGTK, but apparently isn't implemented under Win32, which happens to be my main development platform.
Rather than bash my skull repeatedly against the wall, I did what any sane developer would do: throw out the broken toolkit module, and write one that works. In the end, I ended up with a wxBitmap derived class that has drawing functions that actually work properly and support alpha channel drawing when I need them to.
However, there's a problem with this as well, which I'll detail in a later post.
Encouragingly enough, this appeared to be trivially supported: wxColor included an alpha value, bitmaps could be defined as having an alpha channel, etc. Pretty much everything looked good to go, so I started experimenting with it.
Alas, it was not to be. It turns out that there are precisely three places where the alpha channel is used:
1) loading a PNG from PNG data that has an alpha channel,
2) drawing a bitmap containing alpha channel data using wxDC::DrawBitmap, and
3) wxDC::DrawText, which politely draws text on top of the underlying pixels.
For all other locations, there simply should have been this assert:
wxASSERT("Alpha channel is stubbed here and works on precisely zero platforms ever, ha ha, the joke is on you")
After upgrading to wx 2.9 and a pile of scouting around, I came across wx/rawbmp.h, which has some direct access functions that allow access to the alpha channel. With this, I managed to get alpha channel working - but there were other problems with 2.9, so I downgraded back to stable, 2.8.12.
The alpha channel code that worked under 2.9 works properly under wx 2.8.12 in wxMAC and wxGTK, but apparently isn't implemented under Win32, which happens to be my main development platform.
Rather than bash my skull repeatedly against the wall, I did what any sane developer would do: throw out the broken toolkit module, and write one that works. In the end, I ended up with a wxBitmap derived class that has drawing functions that actually work properly and support alpha channel drawing when I need them to.
However, there's a problem with this as well, which I'll detail in a later post.
Friday, February 25, 2011
If you do it more than once, write a script...
There's a saying often used by Unix System Administrators: if you're going to do it more than once, write code to do it for you. For the last few days, I've been writing code to handle clan equipment automatically, so I don't have to be involved.
This has been a very time-consuming process, so much so that you might be tempted to ask, "if it's going to take so long, why not just do it by hand?" And indeed, that's tempting.
But one of the things that has occurred to me writing this code is that clan equipment is simply a complex process. Every single line of code I've written reflects some check, some detail that I would have had to remember and check manually. Each of these details is important. And the odds of me remembering them all when I do it by hand are near zero.
In short, while doing it by hand would definitely be faster in the short term, it's basically guaranteed to be wrong. And I've got direct proof of this while I'm looking at the existing clan items - they're spread out across the vnum ranges completely at random, built and set up in different ways by different builders, with some of them violating various constraints and some not. It's a complete mess.
That is what 15 years of doing it manually looks like, and that's why I'm not going to allow it to be done manually anymore. I am replacing myself with a (not so short) set of scripts. I'm ok with that, because those scripts will do a better job than I would have.
This has been a very time-consuming process, so much so that you might be tempted to ask, "if it's going to take so long, why not just do it by hand?" And indeed, that's tempting.
But one of the things that has occurred to me writing this code is that clan equipment is simply a complex process. Every single line of code I've written reflects some check, some detail that I would have had to remember and check manually. Each of these details is important. And the odds of me remembering them all when I do it by hand are near zero.
In short, while doing it by hand would definitely be faster in the short term, it's basically guaranteed to be wrong. And I've got direct proof of this while I'm looking at the existing clan items - they're spread out across the vnum ranges completely at random, built and set up in different ways by different builders, with some of them violating various constraints and some not. It's a complete mess.
That is what 15 years of doing it manually looks like, and that's why I'm not going to allow it to be done manually anymore. I am replacing myself with a (not so short) set of scripts. I'm ok with that, because those scripts will do a better job than I would have.
Saturday, December 25, 2010
Christmas Expansion - Update
We finally got everything done and installed, and it went off pretty easy. The biggest hitches were minor display bugs in explorer points, with a few minor typos and what not elsewhere.
One of the things that surprised me was the time breakdown for the various spells and skills. We only added 12 of them this time; here's a rough time estimate for each.
Level 25 mage skill 'melt' - approximately 6 hours. This skill was in multiple pieces, as it can operate on multiple targets. Not much debugging time required.
Level 27 mage skill 'spell lore' - approximately 1 hour. This was a very easy hook into the spellcasting code.
Level 6 cleric spell 'faith shield' - approximately 4 hours, most of it testing and design, trying to figure out how to not break anything.
Level 31 cleric spell 'breath of life' - approximately 5 hours. Required several debug passes to get the math right, largely because I wrote the core of it while I was very tired.
Level 33 cleric spell 'area heal' - approximately 2 hours, because this reused a lot of well-done code from 'area refresh'.
Level 34 cleric spell 'astral bridge' - approximately 16 hours. The spell itself only took about two hours, but before I could implement it I had to completely rework the waypoint code. Refactoring and updating the old code took the bulk of the time, and it broke waypoints for a lot of players.
Level 32 cleric spell 'group waypoint' - approximately 2 hours, because the astral bridge and other waypoint code was already in place.
Level 8 thief skill 'listen' - approximately 4 hours.
Level 29 thief skill 'fast talking' - approximately 3 hours, had to come back and extend its use into other thief skills that didn't have the facility before.
Level 26 warrior skill 'cleave' - approximately 5 hours. Ran into an irritating bug where cleave would cause a death which would cause an autocleave which could cause a death which could cause an autocleave...
Level 32 warrior skill 'whirlwind' - approximately 3 hours. The event handler for this was very straightforward.
Level 8 necromancer spell 'bone blade' - approximately 2 hours.
Explorer points - about 10 hours. This had a lot of hooks for display and interfacing, which usually takes a lot of time. Multiple debug passes.
Various achievements - about ten minutes each when done in groups.
There was also a lot of infrastructure and other work that went on to enable all this; between bug fixes, infrastructure, and design, the times above are perhaps a third of the total.
One of the things that surprised me was the time breakdown for the various spells and skills. We only added 12 of them this time; here's a rough time estimate for each.
Level 25 mage skill 'melt' - approximately 6 hours. This skill was in multiple pieces, as it can operate on multiple targets. Not much debugging time required.
Level 27 mage skill 'spell lore' - approximately 1 hour. This was a very easy hook into the spellcasting code.
Level 6 cleric spell 'faith shield' - approximately 4 hours, most of it testing and design, trying to figure out how to not break anything.
Level 31 cleric spell 'breath of life' - approximately 5 hours. Required several debug passes to get the math right, largely because I wrote the core of it while I was very tired.
Level 33 cleric spell 'area heal' - approximately 2 hours, because this reused a lot of well-done code from 'area refresh'.
Level 34 cleric spell 'astral bridge' - approximately 16 hours. The spell itself only took about two hours, but before I could implement it I had to completely rework the waypoint code. Refactoring and updating the old code took the bulk of the time, and it broke waypoints for a lot of players.
Level 32 cleric spell 'group waypoint' - approximately 2 hours, because the astral bridge and other waypoint code was already in place.
Level 8 thief skill 'listen' - approximately 4 hours.
Level 29 thief skill 'fast talking' - approximately 3 hours, had to come back and extend its use into other thief skills that didn't have the facility before.
Level 26 warrior skill 'cleave' - approximately 5 hours. Ran into an irritating bug where cleave would cause a death which would cause an autocleave which could cause a death which could cause an autocleave...
Level 32 warrior skill 'whirlwind' - approximately 3 hours. The event handler for this was very straightforward.
Level 8 necromancer spell 'bone blade' - approximately 2 hours.
Explorer points - about 10 hours. This had a lot of hooks for display and interfacing, which usually takes a lot of time. Multiple debug passes.
Various achievements - about ten minutes each when done in groups.
There was also a lot of infrastructure and other work that went on to enable all this; between bug fixes, infrastructure, and design, the times above are perhaps a third of the total.
Tuesday, April 13, 2010
Dprocs
We have a rather interesting interpreted language in-game that can be used to perform actions on a handful of triggers; an example might be to add an arbitrary 'use' function to an object. As an example of this, I once built a vacuum cleaner object which could be used to suck up pretty much anything you pointed it at. Sucked up things ended up inside the vacuum.
One drawback of this language is that it didn't have any concept of global data. I added that feature today, using a surprisingly small amount of code.
A surprisingly small amount of code that took me about three hours to write.
Once again, software complexity rears its ugly head. I actually made two false starts on this problem, each of which I had to back out before finally getting a minimal solution to it. The interpreter isn't really that complicated or large by any real programming standard, but it's still larger and more complicated than I could easily wrap my brain around.
It took about two hours to really figure out the best way to handle it, then about an hour to implement it. Most of that time was just trying to understand what was already there and how it all worked. And this is code I wrote!
On the plus side, my actual implementation, when I finished, had 2 bugs in it, both trivial. It's good to know that thorough understanding up front does in fact result in far fewer mistakes.
One drawback of this language is that it didn't have any concept of global data. I added that feature today, using a surprisingly small amount of code.
A surprisingly small amount of code that took me about three hours to write.
Once again, software complexity rears its ugly head. I actually made two false starts on this problem, each of which I had to back out before finally getting a minimal solution to it. The interpreter isn't really that complicated or large by any real programming standard, but it's still larger and more complicated than I could easily wrap my brain around.
It took about two hours to really figure out the best way to handle it, then about an hour to implement it. Most of that time was just trying to understand what was already there and how it all worked. And this is code I wrote!
On the plus side, my actual implementation, when I finished, had 2 bugs in it, both trivial. It's good to know that thorough understanding up front does in fact result in far fewer mistakes.
Saturday, November 28, 2009
Global mapping - a story of invention
When I first started this mapping project, my goals were grandiose. I had a vision of the dclient showing a very high level map with the location of cities and perhaps roads between them, updating a small amount in real time as you walked from place to place.
This large vision seemed so close, and yet so far - while easy in principle, the biggest issue was in generating the maps. Mud areas in general do not have to respect linearity. You can arbitrarily connect any room to any other room; in short, generating a cartesian, mappable layout becomes a builder constraint, not a tool constraint.
When faced with 20k rooms of previously built nonlinear content, the task seemed ridiculous and daunting. Initially, I tried piecemeal mapping: each area would have an internal, generated layout, and we'd try to move them around like puzzle pieces to get everything to fit. This didn't really work that well, but some new mapping tools were built and I was able to collect a lot more data.
The next idea was to try a form of annealing, where rooms would update their x/y/z coordinates based on what rooms they were connected to. This worked slightly better, in that it tended to smooth out big gaps and it forced areas to fit in places they didn't before. But it had the drawback of taking a very long time to settle, and it still didn't deal with nonlinearities.
Around this time, the idea of the big/small room distinction came about. Cities on maps are generally very small compared to the surrounding landscape; in Alter Aeon, this was definitely not true. The main city of Ralnoth was easily half as long as the Great Southern Road. I decided to declare cities, towns, buildings and other terrain as 'much smaller' than outdoor/wilderness/linkage areas.
This big/small transition appeared to work very well on the few initial areas I tried it on. Older areas required the most rework, but a lot of 'small' areas had limited linkage to their surroundings anyway, and the transition was easy. The southeast portion of the continent was hit the hardest, as it was designed on a unified grid. It had to be extensively retconned, but even that work happened quickly thanks to the dedication of Draak and Shadowfax.
As part of the big/small experiments, I extended the previously failed area map code to cross all zones, and used that to generate some cheesy room maps showing absolute and relative positions of everything. After a bit more work, I was able to show nonlinear linkages on the maps as well - and this is when things really started to get done. Seeing the problem spots, it was surprising how few of them there really were. We began to aggressively target those.
Meanwhile, the annealing/smoothing code was running in the background, continuously updating the 'real' positions of rooms, as opposed to where we thought they should be. The most recent advance was the realization that the 'perfect' and 'real' maps should be as close to each other as possible, and in fact the 'real' map should try to match the 'perfect' map. I made a few very minor changes to the algorithm, and the global map consistency (not to mention convergence speed) has increased greatly.
After all of this, I'm almost to the point where I can generate the high-level maps I originally wanted in the first place.
The lesson to take away is that invention happens in bursts and is often triggered by random things. Ideas are easy, but implementing them is hard; further, you never know which experiments are going to fail or succeed until you spend time on them. There were several dead ends along the way in this project, but each provided tooling or insight into new things to try.
Overall, if I had known the final form of things at the beginning, all of the code work could probably have been done in a couple of weeks, and all of the area work to follow in another month (with builder help of course). As it stands now, the mapping saga has been going on since the beginning of the year, with breakthroughs and advances from time to time.
This large vision seemed so close, and yet so far - while easy in principle, the biggest issue was in generating the maps. Mud areas in general do not have to respect linearity. You can arbitrarily connect any room to any other room; in short, generating a cartesian, mappable layout becomes a builder constraint, not a tool constraint.
When faced with 20k rooms of previously built nonlinear content, the task seemed ridiculous and daunting. Initially, I tried piecemeal mapping: each area would have an internal, generated layout, and we'd try to move them around like puzzle pieces to get everything to fit. This didn't really work that well, but some new mapping tools were built and I was able to collect a lot more data.
The next idea was to try a form of annealing, where rooms would update their x/y/z coordinates based on what rooms they were connected to. This worked slightly better, in that it tended to smooth out big gaps and it forced areas to fit in places they didn't before. But it had the drawback of taking a very long time to settle, and it still didn't deal with nonlinearities.
Around this time, the idea of the big/small room distinction came about. Cities on maps are generally very small compared to the surrounding landscape; in Alter Aeon, this was definitely not true. The main city of Ralnoth was easily half as long as the Great Southern Road. I decided to declare cities, towns, buildings and other terrain as 'much smaller' than outdoor/wilderness/linkage areas.
This big/small transition appeared to work very well on the few initial areas I tried it on. Older areas required the most rework, but a lot of 'small' areas had limited linkage to their surroundings anyway, and the transition was easy. The southeast portion of the continent was hit the hardest, as it was designed on a unified grid. It had to be extensively retconned, but even that work happened quickly thanks to the dedication of Draak and Shadowfax.
As part of the big/small experiments, I extended the previously failed area map code to cross all zones, and used that to generate some cheesy room maps showing absolute and relative positions of everything. After a bit more work, I was able to show nonlinear linkages on the maps as well - and this is when things really started to get done. Seeing the problem spots, it was surprising how few of them there really were. We began to aggressively target those.
Meanwhile, the annealing/smoothing code was running in the background, continuously updating the 'real' positions of rooms, as opposed to where we thought they should be. The most recent advance was the realization that the 'perfect' and 'real' maps should be as close to each other as possible, and in fact the 'real' map should try to match the 'perfect' map. I made a few very minor changes to the algorithm, and the global map consistency (not to mention convergence speed) has increased greatly.
After all of this, I'm almost to the point where I can generate the high-level maps I originally wanted in the first place.
The lesson to take away is that invention happens in bursts and is often triggered by random things. Ideas are easy, but implementing them is hard; further, you never know which experiments are going to fail or succeed until you spend time on them. There were several dead ends along the way in this project, but each provided tooling or insight into new things to try.
Overall, if I had known the final form of things at the beginning, all of the code work could probably have been done in a couple of weeks, and all of the area work to follow in another month (with builder help of course). As it stands now, the mapping saga has been going on since the beginning of the year, with breakthroughs and advances from time to time.
Sunday, November 22, 2009
I'm a deadbeat
I was looking at the list of big-ticket/high priority items on my whiteboard, and it occurred to me that a lot of them are half done.
Not the good kind of half done, where it's a huge project and a lot of pieces are in and functional.
The bad kind of done where infrastructure is in place and working, but it's not doing anything valid in the game context.
A good example of this is the recent boat code. This largely works, and probably only needs a couple of hours to get it in god-level beta test. Instead, it's sitting there in the code base, largely complete but unconnected.
How about clan wars? I spent time adding a lot of infrastructure for this, then never hooked it up. It also probably doesn't need more than a few hours to get it in beta test.
Classes are a bigger picture. I have more structure reworks to do, but I have been avoiding them. This one is a bit more understandable, because the project is so large - but this isn't exactly a huge piece of the puzzle.
Part of the reason for this is because I'm constantly 'firefighting' instead of doing new development. I'm always fixing minor bugs, heading off social/political issues, trying to manage things and people. I'm starting to think it would perhaps be better for all involved if I simply stayed invisible most of the time. I used to do that a lot, for exactly this reason, but for some reason fell out of the habit.
Another part of the reason is that the detail work isn't any fun. It's painstaking and precise, and there's always negative feedback from players to look forward to. Nevertheless, leaving a heap of half-finished projects on the queue isn't helping anything either.
No pain no gain, perhaps.
Not the good kind of half done, where it's a huge project and a lot of pieces are in and functional.
The bad kind of done where infrastructure is in place and working, but it's not doing anything valid in the game context.
A good example of this is the recent boat code. This largely works, and probably only needs a couple of hours to get it in god-level beta test. Instead, it's sitting there in the code base, largely complete but unconnected.
How about clan wars? I spent time adding a lot of infrastructure for this, then never hooked it up. It also probably doesn't need more than a few hours to get it in beta test.
Classes are a bigger picture. I have more structure reworks to do, but I have been avoiding them. This one is a bit more understandable, because the project is so large - but this isn't exactly a huge piece of the puzzle.
Part of the reason for this is because I'm constantly 'firefighting' instead of doing new development. I'm always fixing minor bugs, heading off social/political issues, trying to manage things and people. I'm starting to think it would perhaps be better for all involved if I simply stayed invisible most of the time. I used to do that a lot, for exactly this reason, but for some reason fell out of the habit.
Another part of the reason is that the detail work isn't any fun. It's painstaking and precise, and there's always negative feedback from players to look forward to. Nevertheless, leaving a heap of half-finished projects on the queue isn't helping anything either.
No pain no gain, perhaps.
Tuesday, November 10, 2009
Software Reuse
There is an old saying in the nootropic (mental-performance enhancing drug) community:
"The guys in the 60's had this truly brilliant idea of using drugs to expand conciousness and improve thinking. Unfortunately, all they had at the time was LSD." [Just to ensure proper context, this is said without sarcasm. It really was a good idea, however LSD turned out to be a complete flop in achieving this goal.]
I've recently been doing some work that involved C++ templates, so allow me to reword this for modern usage:
"The c++ guys in the 90's had this truly brilliant idea of direct language support for generic code and module reuse. Unfortunately, all they had at the time was c++ templates."
"The guys in the 60's had this truly brilliant idea of using drugs to expand conciousness and improve thinking. Unfortunately, all they had at the time was LSD." [Just to ensure proper context, this is said without sarcasm. It really was a good idea, however LSD turned out to be a complete flop in achieving this goal.]
I've recently been doing some work that involved C++ templates, so allow me to reword this for modern usage:
"The c++ guys in the 90's had this truly brilliant idea of direct language support for generic code and module reuse. Unfortunately, all they had at the time was c++ templates."
Tuesday, October 20, 2009
Software Complexity Crisis
[This rather large post is almost entirely a personal rant. I complain a lot about wxWidgets in here, because it's fresh in my mind and a very good example of a specific type of software engineering crisis. Note that I'm still using the WX libraries for the Alter Aeon client project; clearly it has value to me in spite of its faults, and I appreciate the effort the WX team has put it. That said, if I could easily move to any another library that met my constraints, I would do it in a heartbeat.]
Various recent attempts on my part to use large software libraries have made me re-examine the issue of the general software-engineering crisis. I've run up against typical software crisis problems many times in the past, so I tend to keep my eyes open when I see material related to the topic. This is one of the reasons that Vernor Vinge's book "A deepness in the sky" caught my attention.
One of the basic premises of this book is that complexity failure can be sufficient to bring down entire societies. This was put very succinctly in a blog posting by Jeremy Bowers on his iRi Blog, part of which I quote here:
"One of the less well known concepts which informs his sci-fi writings is one possible fate of societies that do not or can not end in a "singularity", which is the eventual unavoidable collapse of the society in a cascading failure state brought on by excessive, uncontrollable complexity in the ever-more-sophisticated systems that drive the society. In this case, take "system" in the broad sense, including not just software, but business practices, government, and societal mores. A failure occurs somewhere, which brings down something else, which brings down two other something elses, and perhaps quite literally in the blink of an eye, you are faced with a growing complex of problems beyond the ability of any one human to understand or contain."
This relates to software in that I'm beginning to see more and more examples of how this can occur. Two software platforms in particular come to mind - IBM's WebSphere, which I was peripherally involved with a decade ago, and wxWidgets, which I am involved with today.
Both of these platforms are very complex. Both form an abstraction layer which builds on top of other layers - in the case of wxWidgets, there is a huge amount of API reuse from the lower layers of Windows, or GTK, or X, depending on which configuration it's built for. Each of these layers is built upon other layers, and other layers, sometimes with very deep call trees.
The most egregrious example of this kind of layering that I can recall was in WebSphere. A friend had asked me to take a look at a stack trace from a WebSphere crash; somewhere deep in Java land, a 'null object exception' had been thrown. (Thank god it wasn't a NULL pointer, that would have been much worse!) The exception handler that caught it was basically the main loop, because apparently no other layer could be bothered to check for failure conditions along the way.
There were over 160 stack frames to walk through. Not one of them was due to a recursive algorithm or function. I don't know about you, but that level of stack depth is quite frankly beyond my ability to manage or debug. I don't care what it does.
WxWidgets is clearly beginning to show stress of its own, of a different character: it's becoming more and more impossible to guarantee consistency across platforms. The WX guys have made tremendous progress in this regard, so that most of the core features work right, but there are simply too many details to keep track of and too many paths that will never be tested.
Here's a couple of examples of this, one of which I'm STILL fighting:
-------------------------------------------
When I first started switching the Alter Aeon client project over to WX, I initially used the wxTextCtrl class, which is built out of Microsoft system libraries in the Win32 world, or built on the GTK libraries in my development environment. I had hoped to use the class for both the input window, and for the main display; with a small amount of effort, I got the client running and working, but there were minor, very persistent issues.
The first of these was the input window. Various events, such as backspacing when empty, cause a system beep/bell under windows. They don't cause a bell under Linux. And further, there's no way to disable this. I don't know about you, but I'll be damned if I'm going to ship a product that beeps every time someone hits backspace.
I managed to take care of some of this by trapping out various keystrokes in the CHAR handler. It seemed like a poor hack at the time, but at least it helped. However it didn't help enough; a number of keystrokes simply don't generate CHAR events, yet they still fucking beep. I finally ended up writing a raw keyboard event handler, which tracks nearly all of the keyboard state, to trap out events that would generate a beep when passed to the lower layer. In the time it took me to disable beeping, I could have written and debugged a keyboard handler from scratch, with exactly the desired behaviour.
While beeping has largely been taken care of, other issues with this so-called standard class have not. The biggest one is that the color of text displayed in the class returns to black occasionally, and depending on the versions of the system DLLs for the particular Windows installation. My first attempts at fixing this were effective on all my development environments, but failed on about half of the release environments - text typed in the window would occasionally simply vanish.
By adding forced color setting in various places where it shouldn't be needed, I eventually managed to fix this problem for about 90% of my users. Out of sheer disgust at this point, I did some extremely vicious forced color setting in various event handlers, and this appears to have fixed 'most' of the problem. I still am receiving sporadic reports of it happening on current client builds, but at least the problem goes away now and seems to be triggered at random.
When attempting to use this same class for the main window display, I ran into what seemed to be minor issues regarding the scroll bars and scrolling of text in the window. No matter what I tried, I never did find a way to get reliable scroll positioning for this class across all platforms.
After fighting this off and on for several months, I became desperate. I finally wrote my own text display class from the ground up, using nothing more than bitmaps and font drawing routines. The total from-scratch implementation time was less than the time I had previously wasted trying to get the scrollbars to work properly. It's also faster, especially for very large data sets.
-------------------------------------------
This, my friends, is the software engineering crisis in action. Each layer, while hiding some of the problems of the lower layers, introduces its own; the overall result is a system with fewer catastrophic issues, but exponentially more minor issues.
Those minor issues are surely tolerable, are they not? To an extent, yes - but at what point do you die the death of a thousand cuts?
Catastrophic issues might be catastrophic and obvious failures, but that's one of the best things about them: they're catastrophic and obvious. They HAVE to be fixed. They must be understood, they must be cleaned up and dealt with. The minor problems on the other hand, can just keep accumulating. They just keep getting worse, they just keep getting more obscure, more complicated, more difficult to find and rectify. And worse, they compound each other.
In a good scenario over the long term, they become so prevalent that the system no longer becomes usable. In a bad scenario, the system becomes critical and unmaintainable. It's a swiss cheese of buggy modules and misunderstood patches.
Is that really what you want to build critical infrastructure out of? Is this where we're headed? I certainly hope not.
Various recent attempts on my part to use large software libraries have made me re-examine the issue of the general software-engineering crisis. I've run up against typical software crisis problems many times in the past, so I tend to keep my eyes open when I see material related to the topic. This is one of the reasons that Vernor Vinge's book "A deepness in the sky" caught my attention.
One of the basic premises of this book is that complexity failure can be sufficient to bring down entire societies. This was put very succinctly in a blog posting by Jeremy Bowers on his iRi Blog, part of which I quote here:
"One of the less well known concepts which informs his sci-fi writings is one possible fate of societies that do not or can not end in a "singularity", which is the eventual unavoidable collapse of the society in a cascading failure state brought on by excessive, uncontrollable complexity in the ever-more-sophisticated systems that drive the society. In this case, take "system" in the broad sense, including not just software, but business practices, government, and societal mores. A failure occurs somewhere, which brings down something else, which brings down two other something elses, and perhaps quite literally in the blink of an eye, you are faced with a growing complex of problems beyond the ability of any one human to understand or contain."
This relates to software in that I'm beginning to see more and more examples of how this can occur. Two software platforms in particular come to mind - IBM's WebSphere, which I was peripherally involved with a decade ago, and wxWidgets, which I am involved with today.
Both of these platforms are very complex. Both form an abstraction layer which builds on top of other layers - in the case of wxWidgets, there is a huge amount of API reuse from the lower layers of Windows, or GTK, or X, depending on which configuration it's built for. Each of these layers is built upon other layers, and other layers, sometimes with very deep call trees.
The most egregrious example of this kind of layering that I can recall was in WebSphere. A friend had asked me to take a look at a stack trace from a WebSphere crash; somewhere deep in Java land, a 'null object exception' had been thrown. (Thank god it wasn't a NULL pointer, that would have been much worse!) The exception handler that caught it was basically the main loop, because apparently no other layer could be bothered to check for failure conditions along the way.
There were over 160 stack frames to walk through. Not one of them was due to a recursive algorithm or function. I don't know about you, but that level of stack depth is quite frankly beyond my ability to manage or debug. I don't care what it does.
WxWidgets is clearly beginning to show stress of its own, of a different character: it's becoming more and more impossible to guarantee consistency across platforms. The WX guys have made tremendous progress in this regard, so that most of the core features work right, but there are simply too many details to keep track of and too many paths that will never be tested.
Here's a couple of examples of this, one of which I'm STILL fighting:
-------------------------------------------
When I first started switching the Alter Aeon client project over to WX, I initially used the wxTextCtrl class, which is built out of Microsoft system libraries in the Win32 world, or built on the GTK libraries in my development environment. I had hoped to use the class for both the input window, and for the main display; with a small amount of effort, I got the client running and working, but there were minor, very persistent issues.
The first of these was the input window. Various events, such as backspacing when empty, cause a system beep/bell under windows. They don't cause a bell under Linux. And further, there's no way to disable this. I don't know about you, but I'll be damned if I'm going to ship a product that beeps every time someone hits backspace.
I managed to take care of some of this by trapping out various keystrokes in the CHAR handler. It seemed like a poor hack at the time, but at least it helped. However it didn't help enough; a number of keystrokes simply don't generate CHAR events, yet they still fucking beep. I finally ended up writing a raw keyboard event handler, which tracks nearly all of the keyboard state, to trap out events that would generate a beep when passed to the lower layer. In the time it took me to disable beeping, I could have written and debugged a keyboard handler from scratch, with exactly the desired behaviour.
While beeping has largely been taken care of, other issues with this so-called standard class have not. The biggest one is that the color of text displayed in the class returns to black occasionally, and depending on the versions of the system DLLs for the particular Windows installation. My first attempts at fixing this were effective on all my development environments, but failed on about half of the release environments - text typed in the window would occasionally simply vanish.
By adding forced color setting in various places where it shouldn't be needed, I eventually managed to fix this problem for about 90% of my users. Out of sheer disgust at this point, I did some extremely vicious forced color setting in various event handlers, and this appears to have fixed 'most' of the problem. I still am receiving sporadic reports of it happening on current client builds, but at least the problem goes away now and seems to be triggered at random.
When attempting to use this same class for the main window display, I ran into what seemed to be minor issues regarding the scroll bars and scrolling of text in the window. No matter what I tried, I never did find a way to get reliable scroll positioning for this class across all platforms.
After fighting this off and on for several months, I became desperate. I finally wrote my own text display class from the ground up, using nothing more than bitmaps and font drawing routines. The total from-scratch implementation time was less than the time I had previously wasted trying to get the scrollbars to work properly. It's also faster, especially for very large data sets.
-------------------------------------------
This, my friends, is the software engineering crisis in action. Each layer, while hiding some of the problems of the lower layers, introduces its own; the overall result is a system with fewer catastrophic issues, but exponentially more minor issues.
Those minor issues are surely tolerable, are they not? To an extent, yes - but at what point do you die the death of a thousand cuts?
Catastrophic issues might be catastrophic and obvious failures, but that's one of the best things about them: they're catastrophic and obvious. They HAVE to be fixed. They must be understood, they must be cleaned up and dealt with. The minor problems on the other hand, can just keep accumulating. They just keep getting worse, they just keep getting more obscure, more complicated, more difficult to find and rectify. And worse, they compound each other.
In a good scenario over the long term, they become so prevalent that the system no longer becomes usable. In a bad scenario, the system becomes critical and unmaintainable. It's a swiss cheese of buggy modules and misunderstood patches.
Is that really what you want to build critical infrastructure out of? Is this where we're headed? I certainly hope not.
Sunday, March 1, 2009
Chaos seems to reign
The field of software arguably contains some of the largest and most complex functional systems ever created by man. Even relatively simple systems can demonstrate incredibly complex behavior, and can hide potentially major flaws through indefinitely long periods of use, only to demonstrate them at inconvenient times. The Alter Aeon server codebase is no exception.
Pretty much the most complicated discrete object in the server is the socket stack. It handles a huge number of protocols and a bunch of different filtering layers, with hooks in various places. The last major redesign of the socket stack was around 1998; it is a testament to the design of that module that it has required no major modification in that entire time period, though many minor modifications have been made to it.
Unfortunately, the server is not made of such lovely discrete objects. Often, obscure bugs lurk in the sea of wild code that makes up the substructure of the system. This weekend I had the incredible good luck to catch and haul two of these bugs to the surface.
For years, there have been a handful of issues that occurred seemingly at random. In the early, old days, some of these would even cause a server crash and reboot the game; but after being unable to find and fix them, the crashes were gradually protected against and ways were found to ignore these spurious events.
Examples of the two most common of these are deceptively simple: sometimes, the get_room function would be asked to find a room but be given no instructions on how to do so, and other times a character (a monster or player, it was difficult to tell) would die and simply 'get stuck'.
In reality, these are monstrously difficult. The get_room function lacking instruction was mindboggling - it happened perhaps twice a year, the reports were never the same, and there were thousands of places that could be calling the function. Even as our debug facilities improved, no progress was made - there just seemed to be no rhyme or reason to it.
The dead character bug was just as bad. The entire destruct and event handling process was revisited and inspected several times, but no holes were ever found. Until thursday.
---------------------------------------------------
In the course of trying to add some new restrictions to the 'charm' spell, I noticed that the destruct sequences for the charm, possession, and entangling roots spells looked a little goofy. We didn't check or clear certain important things, but there were comments indicating that we didn't need to because something else over there would take care of it. Keep in mind that this was entirely my code; I had written this well over a decade ago.
In the course of looking at this, I suddenly had a realization: there were no 'holes' in the logic for charm or possession, but maybe, just maybe, there was a hole in entangling roots. Entangling roots did come later, and quite frankly the code for it was a complete hackjob.
Within ten minutes, I had my answer. There was indeed a conflict between entangling roots and the 'special' code that made possession and charm work. The problem then became, how exactly do you fix such a mess? After about two hours of thinking about it and five more of carefully backing things out and reorganizing, I got what appears to be a stable fix. There's still some debug logging in it, but this bug appears to be properly killed. The new code is simpler, the checks are stronger, and we don't rely on obscure handlers to clean up messes. I hope.
---------------------------------------------------
I thought that was the end of my troubles in the short term, but then Glorida shows up with some obscure problem of his own. He's been working on mob programs, and had built something rather complex that simply was not working. Not only wasn't it working, but it was doing something weird, and it was doing it reliably.
This is another one of those 'fairly complex' pieces of the system. It took me about two hours off and on to get it loaded into my brain so I could really think about it. It then took me probably another hour to really understand what was going on, and figure out what was happening. And then it occurred to me:
This explains a lot of those debug log reports over the years!
The symptoms he uncovered showed up as a very unusual sort of 'doing things before other things have completed' recursive issue. One example of it is that a monster would 'say' something to trigger another monster, and the second monster would perform its action before the first monster could fully complete its 'say' command. In obscure cases, this chaining could be several layers deep.
For simple things like monsters talking to each other, the worst that can happen is that some things get out of order, and you might not understand why it works sometimes but not others. But monsters do substantially more than just talk to each other.
And this is where the problem arose. In the course of doing more complicated actions, those actions could be interrupted mid-stream by other monsters trying to complete their triggers. One such set of actions would cause the get_room function to be passed trash. Another such set of actions would damage one of the monsters so that it could never properly die. Both of these sets of actions, and a number of others with similar strange effects, were possible and implemented in monsters on the game; and they were sufficiently rare to explain the infrequent bug reports.
This bug was easier to fix than the first, taking only about three hours to really think about and put together a proper solution. This fix also appears to work, though it does break a dozen or so special monsters that relied on the old behaviour to function.
---------------------------------------------------
There are several morals to this story, which all software engineers worth their salt will immediately recognize:
1) Never assume you'll remember anything about code you've written. When you start working on objects so complicated that it takes you an hour every day to load it into your head so you can go to work, what makes you think you'll remember every detail after a year?
2) Think about the design of any halfway complex system and stick with it. The charm/entangling roots bug was caused entirely by undesigned/spaghetti code for the character destruct process. It was never properly designed because I wasn't experienced enough at the time to know how. It's better now.
3) Never underestimate the power of race conditions and call trees. When even simple/obvious actions can invoke arbitrarily complicated effects capable of invoking other effects, you're walking on very, very dangerous ground.
Software continues to become more complex as time goes on. It pushes at the limits of our minds, bringing programmers to the limit of what they can understand and then begging them to add one more thing. It allows arbitrary expression, but with that comes the cost that our comprehension is limited even for structured objects, to say nothing of more arbitrary and abstract ones.
Where does the future of software lie? Undoubtedly toward increasing complexity. But we will need either better tools, or better brains, to be able to manage it. We are such a young species.
Pretty much the most complicated discrete object in the server is the socket stack. It handles a huge number of protocols and a bunch of different filtering layers, with hooks in various places. The last major redesign of the socket stack was around 1998; it is a testament to the design of that module that it has required no major modification in that entire time period, though many minor modifications have been made to it.
Unfortunately, the server is not made of such lovely discrete objects. Often, obscure bugs lurk in the sea of wild code that makes up the substructure of the system. This weekend I had the incredible good luck to catch and haul two of these bugs to the surface.
For years, there have been a handful of issues that occurred seemingly at random. In the early, old days, some of these would even cause a server crash and reboot the game; but after being unable to find and fix them, the crashes were gradually protected against and ways were found to ignore these spurious events.
Examples of the two most common of these are deceptively simple: sometimes, the get_room function would be asked to find a room but be given no instructions on how to do so, and other times a character (a monster or player, it was difficult to tell) would die and simply 'get stuck'.
In reality, these are monstrously difficult. The get_room function lacking instruction was mindboggling - it happened perhaps twice a year, the reports were never the same, and there were thousands of places that could be calling the function. Even as our debug facilities improved, no progress was made - there just seemed to be no rhyme or reason to it.
The dead character bug was just as bad. The entire destruct and event handling process was revisited and inspected several times, but no holes were ever found. Until thursday.
---------------------------------------------------
In the course of trying to add some new restrictions to the 'charm' spell, I noticed that the destruct sequences for the charm, possession, and entangling roots spells looked a little goofy. We didn't check or clear certain important things, but there were comments indicating that we didn't need to because something else over there would take care of it. Keep in mind that this was entirely my code; I had written this well over a decade ago.
In the course of looking at this, I suddenly had a realization: there were no 'holes' in the logic for charm or possession, but maybe, just maybe, there was a hole in entangling roots. Entangling roots did come later, and quite frankly the code for it was a complete hackjob.
Within ten minutes, I had my answer. There was indeed a conflict between entangling roots and the 'special' code that made possession and charm work. The problem then became, how exactly do you fix such a mess? After about two hours of thinking about it and five more of carefully backing things out and reorganizing, I got what appears to be a stable fix. There's still some debug logging in it, but this bug appears to be properly killed. The new code is simpler, the checks are stronger, and we don't rely on obscure handlers to clean up messes. I hope.
---------------------------------------------------
I thought that was the end of my troubles in the short term, but then Glorida shows up with some obscure problem of his own. He's been working on mob programs, and had built something rather complex that simply was not working. Not only wasn't it working, but it was doing something weird, and it was doing it reliably.
This is another one of those 'fairly complex' pieces of the system. It took me about two hours off and on to get it loaded into my brain so I could really think about it. It then took me probably another hour to really understand what was going on, and figure out what was happening. And then it occurred to me:
This explains a lot of those debug log reports over the years!
The symptoms he uncovered showed up as a very unusual sort of 'doing things before other things have completed' recursive issue. One example of it is that a monster would 'say' something to trigger another monster, and the second monster would perform its action before the first monster could fully complete its 'say' command. In obscure cases, this chaining could be several layers deep.
For simple things like monsters talking to each other, the worst that can happen is that some things get out of order, and you might not understand why it works sometimes but not others. But monsters do substantially more than just talk to each other.
And this is where the problem arose. In the course of doing more complicated actions, those actions could be interrupted mid-stream by other monsters trying to complete their triggers. One such set of actions would cause the get_room function to be passed trash. Another such set of actions would damage one of the monsters so that it could never properly die. Both of these sets of actions, and a number of others with similar strange effects, were possible and implemented in monsters on the game; and they were sufficiently rare to explain the infrequent bug reports.
This bug was easier to fix than the first, taking only about three hours to really think about and put together a proper solution. This fix also appears to work, though it does break a dozen or so special monsters that relied on the old behaviour to function.
---------------------------------------------------
There are several morals to this story, which all software engineers worth their salt will immediately recognize:
1) Never assume you'll remember anything about code you've written. When you start working on objects so complicated that it takes you an hour every day to load it into your head so you can go to work, what makes you think you'll remember every detail after a year?
2) Think about the design of any halfway complex system and stick with it. The charm/entangling roots bug was caused entirely by undesigned/spaghetti code for the character destruct process. It was never properly designed because I wasn't experienced enough at the time to know how. It's better now.
3) Never underestimate the power of race conditions and call trees. When even simple/obvious actions can invoke arbitrarily complicated effects capable of invoking other effects, you're walking on very, very dangerous ground.
Software continues to become more complex as time goes on. It pushes at the limits of our minds, bringing programmers to the limit of what they can understand and then begging them to add one more thing. It allows arbitrary expression, but with that comes the cost that our comprehension is limited even for structured objects, to say nothing of more arbitrary and abstract ones.
Where does the future of software lie? Undoubtedly toward increasing complexity. But we will need either better tools, or better brains, to be able to manage it. We are such a young species.
Tuesday, February 10, 2009
wxWidgets sucks...
... but even though it sucks, I can't really blame it. I think the problem is endemic when using pretty much any large toolkit, especially younger ones or ones that are built on top of other toolkits.
First, some background. wxWidgets is a windowing library to allow people to quickly and easily develop GUI applications. It's cross platform and in general built on top of other toolkits. This means that the layers of toolkits upon toolkits is often at least two levels deep, exponentially increasing the likelyhood that bugs in one layer will foul up things elsewhere.
These sorts of interactions are causing me huge difficulties with the display and input windows in the client. After much soul searching, I have what appears to be three choices going forward, each with various pros and cons, none of which are ideal. In short:
1) Use the native wxTextCtrl class. Pros - some minimal screen reader support, cut/paste works, already mostly working. Cons - scrolling doesn't work worth a shit, different performance on different windows versions, urls are unreadable for at least some people, no way to disable beeping on backspace, and easily a third of the style flags and other functions don't work or don't work correctly, often with no workaround. Further, the god damned thing intercepts certain keystrokes, specifically Control-C, before I can get at them, and there appears to be no workaround for this.
2) Use the scintilla derived display class. Pros - someone else maintains it. Cons - pain in the ass to get working, manual url and cut/paste handling, bigger, vastly more complicated than my application requires, and inability to configure simple, important things like the background color.
3) Write my own. Pros - it will do exactly what I want, when I want, with the interface that I want. Should be fast and small. Cons - I have to do all the work.
After all the crap I've been through in the last month trying to get the default text controls to do incredibly simple, obvious things, I'm heavily leaning toward option 3 at this point. In the time I've spent trying to get existing controls to work, I could have without question written my own custom primitive. Or rather, I now know enough to write my own. The road would have been rockier in the beginning.
I think this is a prime example of why I rail against toolkits in general. They are very risky; there's no guarantee that what you're using will work, or can be made to work the way you want it to. In this case, I have lost a huge amount of time trying to work around toolkit limitations.
Option 3, here I come.
First, some background. wxWidgets is a windowing library to allow people to quickly and easily develop GUI applications. It's cross platform and in general built on top of other toolkits. This means that the layers of toolkits upon toolkits is often at least two levels deep, exponentially increasing the likelyhood that bugs in one layer will foul up things elsewhere.
These sorts of interactions are causing me huge difficulties with the display and input windows in the client. After much soul searching, I have what appears to be three choices going forward, each with various pros and cons, none of which are ideal. In short:
1) Use the native wxTextCtrl class. Pros - some minimal screen reader support, cut/paste works, already mostly working. Cons - scrolling doesn't work worth a shit, different performance on different windows versions, urls are unreadable for at least some people, no way to disable beeping on backspace, and easily a third of the style flags and other functions don't work or don't work correctly, often with no workaround. Further, the god damned thing intercepts certain keystrokes, specifically Control-C, before I can get at them, and there appears to be no workaround for this.
2) Use the scintilla derived display class. Pros - someone else maintains it. Cons - pain in the ass to get working, manual url and cut/paste handling, bigger, vastly more complicated than my application requires, and inability to configure simple, important things like the background color.
3) Write my own. Pros - it will do exactly what I want, when I want, with the interface that I want. Should be fast and small. Cons - I have to do all the work.
After all the crap I've been through in the last month trying to get the default text controls to do incredibly simple, obvious things, I'm heavily leaning toward option 3 at this point. In the time I've spent trying to get existing controls to work, I could have without question written my own custom primitive. Or rather, I now know enough to write my own. The road would have been rockier in the beginning.
I think this is a prime example of why I rail against toolkits in general. They are very risky; there's no guarantee that what you're using will work, or can be made to work the way you want it to. In this case, I have lost a huge amount of time trying to work around toolkit limitations.
Option 3, here I come.
Thursday, November 6, 2008
Software Complexity and Human Limits
As you might have read from other posts here, I've been working on a gui client for Alter Aeon. Gui code is fundamentally different from what I'm used to working on, in that it has loads of function pointer handlers and 'virtual internal state' that is mostly held by which objects are instantiated and what they're presently doing. At its lowest level, this can of course be represented by a perfectly ordinary state machine, however these state machines are fairly large and the state it's currently in isn't always obvious.
[As a side note, I do work in FAX software in my off time, and this also has some pretty huge state machines. However, those state machines are really well defined in one central location, and it's much more obvious what's going on. Even saying that, I still have a hard time with those.]
While working on this client, I quickly lose track of what needs to be done and how the states work. The client itself is big enough that I can't remember everything that is in it; and after not working on it seriously for almost two years, I had a lot of trouble getting back up to speed.
A good part of this is that the code itself was basically built from the ground up, and then never cleaned up by a maintainer. Maintenance is more than just adding a button here or there and fixing minor bugs; maintenance of software means getting in there and saying "why the hell are these drawing functions intermingled with scroll locking?", then moving things around so it makes more sense. A good example of this is the button system, the entirety of which is contained in the main window implementation file. This serves only to clutter up the main window implementation, and further make the button code hard as hell to understand.
So I've been doing the proper thing and moving functions around, grouping them by functionality, and trying to make useful classes on the side where I can hide details. But even so, I can tell that my standard way of thinking about and writing code is showing signs of strain. Much like I'm having to retool my picking technique on guitar, I need to retool my brain to handle this kind of abstraction better.
I don't really need any advice or recommendations for books here; I know where to find that sort of thing if I need it, and I've done this enough to know when I'll need it. I even know conceptually how to approach this kind of problem; I think I'm just currently bad at it. Fortunately (or perhaps unfortunately), there's a lot of work left to do on this client, so I'll get lots of practice.
[As a side note, I do work in FAX software in my off time, and this also has some pretty huge state machines. However, those state machines are really well defined in one central location, and it's much more obvious what's going on. Even saying that, I still have a hard time with those.]
While working on this client, I quickly lose track of what needs to be done and how the states work. The client itself is big enough that I can't remember everything that is in it; and after not working on it seriously for almost two years, I had a lot of trouble getting back up to speed.
A good part of this is that the code itself was basically built from the ground up, and then never cleaned up by a maintainer. Maintenance is more than just adding a button here or there and fixing minor bugs; maintenance of software means getting in there and saying "why the hell are these drawing functions intermingled with scroll locking?", then moving things around so it makes more sense. A good example of this is the button system, the entirety of which is contained in the main window implementation file. This serves only to clutter up the main window implementation, and further make the button code hard as hell to understand.
So I've been doing the proper thing and moving functions around, grouping them by functionality, and trying to make useful classes on the side where I can hide details. But even so, I can tell that my standard way of thinking about and writing code is showing signs of strain. Much like I'm having to retool my picking technique on guitar, I need to retool my brain to handle this kind of abstraction better.
I don't really need any advice or recommendations for books here; I know where to find that sort of thing if I need it, and I've done this enough to know when I'll need it. I even know conceptually how to approach this kind of problem; I think I'm just currently bad at it. Fortunately (or perhaps unfortunately), there's a lot of work left to do on this client, so I'll get lots of practice.
Subscribe to:
Posts (Atom)