KodiaX987
Aug 28, 2008, 06:55 PM
This is a callout to those would-be programmers and gamers who believe that any glitch in any game can be fixed with the press of a button and a few lines of code.
For starters, if you are reading this and are not a professional programmer, then chances are your programming experience is limited to simple database management tools, console calculators and Hello World. If at this point you are tempted to reply saying "Actually I do this and that and program all sorts of stuff even though it's not my job", please close your browser window, take the cable off your computer and go kill yourself. I don't need some fucking nitpicker to ruin my thread with his needless attempts to get the last word on an argument that's never begun.
So now, what's the average size of a program you've done at school? Possibly a few hundred lines, a handful of forms and some reference files, be it text or images, what have you. Bugs? Debugging is easy: you just have one main function, a few sub-functions, you can find and solve any problem in a manner of minutes.
Now, where did I go after computer school? I went to get a job of course!
My first job was in a team of three programmers in an industrial company. The database was so large that loading the record of all orders took a full minute. The application that everyone used to communicate with the database had been made in-house and the executable alone weighted in at 80 megabytes. The extensions and other resources popped that up to 100 MBs. But still, 80 megabytes of code and form design. Plus the database itself, with all its stored procedures and subfunctions. Plus the in-house programming language that was used to create the virtual blueprints to be sent to the cutting machine.
Now, somebody comes along and reports that a door hinge hole has been pierced 1/16th of an inch too far to the left. This is not good for us because the machine is supposed to be precise up to 1/1000th of an inch and all the other holes are in the right places. Henceforth, we look up the code.
That sort of problem, in theory, takes minutes to solve. Just tell the program to place the hole 1/16 inch more to the right and away you go. But that's without counting the possible ways this particular piece can be done:
-The dimensions are full-parametric. The larger a piece is, the more holes we add because a bigger door needs more hinges for support. This needs to be programmed. Thresholds to say how many holes to put in and where.
-The hinge holes are placed depending on the door's type, material and dimensions.
In the end, the code that governs something as simple and stupid as punching holes in a piece of wood takes up roughly five pages, printed. Among them, somewhere, is the code for the holes that were misplaced. And I'm never guaranteed that the previous programmer has commented his stuff, which means I'm going to spend some time just looking for where things are going wrong. And then I need to fix it without breaking anything else in the process, because if I want to move a hole 1/16 inch to the right in one instance combination of door type, measure section and material; not for every single piece we make.
Solving a bug like that? If I'm lucky and the code's short and simple, roughly 15 minutes to fix it, test it and confirm verification. For longer and more complex pieces, this can be half a day. Perhaps even a full day. For older code referring to things done in some outdated tacked-on manner backed when things weren't as mature? I can be breaking my head over that sort of bug for two or three days. Even though the end result and the problem in the first place looked pretty minimal.
We'll put this in perspective. If there's a faulty code that misplaces one component on your brand new CPU so much as one nanometer off, you're screwed. One nanometer just ruined a 200$ piece of hardware.
Now, let's move on.
Second job. I am working with about 30 or so programmers on an enterprise management software. We have several customers with each their own specifications and demands. The code weighs in at 2 gigabytes. Two gigabytes of nothing but text.
Right away, you can tell things are getting complex.
We have a module for the management software. We have a module for the product configurator. We have a module for the product bank. We have a module for the LAN and server installations. We have over 60 different modules and they must all work well. Have fun.
The average module spans about 150 code files of varying length, most of them about 10 pages long, and every module relies on external plugins that do not belong to us and who sometimes do not send us the data in a way that we like, and we must find a way around that. Most of the code I trace through is obfuscated: it's encrypted so that outsiders can't read it.
Oops! We found a bug!
I am about to tell you a real example of a bug I investigated and solved this week.
The symptom: we have a hierarchical table, a table that contains items and sub-items. The items can be expanded and collapsed to see the sub-items - think of Windows Explorer's tree view, and you get the idea. Now here's the problem: we expand one item, and then expand another. Suddenly, the sub-items of the first expanded item become the sub-items of the second. Worse, if the two lists of sub-items are of different size, the program crashes.
I make a quick run-through of the code and form two hypothesises: either the displaying engine chooses the wrong items to display, or the items that are being given out are not the right ones.
Investigation: 12 hours.
Twelve hours of going through the code line by line, checking each suspected variable and object (most of which have a sub-list of variables and other lists that quickly takes on astronomical proportions) and making sure that everything is in its right place. Several times, I hit a lead, only to find out it was a red herring.
Eventually, I find the cause: the items being given to the displaying engine are only the sub-items generated by the expansion of the last item we clicked on. Result: the items of the other expanded lists are not in the pool for the displaying engine to pick up.
Solving the problem: 3 hours.
Now that I have found the culprit, I need to work around the problem and create the solution. Each time we expand an item, the pool of sub-items is refreshed, and this is where things go wrong, because some items we want to keep are being flushed. Because the table's software is third-party, we only have access to the displaying functions: we want to show this, and the table does its thing. We lack a crucial information: it's impossible for us to know which items of the table are expanded.
Final solution: we store every sub-item in the pool, whether its parent items have been expanded or not. It's not a good solution. What if we have a list of a hundred items, each containing several dozen sub-items? We'll have memory management problems later on, but for now, this is the best we can muster. I have other bugs to solve, which are just as urgent, and the release is scheduled on Monday.
I have until Monday to get the highest amount of bugs sorted out and stable. We can't push it back. The customer has already been waiting on us. We fix what we can, and we do our release. That's final. The other bugs will be fixed in the next update.
Other examples of actual bugs solved at my workplace:
-Generalized errors in the displaying engine. For two months, the person assigned to this task of "stabilizing" the render has taken an average of 4 days to investigate each bug, each of which has been solved with just one or two characters. Each time he corrects a problem, he must re-test the render to make sure this modification has not broken something else elsewhere. He has been correcting and stabilizing the same code sheet, a monster 10,000 lines long, for two months.
-A rendering error causes arc-circles to face the wrong direction. We have not found the cause of this problem yet.
-For a specific line of products, attempting to import them into a quotation from another quotation causes rendering errors. The culprit turned out to be the way the importation is handled within a very precise combination of factors.
-The program wrongly announces that an upload has completed even if the master server is down. It turned out that the code that manages the upload was naive instead of paranoid: If it found nothing wrong, it reported a success - when it should have been coded in such a way that if it didn't get a positive response from the server, it should've announced a failure.
Are we incompetent? Far from that. We have a centralized bug database, a history of modifications, and backups of all our code. I go through code all day, and I can judge that the people who work here are competent, knowledgeable and capable. However, the average program is so complex that it's impossible to escape oversights. As the program grows, errors accumulate themselves and generate bugs that are sometimes extremely hard to understand. Not only this, but we are also at the mercy of programming errors in the third-party software we use and incorporate into our own program. In the end, the behavior of one single button can be the fruit of thousands upon thousands of lines of code. It seems simple to the user, but a whole lot of work has gone into it. Making just one function, one button that does something, can cost the customer upwards of 1000$.
So if you are tempted to rant about the developers of a game being incompetent for introducing "obvious" glitches and other "easy" mistakes, think about this:
-A professional-level program can contain hundreds of thousands, sometimes millions of lines of code.
-A single mouse click in a game can be driven by and generate an effect managed by tens of thousands of lines of code. Gordon Freeman firing one round from his pistol and killing a Metrocop takes into account a dizzying array of factors. If the Metrocop is not hurt by the bullet, and we are sure that Gordon fired accurately, then we have that much code to wade through to find the cause of the malfunction.
-Bugs are given priority levels. Some mildly annoying and very visible bugs are low-prioritized in favor of unseen bugs that compromise the security and stability of the program.
-Very often, solutions tested and marked as correct in a development environment turn out to be inadequate in the actual end-user client environment.
If, at any point, you feel that you can do better than "those no-good devs who obviously don't care about us", then I invite you to learn programming and get hired. If you don't like to see bugs, come here with us and work to solve them. Programmers don't have a foosball table or a miniature basketball net in their office. Stop thinking in dotcom bubble terms. Programmers program. They spend their days investigating bugs and solving them. They spend their days developing new modules and testing them in every possible way they can think of. Programmers must be patient, tenacious, organized and creative.
We are not boring either. We, like all workers, take an occasional break and shoot the puck. We play a quick LAN party at lunch to ease the pressure. We go out to have a drink and play some pool when we can afford the time to. Most of us work beyond the usual 40 hours a week. We help each other. We help on each other's projects because we know that casting a different pair of eyes on an old problem can give birth to a new solution.
If you think programmers are incompetent, then sit down and make a program. Make it an endless project. Perform in survival mode. Start your first module, and expand, every day, without fail. Consider it a mini-job. Dedicate an hour of each business day to advancing your program. Once you're done making a module, create another on the same theme. Keep going and going. Do not miss a day unless the circumstances are outside of your control (sickness, death in the family, family outing, etc.) Do not take a day off because "you don't feel like it". If you do, stop your project and assess how far you've gone, how long you lasted, and the overall quality of the program you have made.
In most cases, you will stop after a few days, with an incomplete and very buggy program, which will sit on your hard drive untouched for weeks before, finally, being deleted.
In rare cases, you will stop after a few weeks, with a working program that contains some unfinished modules, but the essence is there.
In extremely rare cases, you will end up with a program that keeps on expanding and gaining features, as you dilligently work on it with regularity and consistency, for several months, perhaps even years in a row (http://www.bay12games.com/dwarves/dev.html).
And even this game I mentioned a paragraph up has a large amount of things that could be fixed. The maker of this game has graciously listed them all for us to see, to know what's going on with his project.
Do you think programming is easy?
All right then.
Get out there and program.
For starters, if you are reading this and are not a professional programmer, then chances are your programming experience is limited to simple database management tools, console calculators and Hello World. If at this point you are tempted to reply saying "Actually I do this and that and program all sorts of stuff even though it's not my job", please close your browser window, take the cable off your computer and go kill yourself. I don't need some fucking nitpicker to ruin my thread with his needless attempts to get the last word on an argument that's never begun.
So now, what's the average size of a program you've done at school? Possibly a few hundred lines, a handful of forms and some reference files, be it text or images, what have you. Bugs? Debugging is easy: you just have one main function, a few sub-functions, you can find and solve any problem in a manner of minutes.
Now, where did I go after computer school? I went to get a job of course!
My first job was in a team of three programmers in an industrial company. The database was so large that loading the record of all orders took a full minute. The application that everyone used to communicate with the database had been made in-house and the executable alone weighted in at 80 megabytes. The extensions and other resources popped that up to 100 MBs. But still, 80 megabytes of code and form design. Plus the database itself, with all its stored procedures and subfunctions. Plus the in-house programming language that was used to create the virtual blueprints to be sent to the cutting machine.
Now, somebody comes along and reports that a door hinge hole has been pierced 1/16th of an inch too far to the left. This is not good for us because the machine is supposed to be precise up to 1/1000th of an inch and all the other holes are in the right places. Henceforth, we look up the code.
That sort of problem, in theory, takes minutes to solve. Just tell the program to place the hole 1/16 inch more to the right and away you go. But that's without counting the possible ways this particular piece can be done:
-The dimensions are full-parametric. The larger a piece is, the more holes we add because a bigger door needs more hinges for support. This needs to be programmed. Thresholds to say how many holes to put in and where.
-The hinge holes are placed depending on the door's type, material and dimensions.
In the end, the code that governs something as simple and stupid as punching holes in a piece of wood takes up roughly five pages, printed. Among them, somewhere, is the code for the holes that were misplaced. And I'm never guaranteed that the previous programmer has commented his stuff, which means I'm going to spend some time just looking for where things are going wrong. And then I need to fix it without breaking anything else in the process, because if I want to move a hole 1/16 inch to the right in one instance combination of door type, measure section and material; not for every single piece we make.
Solving a bug like that? If I'm lucky and the code's short and simple, roughly 15 minutes to fix it, test it and confirm verification. For longer and more complex pieces, this can be half a day. Perhaps even a full day. For older code referring to things done in some outdated tacked-on manner backed when things weren't as mature? I can be breaking my head over that sort of bug for two or three days. Even though the end result and the problem in the first place looked pretty minimal.
We'll put this in perspective. If there's a faulty code that misplaces one component on your brand new CPU so much as one nanometer off, you're screwed. One nanometer just ruined a 200$ piece of hardware.
Now, let's move on.
Second job. I am working with about 30 or so programmers on an enterprise management software. We have several customers with each their own specifications and demands. The code weighs in at 2 gigabytes. Two gigabytes of nothing but text.
Right away, you can tell things are getting complex.
We have a module for the management software. We have a module for the product configurator. We have a module for the product bank. We have a module for the LAN and server installations. We have over 60 different modules and they must all work well. Have fun.
The average module spans about 150 code files of varying length, most of them about 10 pages long, and every module relies on external plugins that do not belong to us and who sometimes do not send us the data in a way that we like, and we must find a way around that. Most of the code I trace through is obfuscated: it's encrypted so that outsiders can't read it.
Oops! We found a bug!
I am about to tell you a real example of a bug I investigated and solved this week.
The symptom: we have a hierarchical table, a table that contains items and sub-items. The items can be expanded and collapsed to see the sub-items - think of Windows Explorer's tree view, and you get the idea. Now here's the problem: we expand one item, and then expand another. Suddenly, the sub-items of the first expanded item become the sub-items of the second. Worse, if the two lists of sub-items are of different size, the program crashes.
I make a quick run-through of the code and form two hypothesises: either the displaying engine chooses the wrong items to display, or the items that are being given out are not the right ones.
Investigation: 12 hours.
Twelve hours of going through the code line by line, checking each suspected variable and object (most of which have a sub-list of variables and other lists that quickly takes on astronomical proportions) and making sure that everything is in its right place. Several times, I hit a lead, only to find out it was a red herring.
Eventually, I find the cause: the items being given to the displaying engine are only the sub-items generated by the expansion of the last item we clicked on. Result: the items of the other expanded lists are not in the pool for the displaying engine to pick up.
Solving the problem: 3 hours.
Now that I have found the culprit, I need to work around the problem and create the solution. Each time we expand an item, the pool of sub-items is refreshed, and this is where things go wrong, because some items we want to keep are being flushed. Because the table's software is third-party, we only have access to the displaying functions: we want to show this, and the table does its thing. We lack a crucial information: it's impossible for us to know which items of the table are expanded.
Final solution: we store every sub-item in the pool, whether its parent items have been expanded or not. It's not a good solution. What if we have a list of a hundred items, each containing several dozen sub-items? We'll have memory management problems later on, but for now, this is the best we can muster. I have other bugs to solve, which are just as urgent, and the release is scheduled on Monday.
I have until Monday to get the highest amount of bugs sorted out and stable. We can't push it back. The customer has already been waiting on us. We fix what we can, and we do our release. That's final. The other bugs will be fixed in the next update.
Other examples of actual bugs solved at my workplace:
-Generalized errors in the displaying engine. For two months, the person assigned to this task of "stabilizing" the render has taken an average of 4 days to investigate each bug, each of which has been solved with just one or two characters. Each time he corrects a problem, he must re-test the render to make sure this modification has not broken something else elsewhere. He has been correcting and stabilizing the same code sheet, a monster 10,000 lines long, for two months.
-A rendering error causes arc-circles to face the wrong direction. We have not found the cause of this problem yet.
-For a specific line of products, attempting to import them into a quotation from another quotation causes rendering errors. The culprit turned out to be the way the importation is handled within a very precise combination of factors.
-The program wrongly announces that an upload has completed even if the master server is down. It turned out that the code that manages the upload was naive instead of paranoid: If it found nothing wrong, it reported a success - when it should have been coded in such a way that if it didn't get a positive response from the server, it should've announced a failure.
Are we incompetent? Far from that. We have a centralized bug database, a history of modifications, and backups of all our code. I go through code all day, and I can judge that the people who work here are competent, knowledgeable and capable. However, the average program is so complex that it's impossible to escape oversights. As the program grows, errors accumulate themselves and generate bugs that are sometimes extremely hard to understand. Not only this, but we are also at the mercy of programming errors in the third-party software we use and incorporate into our own program. In the end, the behavior of one single button can be the fruit of thousands upon thousands of lines of code. It seems simple to the user, but a whole lot of work has gone into it. Making just one function, one button that does something, can cost the customer upwards of 1000$.
So if you are tempted to rant about the developers of a game being incompetent for introducing "obvious" glitches and other "easy" mistakes, think about this:
-A professional-level program can contain hundreds of thousands, sometimes millions of lines of code.
-A single mouse click in a game can be driven by and generate an effect managed by tens of thousands of lines of code. Gordon Freeman firing one round from his pistol and killing a Metrocop takes into account a dizzying array of factors. If the Metrocop is not hurt by the bullet, and we are sure that Gordon fired accurately, then we have that much code to wade through to find the cause of the malfunction.
-Bugs are given priority levels. Some mildly annoying and very visible bugs are low-prioritized in favor of unseen bugs that compromise the security and stability of the program.
-Very often, solutions tested and marked as correct in a development environment turn out to be inadequate in the actual end-user client environment.
If, at any point, you feel that you can do better than "those no-good devs who obviously don't care about us", then I invite you to learn programming and get hired. If you don't like to see bugs, come here with us and work to solve them. Programmers don't have a foosball table or a miniature basketball net in their office. Stop thinking in dotcom bubble terms. Programmers program. They spend their days investigating bugs and solving them. They spend their days developing new modules and testing them in every possible way they can think of. Programmers must be patient, tenacious, organized and creative.
We are not boring either. We, like all workers, take an occasional break and shoot the puck. We play a quick LAN party at lunch to ease the pressure. We go out to have a drink and play some pool when we can afford the time to. Most of us work beyond the usual 40 hours a week. We help each other. We help on each other's projects because we know that casting a different pair of eyes on an old problem can give birth to a new solution.
If you think programmers are incompetent, then sit down and make a program. Make it an endless project. Perform in survival mode. Start your first module, and expand, every day, without fail. Consider it a mini-job. Dedicate an hour of each business day to advancing your program. Once you're done making a module, create another on the same theme. Keep going and going. Do not miss a day unless the circumstances are outside of your control (sickness, death in the family, family outing, etc.) Do not take a day off because "you don't feel like it". If you do, stop your project and assess how far you've gone, how long you lasted, and the overall quality of the program you have made.
In most cases, you will stop after a few days, with an incomplete and very buggy program, which will sit on your hard drive untouched for weeks before, finally, being deleted.
In rare cases, you will stop after a few weeks, with a working program that contains some unfinished modules, but the essence is there.
In extremely rare cases, you will end up with a program that keeps on expanding and gaining features, as you dilligently work on it with regularity and consistency, for several months, perhaps even years in a row (http://www.bay12games.com/dwarves/dev.html).
And even this game I mentioned a paragraph up has a large amount of things that could be fixed. The maker of this game has graciously listed them all for us to see, to know what's going on with his project.
Do you think programming is easy?
All right then.
Get out there and program.