Creative ways to find software bugs

[1337]

Today in the lab, we used a band saw and binary search to find a bug in a program that we used (we didn't write it, but it's open source).

Since this sounds sooo weird, I'm sure you can have some ideas as to what the heck we might have done in the lab ^^

I'm looking forward to hearing to some interesting ideas! And of course, in due time, I'll post our real world example

[Mozork]

This really sounds weird. ^^
The binary search I get, but the band saw?

Ok, here's my scenario: You were testing the vision system of a automatic wood splitter machine thingy, that looks at each wood block and decides how and in how many pieces it has to be cut, when you noticed an oddity. From time to time the machine would just go wild and reduce a block of wood into tiny bits, other times it would work perfectly fine and you had no idea what was going wrong.
First you had the idea that it was just some uninitialized value, as always when something entirely strange happens, but after thorough examination of your code you came up empty and confused.
So you just thought, what if it had something to do with the size and shape of the wood block to be split? You got your band saw (and an ax for good measure of course) and started producing differently shaped wood block of various sizes and fed them to the machine. Soon a pattern emerged. Particularly large blocks were split just fine, where the small ones seemed to trigger some kind of machine chopping frenzy. However you needed to know where and why that particular distinction happened so you started making and inserting your blocks in a binary search fashion. Finally you arrived at two block which were almost the same but only one of them ended up being completely shredded. After looking at the data from your classifier, comparing the different results, you realized that not your code was faulty but that of an open source library you used (which will go unnamed, to not expose them to ridicule).
That is the story of how a band saw (with some help of an ax) and binary search led to a bug being found.

[1337]

Very, very entertaining story You just earned yourself lunch IFF you are still at ETH when I get back Please remind me then.

The real story however involved a multimeter for the binary search: divide by two and test both for the fault and repeat that iteratively until the problem has been located.

Maybe you can make a guess... (the multimeter was set in "Ohm" mode)

[x3n]

Ok let me make another shot in the dark.

You're creating a robot which is able to move and is equipped with some sensors. You notice that it usually behaves as expected, but sometimes, often after a quick move, it loses orientation. You guess that the IMU (inertial measurement unit) is bugged and decide to test it's response for high accelerations/velocities. But how can you achieve constant/known movement? Simply moving it by hand doesn't work because it's too inaccurate.

So you decide to move it with a machine and the simplest solution you find is to attach it to the blade of a band saw. You let it move for some time, then plug it back to the computer to check the log. Afterward you change the speed of the saw and repeat the process until you find the critical speed.

Because it takes way too long to test all possible speeds of the band saw, you go for binary search: Let it run at maximal speed, check if it fails. Indeed it does, so you halve the speed. Unfortunately the steering of the band saw is logarithmic, to achieve higher accuracy at low speed. So you open it and use your multimeter to measure the resistance of the potentiometer. You assume that the speed of the saw depends linearly on the resistance, so you can now easily control the speed of the saw.

[1337]

And who says computer guys are not creative ^^

I think I owe you the real story:

A few weeks ago, a colleague of mine developed a PCB with several sensors on it. Since all the circuits for the different sensors and the power supply are publicly available, he "only" had to connect them, but the entire layout had to be done by hand (including the routing). A first board was then ordered according to his design and I helped him solder everything onto it. Apart from a few issues, the circuit worked just fine.
After that, a second iteration was issues by our supervisor because he wanted the board to be smaller. So that's what my colleague did. But to make it smaller, 2 copper layers were not enough anymore, so he went for a 4 layer board with the in-between layers being almost completely filled for ground and 2.3V power.
Again, the board was shipped and we started soldering components onto it but weren't sure about a capacitor's connection to the board. So we checked with the multimeter: connection OK. For some unknown reasons I checked the capacitor resistance as well, 0.1 Ohm. Well, not exactly what you expect... Capacitor fried because of the heat? Maybe, so we unsoldered it just to see that it was fine. There was another capacitor in parallel, so we unsoldered that as well --> fine too. But curiously, there was still a short over the capacitor pads and these were connected to Ground and Power.
Hmm, what the heck? Since we already soldered about 8 chips and another 2 components onto the board, we started removing them. Guess what, the short was still there once the board was empty ;(
We each double, triple and quadruple checked the pads for shorts, none. And long before that, my colleague triple checked the design on the computer (he had three almost independent ways to assert that there was no Ground-Power connection).
So, it had to be either a bug in the design program or a manufacturing error.

Around that point I made a joke by suggesting to do a binary search for the fault: just cut it in half, check, repeat.

Anyway, the next morning we talked to a guy who know more about PCBs than we did, but it didn't get us anywhere and it was pretty obvious that the board was useless anyway.

So my colleague turned my previous joke into reality and we started cutting it with a band saw. After the first cut, the fault was only in one half, phew. Then there were a lot of vias in the faulty part, so we cut off only about a quarter where there was just space for the Bluetooth module with very little vias and wires. And to our surprise, this was actually the faulty part (about 1.5cm x 3cm). With little to check, we pulled out the design and examined the vias. And when I checked the top left one (almost touching the corner), I could actually see that both in-between layers were connected to the via. That's bad... But in his design, that wasn't the case. So what the heck?
A hunch told me to check the actual files we submitted and the project file that was used to created it. And there it was, the fault. However, doing only some simple operations in the program already fixed the problem (leaving some space around the via in bother in-between layers as it was designed to have).

At least we knew where the problem was, but we didn't know what caused it. Anyway, a few hours later, when correcting a mechanical problem with the USB connector, my colleague used the auto-router program again which was different from the design program. And upon transferring the design back, the error was there again! And you could even see the problem in the auto-router program. So there was our bug!

And what did we learn about this? Finding bugs is not only about compiling a program a hundred times over ^^

[x3n]

haha nice story

yeah I thought at this as well, but I couldn't imagine how cutting a board in half would make any sense. But of course if you don't search for a short circuit between A and B but rather between the two sides of a board, it makes more sense

[BadElvis]

Cool story, I tought of pcb errors because, once did a similar thing. Only I was too impatient so I broke the board apart...

In my case however, it was a soldering problem, not software . Btw, when will you be back in CH?

Cheers!

[1337]

I will be heading home for a few days on August 4, but my work contract ends February 16. So I'll stay for a while