Valgrind etc.

| categories: luftballons, tech, valgrind | View Comments

My last post attracted some... awkward press. I didn't intend to cause a panic, and I'm hoping nobody other than Phoronix took it that way.

I hemorrhaged disclaimers in the previous post, so let's skip the apologies and get right to the point: when I took the advice I got from the Mesa devs and rebuilt libdrm with Valgrind's development headers installed, the number of errors was reduced to 3 (filed). Turning up leak checking revealed some more intrigue, but still nothing to panic about.

The point I was making was about Valgrind itself and development practices, and that still stands. I wish we could build more of our packages against Valgrind, so packages with special needs like Mesa could offer a better developer experience out of the box, but it would make the dependency tree fairly ugly, which can be its own pain.

So let's all relax. I fixed the problem I originally had and have pushed a pile of Luftballons changes. It's closer than ever to having a complete (though admittedly slow) graphics stack. Good things are happening. Everything is going to be ok.

Read and Post Comments

Valgrind is Not Optional

| categories: tech | View Comments

Disclaimer: I really don't mean to be picking on the Mesa guys. Libre graphics folk are already terribly overworked, and it's neither my place nor my intention to tell them to try harder. Nonetheless, given what I'm working on, this is the example I have.

I'm working on a rather large and complicated change to Luftballons, and I've managed to get myself into trouble with what looks like a rather nasty use-after-free bug.

No problem, I'll just fire up valgrind.

==31830== For counts of detected and suppressed errors, rerun with: -v
==31830== Use --track-origins=yes to see where uninitialised values come from
==31830== ERROR SUMMARY: 507326 errors from 743 contexts (suppressed: 2 from 2)

That's a lot of errors. And this is after I used --gen-suppressions with a known-working version.

They can't all be me, can they?

==31830== Invalid read of size 4   
==31830==    at 0x64311D0: gen7\_update\_texture\_surface (gen7\_wm\_surface\_state.c:325)
==31830==    by 0x64238F3: brw\_update\_texture\_surfaces (brw\_wm\_surface\_state.c:1497)
==31830==    by 0x63FAD91: brw\_upload\_state (brw\_state\_upload.c:501)
==31830==    by 0x63B844A: brw\_draw\_prims (brw\_draw.c:501)
==31830==    by 0x67C02E9: vbo\_handle\_primitive\_restart (vbo\_exec\_array.c:549)
==31830==    by 0x67C143C: vbo\_validated\_drawrangeelements (vbo\_exec\_array.c:968)
==31830==    by 0x67C195A: vbo\_exec\_DrawElementsBaseVertex (vbo\_exec\_array.c:1141)
==31830==    by 0x4C1FC25: mesh\_draw (mesh.c:232)
==31830==    by 0x4C23E17: draw\_op\_exec (draw\_op.c:275)
==31830==    by 0x4C274EA: draw\_proc\_run (draw\_proc.c:182)
==31830==    by 0x4C27528: draw\_proc\_run (draw\_proc.c:184)
==31830==    by 0x4C27528: draw\_proc\_run (draw\_proc.c:184)
==31830==  Address 0x7f16da073d20 is not stack'd, malloc'd or (recently) free'd

Nope. Mostly they look to be in Mesa. Thousands of them.

I've previously lost almost an entire day trying to clean this mess up. I almost thought I had the suppressions manicured enough to be worth something, and then another pile broke through.

Some of this is valgrind's fault for not having the best suppression tools in the world, but there's also something a bit worrying about a library making this much noise during normal use. The Java fans have been screaming about the dangers of using "unsafe" languages like C since as long as their kind have existed. I've always disagreed, but that was partly on the grounds that tools like valgrind existed, and, more importantly, that they were being used.

Again I can't pick on the Mesa devs too much, but as a general rule, if you are writing a C project, valgrind is not optional, released code should generate no valgrind errors, and in the case of an exception or valgrind bug, you should ship a suppression file. With the great power of a low level language comes great responsibility, and this simple and easy acceptance test finds many dangerous errors.

Valgrind is about the fourth tool that has broken on me trying to find this bug (surge of GDB bug reports to come soon). Now I'm sitting here trying to determine if hunting a few of these errors constitutes good citizenship or a dangerous slide into yak shaving. I guess it's ironic that it's my own memory mismanagement defect that's causing all the effort. No rest for the wicked I suppose.

Read and Post Comments

Luftballons

| categories: tech, free software, luftballons | View Comments

Remember in my last post when I talked about video games as an example of economic inefficency created by proprietary software, because they duplicate so much effort? I've actually been thinking about that idea a lot, for a long time. More specifically I've been thinking about it in terms of the technology; the graphics engines etc. Studios are constantly competing to best eachother in graphics, and they end up creating more or less the same product; most games are about on a level. The competition is clearly driving some innovation, but the fact that they can't build on eachother's work means a lot of wasted effort to maintain what tends to look like a stalemate from without. It's not a terribly gross example of IP going haywire; again, the innovation is getting driven, but open source has been very good at cutting out this kind of inefficiency, for better or worse.

I guess what I'm trying to say is I'm writing a video game engine.

Meet Luftballons.

There's more than a few open video game engines out there. It's probably fair to ask why write another one. To be fair, the eventual scope of Luftballons is to be much more than an engine; I'd like to build an entire suite of open source game development tools that will allow users to develop games from start to finish (there's some proprietary things like this already, but for paranoia's sake I'm not mentioning them). I could build that around another engine, and of the other options out there I'd say I probably looked closest at the nominally successful Ogre3d, but after reading that it did not intend to support deferred rendering pipelines, I decided to sniff around elsewhere. Eventually I conceptualized something unique enough that I felt it was worth striking out on my own.

Luftballons is basically an experiment in how much difficulty I can remove from the OpenGL API without removing any significant functionality. I chose to aggressively target modern OpenGL (somewhat tempered by Mesa only just barely implementing OpenGL 3 on most hardware), and build a framework that would cater to most of the use cases in question, but be dramatically less intricate than OpenGL. The trick is in considering what the OpenGL API can do, but abstracting away the how; removing all the tiny decisions around order of operations and allocation methods and whether to use instanced geometry, etc. The theory is that we can expose a simpler API that is feature-competitive with the raw drawing API if we don't worry about performance and just assume the most obvious way to draw is fine for everyone. The much more exciting theory is that we can then teach the implementation to be clever and do the optimization behind the scenes without hassling the user with more details. More importantly, a simple API is inherently more future-proof, as new OpenGL features just mean small API extensions or improvements to that under-the-hood optimization.

There's obviously a lot more to do, as well as some cleanup on what I already have (hand-rolling makefiles is great! I've only done it for one project and already my hatred of Automake is completely gone! (seriously, early adopters, I swear it will be in autotools within a week. Promise.)) but I have high hopes. Absorbing that inefficiency I mentioned means creating a ubiquitous open source game development platform. Whether it is this project or something else that does it, I'm hoping to one day reach a point where building a new game without using the most common open game engine is as absurd as building a new embedded platform without using the Linux kernel.

In the meantime, who wants to play with GLSL?

Read and Post Comments

The Future of Information Exchange

| categories: politics, free software, tech | View Comments

I've been dead for a bit. Mostly because I've been working on a little something that I should be able to announce on this blog soon (something much more on-topic for all of you planet readers). But, business first: my last two posts were on piracy, and I think I need one more to leave things neat and tidy before we can all move on for a bit.

In my last post I discussed how piracy can be treated as an "externality," a situation where a transaction that is beneficial to both direct parties harms a third party. I then asked if we were defining externality too broadly to be useful, given that value and harm are subjective. I'll rework an example I used there: if a person finds their house less valuable due to a black neighbor moving in, does that neighbor's purchase of the house next door contain an uncompensated externality?

I put this to my private economist, and he cringed a bit, then answered with a simple "yes." Perceived value is, as usual, the only value, and if racism makes you perceive value differently, non-whites moving in next door does have an externality.

The conclusion we came to was that society isn't necessarily obligated to correct all externalities, and morality may be one reason it chooses not to. We could think of it as a sort of punishment for racism; suffering the externality is merely the wages of being racist.

This gets interesting when we bring it back to our piracy discussion, though. Society has a choice in whether it compensates an externality. That choice may be a moral choice, but really the choice could be made for any other reason as well. Remember, it is this same society that outlines property rights to begin with, and for many information freedom advocates, intellectual property is even more artificial.

So we come full circle: we want to establish a system that is both just and efficient, and we can set up any set of rules we like. IP law as is attempts to apply markets to the problem, which are just where certain common and presumably attainable conditions are met, and which are preceded by a reputation for efficiency. IP law remains a good solution unless:

  • The conditions that establish justice don't work for information goods. This is more or less what Stallman is on about; he basically asserts that closing the externality of piracy is unjust toward the user (and his requirements for accessible source code go further than that).
  • We can't effectively set up markets for information goods.
  • The efficiency markets create can't be created for information goods.
  • There is another system which is better when measured in terms of efficiency and justice.

There's probably more ways to approach the first point than swallowing the entirety of FSF ideology whole, but we'll call Stallman good enough for now.

The second point is interesting. It gets back to the old excludability/rivalrousness debate around intellectual property. When presented with the remark that "information is non-excludable and non-rival, and therefore the market should push its price to zero," most economists will retort that IP law creates that excludability, which is largely its purpose. This is true, but at what cost? Information is excludable under IP law. Also, alcohol was unavailable within US borders from 1920-1933. Draw whatever other comparisons you want, but there's precedent in history of "enforceability" dictating the justice of the government's actions. In the case of prohibition, legal signaling never quite produced a popular moral sense that drinking was "wrong." Eventually the government relented.

The third point is even more exciting. It's hard to say how much could be done with more conservative IP law reforms, but right now, IP, and patents in particular, are absolutely disastrous in terms of efficiency. We can pretty much just say "patent trolls" and end this here, but this argument can actually go further. Surprisingly far, even. Look at this and then look at this. Each of these games features similar play style, probably similar weapons, environments, even roughly similar voice acting. Every year the respective owners of each franchise spend millions to update each game, without any cost-sharing, despite immense amounts of reusable work between them. Market IP law means two people make a new 3D mesh of the same M16 each year. It's a sharp highlight on the reality that information is more efficiently produced when existing information can be freely reused. It's the power of "collaboration," and you hear about it a lot in the open source world, while in the proprietary world you have two giant companies duplicating eachothers' efforts at great expense.

And all of that leads nicely to point four: is there a better way? I have some ideas, but much as I'd like to end this piracy series, I think that should wait for another post. I've scratched my piracy itch for awhile now though, so I should be able to leave it be for awhile. Exciting news in posts to come!

Read and Post Comments

Piracy and Stealing

| categories: politics, free software, tech | View Comments

Arguing is hard.

Even as the only species we know to have discovered the idea, objective discourse doesn't come naturally to humans. Dispassionate discussion of ideas is thwarted by our internal frustration at every turn. And one of the most frustrating parts of arguing is the meta-debate.

We've all had the meta-debate before. It's a superset of the semantic argument: the fight we must win, not to prove our point, but to even be able to talk about it. The argument over what terms and nomenclature should be used in the discussion inevitably precedes the discussion, and often the discussion itself is postponed indefinitely by it.

"Piracy is stealing" is such a meta-argument. Like most meta-arguments, there is a side which believes it to be the argument itself: those who are vehemently against piracy (and potentially there's good reason for everyone to be against piracy, as I shall explain soon), often proceed as though equating piracy to stealing is making a great moral claim, and will defend it as the central tenant of the morality that justifies their argument.

This isn't actually the case. Neither rape nor murder is, strictly speaking, "stealing," yet both are considered morally wrong. Whether piracy is stealing or not has no bearing on when and whether it is moral. Yet a vocal group of anti-piracy activists are insistent on equating piracy to stealing, primarily due to its moral implications.

And certainly, for the majority, calling piracy stealing simplifies the debate. Piracy is stealing, stealing is wrong, therefore piracy is wrong. This is the crux of the meta-rhetorical strength of calling piracy stealing: you can stop the conversation there. You don't have to explain why piracy is wrong if piracy is stealing, because "as we all know," stealing is wrong. Jim Sterling was the first such commentator I heard admit it; that the real motivation in calling piracy stealing was to make pirates "feel" wrong about what they were doing:

We recognize that what they do actually is theft, for no reason other than they don't deserve the dignity of having it called anything else.

Ah, one of those joyous statements that simultaneously shows off the speaker's astuteness, and their political sliminess.

The position is both wrong and poisonous: its real tendency is to shut down argument, not win it, thus exempting anti-piracy rhetoric from intellectual review. It tends toward marketing rather than vetting the stance against piracy. It's the Michael Moore methodology: lying because telling the truth requires placing a bet on your audience's intelligence that you aren't willing to take.

But, I am falling in to the same trap myself. Is piracy stealing? The emotionally uninvested seem to be satisfied that the answer is "no" when presented with the simplest argument: theft deprives the owner of an object, piracy doesn't. It's an old refrain, and it's won all the converts it's going to win. There's really no sense in repeating it anymore.

But, we can go deeper. There are more powerful economic tools with which to discuss piracy, and we can apply them. Furthermore, we can apply them without having to make an argument against piracy. I will do so now, proceeding on the assumption that "intellectual property" is as valid a kind of property as any other, and the property rights thereto deserve just as much respect.

We can build an accurate picture of piracy around the concept of euvoluntary exchange. Euvoluntary exchange, which I may have mentioned in passing before, roughly etymologically translates to "truly voluntary exchange," which is an exchange in which absolutely every party concerned is a truly voluntary participant. While I won't go too far into the ideas around euvoluntary exchange (go ahead and click the link if you like), the theory that generally accompanies it is that euvoluntary exchange is always just. I'll leave proving that to the professionals (again, go ahead and click the link).

The criteria for proving an exchange is euvoluntary is simple (taken from the link above):

  1. Conventional ownership.
  2. Conventional capacity to buy/sell.
  3. Absence of regret.
  4. No uncompensated externalities.
  5. Neither party coerced by human agency.
  6. Neither party coerced by circumstance; the disparity in BATNAs is not "too large"

One and two are fairly intuitive. Three is a bit strange: we're using the decision-theory definition of regret, which is more precise than the intuitive one, and generally means that the thing we got wasn't as good as we had assumed when we made our decision. Four is going to be important, and means that if anyone other than the two people engaging in the exchange was somehow made worse off by it, they must be compensated (this is why slave trading is not euvoluntary; the slave doesn't really get much out of it). Five is obvious again; it's only euvoluntary if nobody held anyone at gunpoint. Six is probably the most complicated to explain, but we won't have much need of it here, so I'll simply leave it off.1

If euvoluntary exchange is always just, then for piracy to be wrong it has to break one of the rules above. However, for piracy to be stealing, it has to break the same rule that stealing breaks, in the same way. This is where the argument that piracy is stealing breaks down, because, as I will show, stealing is a coerced transaction and piracy is a transaction with an uncompensated externality. It's a rule five violation versus a rule four violation.

Stealing as a coercive transaction is simplest seen in a gunpoint robbery: you're being forced to "sell" your goods for no compensation. It's a bit fuzzier if we look at a cat burglar or pickpocket, who simply takes your possessions while you're unaware or sleeping, but we can stretch our definition of coercion to cover these without stretching it too far: you're being deprived of your right to evaluate the transaction and make a personal decision. If we take the middle case of a person drugging you, and then politely asking for you wallet, the connection is perhaps more intuitive. We could also establish coercion as "the opposite of consent," and borrow our definition from sexual assault law: you can't consent while drunk, you can't consent while asleep, and you can't consent while unaware of the other person's presence. You are coerced in all of these cases.

Piracy is, of course, more fickle to pin down. Let's analyze each stage of the transaction. We'll use the case of a physical DVD (season three of Teletubbies perhaps) as our example. Finding any oddities that emerge when we apply our model to pure digital distribution can be left as an exercise for the reader.

First, the purchase. We enter our local gadget warehouse, and, drawn to the peculiar foam-rubber gravitas of Tinkiewinkie &co, we purchase and take home our copy of Teletubbies Season Three. It's the most typical economic transaction in our whole scenario, and yet thanks to the mess that is IP debate, we're already in some euvoluntary trouble.

The question that arises is this: do we own the DVD? Some people on the freedom of information side of the argument have argued no; since we are unable to make copies or use the DVD as we see fit, we don't actually have conventional ownership of the DVD. This seems to violate rule two, since the store was not really able to "sell" us the DVD, in that they could not actually establish our ownership of it.

This is a bit of a mess, and fortunately it's specious. Restrictions on use of objects hasn't historically been considered to compromise our ownership of them. Outlawing murder doesn't lessen your ownership of a gun, or a knife. Noise ordinances don't lessen your ownership of your stereo or ridiculous car. Ownership and freedom of use are separate concerns. It's possible that we'd want to re-evaluate this in the light of specific extensions of copyright law, but we're interested in the general case today, and at any rate I don't foresee a lack of conventional ownership or capacity to buy and sell as frequently being the simplest way to discuss a clash between IP and euvoluntary justice, even if it does become applicable.

Now let's move to phase two of our transaction: for presumably archival purposes, you burn a copy of your Teletubbies DVD. While it may have changed, U.S. IP law did at one time recognize your right to make your own copies. Euvoluntary exchange isn't going to easily talk about this action anyway (nothing is exchanged, euvoluntarily or otherwise), so I think we can safely call this fair.

Finally, the act itself: you give one of the DVDs to your friend, the weary parent of a wayward toddler who's young life is crying out for some fluffy, primary-colored guidance. Let's run down the list:

  • We established at the original sale that you now have conventional ownership of the original DVD. Giving either one to your friend constitutes piracy, so we can assume you give her the original, which we've already established you own.
  • We get sticky with two again, since it's arguable that the effect of IP law is to interfere with your capacity to sell. However, we're interested here in the moral structure implied by IP law, not the law itself, so we'll allow that you're as able to "sell" the DVD as anyone.
  • Rule three is easy: you don't regret anything because you still have a copy of the DVD. Your friend doesn't regret anything because nobody regrets bringing the Teletubbies into their life.
  • Let's skip rule four for a second.
  • Rule five seems to check out. You willingly gave the DVD to a willing recipient.
  • Since I haven't defined it, I'll ask you to trust me that rule six is probably not violated in the general case.

So the exchange is very nearly euvoluntary. Moreover it is not coercive, and thus not stealing.

But, we left out rule four, for a very good reason. The argument against piracy states that by making this exchange, you deprive the original creator of potential money. The two participants are both happy and justly served, but a third party is worse off. That, if we believe it, is an uncompensated externality.

Intellectual property, which I stated before I will treat as a valid form of property for this exercise, is not stolen by piracy. It is only damaged by it. The value of intellectual property is tied to the value of selling copies, and that value goes down when you soak up the market with pirated copies. I've likened piracy to vandalism rather than stealing before, in an effort to placate some opponents. It hasn't taken, and in our new euvoluntary model I'm not sure if it's as powerful as it seemed, but it's certainly a better fit than theft.

And so we finally arrive at the real stickiness of the IP debate. The trouble for us now is that we haven't sufficiently defined externalities, and thus a clever person might make them appear everywhere. If a racist couple lives in a suburb, and a black man buys a house next to them, does his transaction with the realtor contain an uncompensated externality? If a new business opens up and competes with other similar services in the area, do all of that businesses transactions produce uncompensated externalites by reducing the market share of the incumbents? Silliness to be sure, and also possibly my fault, since I haven't even begun to dig in to prior work in defining what externalities are admissible under what circumstances. Will a firmer model of externalities squash all subjectivity, or will the debate rage on? And if our understanding of externality does become complete, will it vet IP law or crush it?

We seem to have just now truly begun to think about piracy, and we are able to do so only by discarding a three-word argument that was designed to end the discussion entirely. This is the frustration of meta-debate: we have to defeat our opponents just to get them to begin the actual conversation with us.


  1. We could technically just use the economic definition of "voluntary exchange," which leaves off item six. I stick with euvoluntary because voluntary exchange doesn't line up with our colloquial understanding of the word "voluntary," which tends to include only item five and nothing else (for example, "voluntary manslaughter" isn't voluntary in an economic sense). With euvoluntary exchange, you get one new word and one new concept, and there's less internal confusion. 

Read and Post Comments

« Previous Page -- Next Page »