It's Time to End the War On Stupid People

| categories: politics, tech | View Comments

I never knew Sarah Sharp personally, but I was aware of her.

On one hand, it's hard to say how; I've never had particular interest in the USB stack, or much at all outside of the core kernel, and even there I've never managed to actually contribute beyond a couple of cosmetic patches.

On the other hand, it's little mystery at all: Sarah was the first female kernel contributor I'd ever heard of, and the only one I can readily name now. It's an uncomfortable answer, because when someone breaks into a space that doesn't often include their gender or background, we feel we avoid culpability by being nonchalant. No exclusion here, nosiree. Didn't even notice you were a woman. It's comforting and dishonest; when someone breaks a boundary of cultural exclusion, regardless of how your reaction may later be judged, the fact is you notice.

Although apparently nobody noticed when Sarah quietly disappeared over the past year, finally coming out to cite now-familiar complaints about the toxic and hostile atmosphere on LKML and in the kernel community in general.

In an ignorant youth that I'd rather was further away than it is, I remember liking the atmosphere on LKML. Mentioning being in the kernel community seemed to have the same effect on the right kind of crowd as saying you were from Brooklyn did in the early 90s. The threads of LKML were mean streets that would chew you up and spit you out if you weren't tough enough to handle it. But I was, of course. I was tough, you betcha, and only real tough guys have what it takes to stand up for code the way it should be written.

It's not hard to see the appeal, for a certain sort of ego, but there's something deeper. "Nerd" communities gravitate toward this form again and again. Other open source forums, video game communities, 4chan, the marks are similar no matter how distant the topics get. And why? What is it about geeky types that produces this sort of structure?

What is the war on stupid people?

Being cursed with doting parents, I got called a "genius" from a very young age, well on in to my adult years. I had my share of unusual feats, and I liked computers which were proper "genius" things to like, and at some point I took an IQ test and a very impressive number was produced. "Genius" it was then.

Of course, the older I get, the less I'm convinced there really is any such thing. Getting anything meaningful out of as gloriously defective a thing as a human brain, no matter what condition it may be in, is a skill and an art, not a product of ordainment into a pantheon of great thinkers. So I stopped thinking of myself as a genius.

But before I did, I remember thinking everyone else was stupid.

Perhaps it's too much praise, or some failure to notice the dissonance in thinking you're better than everyone else and then also expecting them all to keep up, but I'm hardly the only sufferer of the condition. To a certain sort of mind, stupidity is the chief moral and cultural demon in the modern world. When you have the answer to everything, it suddenly becomes clear that the only thing between the world and paradise is all of those people who just won't fucking listen no matter how many times I explain the most obvious thing God damn them!

It is human nature to define our enemies, and soon the abstract and ill-defined idea of "stupid" begins to take on concrete and fantastically fictional features. It becomes intentional; every futile political debate, every badly indented piece of forgotten code, every near-mishap on the freeway, all engineered by some malevolent demographic of those who wish to fuck everything up for everyone.

And so protectorates are established. Bastions to hold back the encroaching stupid masses. The mean streets of LKML, where yes you will be lambasted and degraded and torn apart daily, but you're ok with it because you're tough. You're a real tough guy who can take this sort of thing because you're not one of them. It is all necessary to protect you from them. And if you can't, well maybe you're one of them after all and you should leave, traitor.

As the airport screenings protect you from Muslims and the mass incarcerations protect you from drugs, so this sort of personal violation protects you from idiocy.

I'm prone to hyperbole, on this blog especially, but I have also witnessed a contributor in #fedora openly proclaim that someone seeking help in the channel was being deliberately incompetent for the purpose of causing him frustration. On multiple separate occasions. If one ever wanted to destroy the field of economics entirely, one need only come up with a single example of a person dedicating time and effort to such a perfectly and absolutely unproductive task. What can we call the belief that such people exist and are out to get you other than psychotic paranoia?

And so, I ask, what is it to be stupid, really? Are some slower to process information or discover novel solutions to problems? Certainly, and some run less quickly than others. If we leave behind our childish desperation to invert the high school hierarchy, I can scarcely find much reason to value one much over the other. A person who is "stupid" in this way does me no harm whatsoever. Is irrationality and succumbing to bias a better definition? Doubtful. These are real and dangerous problems, but irrationality is not so intrinsic as we claim of "stupidity." Rationality is a procedure, the "skill and art" I mentioned earlier. It can and has been described as a step-by-step process, and it requires only minimal mental power to execute. What it requires a great deal more of is self-discipline. I hardly think the behaviors we describe here demonstrate much of that.

So let's bring an end to it. Call of the war on stupid people. Respect others as a matter of policy. I promise you the kernel will be fine. All of us will be. Most of us better.

Read and Post Comments

libason 0.1

| categories: tech, libason | View Comments

I spoke quite a bit about ASON itself, what it is, how it might be useful, and its birth implementation, in the last release, so I'd check that out first.

I've waited much longer for a new release than I would have, but the result is a much more stable product. I'm calling the API stable-ish, and the soname will increase from here on out.

The API itself has been rearranged quite a bit. I'm not sure who checked out the last release, but everything has been condensed, and it feels much tighter than before.

More importantly, the algorithm for reducing and matching is now much, much more complete. We have heavy test coverage, and until someone points out a surprise to me, I can say pretty confidently that we can correctly handle anything ASON can express.

For the nerds, the spec has a new draft. It also has a version stamp now, so we can make drafts without creating confusion.

Most of the persistence/"database-y" code has been removed from this release. I hated all that stuff the moment I wrote it, but when it comes time to work on it again, I'll probably revert the removal and go straight from there.

With everything stable here, I intend to turn my full attention toward building a python bridge. I know that's probably where most of the early users are going to be, so it'll be top priority.

Go grab the release on github and file your bugs!

Read and Post Comments

Side project update

| categories: luftballons, tech, libason | View Comments

Consider this the obligatory "sorry I don't post more" post.

I have a couple of econ/political rants brewing, and I'll get to one of those in a minute, but for today let's talk about software.

Luftballons

Making a game engine is hard.

I knew this, of course, and I planned for it. There's a reason that despite being little more than a messy OpenGL demo, Luftballons arrived on the net with a website, a mailing list, an IRC channel (#luftengine on Freenode), and a big list of lofty goals. I never thought I could do it alone, and, not being terribly sure how to promote a project that wasn't going to have any usable form for quite some time, I just put as much community infrastructure in place as I was able to and made some noise.

I have taken a single patch.

The internet is littered with the remains of game engines that never took off. If this just ends up being an ever-unfinished personal tinkering project, I've hit par for the course.

That said, a couple of personal friends have shown interest in poking around in the code lately, and I worked up the energy to dust the code off and look it over.

It no longer worked.

It seems one of the libraries I used to parse a certain thing I hate not only managed to change APIs in the gap, but introduce a fatal bug in the process. Bug filed. I've exercised some neighboring open source code, the project has done good in the world. Plus the experiment the developer had me try seems to work, so the demo works again on my laptop at least. Hopefully there will be a consumable patch soon.

libason

Databases? New forms of algebra? Also hard.

Since release, the biggest thing that's shown up in libason is a disk B-tree file format which was intended to be the beginning of persistent storage. I'm a big believer in the "write one to throw away" software approach, and I will probably throw most of this away. I still don't know what I was thinking. I'm sorry. It's all wrong.

I'm still finding bugs in how we reduce values to simplest form too. That code is due for some major renovation soon. It's way too intricate for steady maintenance.

And that's it. See you in six months for my next apology.

Read and Post Comments

Complete Idiot's Guide to Bitcoin

| categories: tech, bitcoin | View Comments

Bitcoin is trickling into the mainstream, and the mainstream response is that they don't understand a lick of it.

The tech crowd perennially runs in to this trouble of explaining what the hell they just invented and why people want it. In industry settings there's usually a marketing department to help along. With more grass roots, open-sourcey movements there's often no such luck.

I think the problem, at least in Bitcoin's case, comes from three places:

  1. We can't seem to explain Bitcoin without explaining distributed computing, cryptography, or some other such complicated thing that the listener doesn't need to hear about.
  2. If we manage to get past the technical stuff, we somehow end up in some mind-bending philosophical macro-economics discussion. Turns out "what is currency?" has a very strange answer for a lot of people.
  3. When we do manage to clear out the theoretical cruft, we feel like the result is insulting to the listener's intelligence.

I've been thinking a lot on how I'd explain Bitcoin to the uninitiated, and I think I've figured out where to gloss over to take care of 1 and 2, and what calculated risks I'm willing to take on 3.

Disclaimer: Like I said, I'm glossing over. Hard. Really hard. For the technically inclined folks reading this, I'm going to come forth right now and say that there are some little things below that are just flat out incorrect. I know it feels strange; you're feeling point 3 right now, but to all of your corrections, I say: SHHHH! We're keeping this simple. Some facts are going to go missing.

So without further ado, here's my explanation of Bitcoin:

Let's start by talking about your debit card. Most of you probably have one, and have used it for some sort of over-the-counter purchase. As an example, we'll say your local coffee shop.

When you walk up to the counter and order your latte, the barista is standing in front of a computer connected to the internet. It may look like a little console with a keypad on it, it may be a normal desktop like you have at home with a big drawer underneath, or if you're at a newer, trendier coffee shop, it may well be an iPad with a small white square doohickey sticking out the side. All of these things are computers, and all of them need to be connected to the internet to do what they're about to do.

Now you hand over your card. The card gives the computer some information:

  1. Your name and bank account number.
  2. Where on the internet to contact your bank.
  3. Some security stuff to prove that this really is your card.

The little computer then connects to a big computer at your bank and says "Please transfer $4 from Pat Clemson's account to The Feisty Bean's account. Here's that security stuff to prove we have his debit card, and some other security stuff to prove that we really are The Feisty Bean."

That big computer at the bank has a list of all of the bank accounts, and how much money is in each, so it finds your bank account on the list, subtracts $4 from your balance, then finds The Feisty Bean and adds $4 to its balance.

And that's it. The numbers on the list changed, money has moved. There's nothing else to it. If that feels insubstantial just try not to think about it too hard.

So Bitcoin, like so many amusing things, begins with crazy libertarians.

One day a crazy libertarian says: "I don't like banks. They're big and central and everyone depends on them and the government is involved and it's scary and I hate it. I want to keep my own bank account."

So he puts a little file on his computer that says "Rand Atlas Taggart has $300." Then he calls his bank and, after an awkward conversation, convinces them to simply obliterate his account record. Now his $300 is only recorded in one place: on his computer, in a file he calls his "wallet file."

After a bunch of calls to all his crazy libertarian friends, wherein he convinces them to do the same thing, Rand goes out for coffee.

Like all great coffee shops, The Feisty Bean is owned by an ex-hippie, who totally digs Rand's style on the whole challenging the bank establishment thing. With some computer help from Rand, he puts all of The Feisty Bean's assets into a little file on the cash register computer's hard drive. Now, when Rand buys a cup of coffee, he tells the cashier some information about his computer and a password, and the cash register logs into his computer at home, reduces the amount of money in his wallet file by $4, and increases the amount of money in the wallet file on the cash register by $4. All of Rand's libertarian friends buy their coffee the same way.

And by now you've noticed the big problem:

"What if Rand just changes his file to make himself a millionaire? Can't he just open it and change how much money is in his account?"

Yep. He could, couldn't he? All he has to do is open that file again and change it to say "Rand Atlas Taggart has $300,000,000" and he's supposedly a millionaire. Basically, The Feisty Bean only knows Rand has enough money for his coffee because Rand's computer says so. With the old debit card system, they knew he had the money because the bank said so, and the bank has all sorts of regulation and monitoring from the government to keep them honest (don't get Rand started about whether or not that's actually working. Whoo boy).

Enter Bitcoin.

Are you ready for this? Bitcoin is a magic trick that keeps the crazy libertarians from editing their own bank files. You can pretty much stop there. You understand the core of Bitcoin now.

The way it works is each of the crazy libertarians, and The Feisty Bean, all run a little program on their computer that makes it talk to all the other computers and make sure they aren't modifying their wallet files. It's sort of a neighborhood watch program. This is the biggest gloss-over of them all so far, and if it feels shaky, you'll just have to take the leap of faith. It works, and it works really, really well. If Rand modifies his wallet file, not only will The Feisty Bean's cash register know it, but it will know exactly how much fake money he added. The more suspicious my explanation seems, the more impressed you should be with Bitcoin; it really was quite a feat to make it work and make it as reliable as it is.

That's basically the technical portion of it, now for the economics. In our example, Rand stored dollars in his wallet file. If he'd been Japanese, you might imagine he'd use Yen, or if he were Italian his wallet file might reference Euros. In reality, Bitcoin wallet files always store Bitcoins, which are a new currency that is separate from any country. This comes to a niggle in our original problem: so you can't modify your wallet file to give yourself money, but what about when it's created. How do we know you had that conversation with your bank? The reality is that was a bit of a lie; Rand would never be able to talk to his bank into destroying the money in his account. And his wallet? Our Bitcoin magic trick makes a new wallet start at zero.

So how do you get Bitcoins? Two ways. One is to simply buy some. This is the obvious way. You give someone your American Dollars or whatever have you and they put Bitcoins in your wallet in return.

The second way is called mining. Mining is a way you can create new Bitcoins from scratch. If all wallets started at zero, there had to be a process to make some initial Bitcoins, and mining is it. Mining is very hard; it involves solving a very complicated math problem, which requires specialized computers and takes a long time. This is on purpose; obviously you don't want anybody to be able to create a bunch of money out of thin air. Making it very hard, but still possible, means there's a place for new Bitcoins to come from, but since it takes work, you won't do it unless you think it's worth the work. Since mining actually gets harder and harder over time, less and less people should find it worth it. At present it usually requires special custom computing devices, and some experts calculate that the electricity to power the computer costs more than the money you can make doing it. Note that since mining gets harder over time, we can assume that it used to be much easier. The founders of Bitcoin mined the first coins, and were able to do it much more quickly. While some of them are now very hard to find (paranoid libertarians) we do know that many of them became very rich once people started being willing to pay real money to get Bitcoins.

Now let's talk about the controversies. First there's the economic issue. As mentioned, Bitcoins are created by mining. Mining seems to happen at a pretty steady rate, and as it gets harder, it's expected to slow down and eventually stop. Even if it doesn't there's a limit on how many Bitcoins can exist ever. By contrast, the U.S. Dollar is created and destroyed when the Federal Reserve decides it should be, which they do based on all sorts of factors, usually to try to keep the economy working well. Economists are arguing a lot right now over whether Bitcoins will work right given the way mining works. So far there's a little evidence to prove them right: there's been two incidents where the price of a Bitcoin was cut in half over night.

Then there's the legal issues. Governments like being in charge of currencies; so much so that they usually make it illegal for anyone else to do it. The U.S. Constitution forbids people from using other currencies. So far Bitcoin hasn't been tested in court in this country, but there's some worry that the whole enterprise could be outlawed.

And that's it. Maybe I've clarified something, or maybe I've just thrown another entry onto the "explain Bitcoin to me" pile. Here's hoping.

Read and Post Comments

Introducing ASON, and announcing libason 0.0.1

| categories: tech, libason | View Comments

I never much liked SQL.

I don't think it's a terribly controversial opinion. Apart from being pretty much the only option in its space, SQL doesn't have much to recommend it. It has confusing pretensions to natural language, it expects us to nest its expressions despite being incredibly verbose, it can't seem to shake a gross affection for capital letters, every single implementation is piled high with custom features, missing features, and strange voodoo, and it's got almost an entire imperative language built in to it, despite seeing most of its use from inside of other imperative languages. I like relational calculus well enough (it's the sort of crunchy, smarty-pants, pretensions-to-high-mathematics kind of comp sci I've always been in to), but its usual "business implementation" has never gotten along with me.

At the same time, I've always liked table-less databases. Designing and maintaining a schema is always more work than you expect, so a "just dump your dictionaries here" approach has instant appeal for the lazy and impatient, not to mention being simpler to reconcile with most modern imperative programming languages. Migrate-on-read also seems more engineer-friendly than trying to maintain migrations. I even prefer the atomicity model of some of the newer NoSQL databases like CouchDB to transactions, though I've yet to find an ORM that exposes it well.

It was my dislike of SQL and my preference for table-less that led me to look for an alternative to SQLite. I wanted a light, file-backed data store that could store table-less data (preferably JSON) and index it. I found tokyocabinet early, but while the author is being vague about it, it seems to be obsolete. kyotocabinet is the author's replacement, written in C++, and packaged for Fedora by yours truly, but it drops the table-less storage for simpler key-value stores, and the code quality leaves a lot to be desired. I will probably drop the package soon if nobody else is interested.

Then I started thinking about rolling my own. It was a problem that dealt in a lot of areas I happen to find fun (data structures, parsing, getting information through the VM to the disk without using O_DIRECT like an idiot), and while almost certainly a yak shave, I kept running in to this same yak, and I'd never once liked its haircut.

So, since formalism is how I personally drum up that froth of false confidence that precedes the birth of all great open source software, I set about formalizing a data model. I wanted to lend a semantic to JSON, which has some popularity already as a data format for table-less databases, and also extend it so I could reflect queries. I wanted to be able to describe not just a JSON value, but a set of such values, and from that set a subset, or a revision of that set with an item updated or removed.

And in this way was born Algebraic Serialized Object Notation, and a lengthy, crunchy whitepaper explaining how it works. Intimidating, perhaps, but at some point in the process I must have hit on some luck, because while specifying ASON is complicated, using it is simple.

Here is an ASON object:

{ "foo": 6, "bar": 7, "baz": "Cookies 'n' Cream" }

Just like JSON. Here is a collection of three ASON objects:

{ "type": "Student", "name": "Joe Thompson", "GPA": 3.5 } |
{ "type": "Student", "name": "Jeff Foxworthy", "GPA": 3.7 } |
{ "type": "Student", "name": "Smythe Littlesmythe", "GPA": 3.5 }

Here is a query to find all students with a GPA of 3.5:

{ "type": "Student", "GPA": 3.5, * }

Here is how to apply that query to the records above:

(   { "type": "Student", "name": "Joe Thompson", "GPA": 3.5 } |
    { "type": "Student", "name": "Jeff Foxworthy", "GPA": 3.7 } |
    { "type": "Student", "name": "Smythe Littlesmythe", "GPA": 3.5 }
) & { "type": "Student", "GPA": 3.5, * }

And here's the results:

{ "type": "Student", "name": "Joe Thompson", "GPA": 3.5 } |
{ "type": "Student", "name": "Smythe Littlesmythe", "GPA": 3.5 }

ASON's extensions should be apparent: we've basically allowed JSON patterns to be expressed as well as single values. While my intent was to specify a query language for a table-less database, using ASON much like regex to validate JSON values as you would strings is an obvious application.

For the database case, I opt to use the conjunction operation to join entries in a table. For example, that set of student values above could literally be read as "Joe Thompson or Jeff Foxworthy or Smythe Littlesmythe" if we interpreted it as a pattern rather than as a data set. The "&" operator calculates "pattern intersections," which effectively returns our matches (intersecting patterns is possible for regular expressions as well, though few practical syntaxes implement this).

This is all well and good, but a cute formalization is little more than cute. Without an implementation to play with, there's not much use in it.

And thus the point of this post. I would like to announce the first early alpha release of libason, version 0.0.1. Download here. This is a more-or-less complete implementation of ASON as it is today, in the form of a very, very slow in-memory database (what do you want? it's an early alpha). It comes with a handy REPL program called asonq, which will let you play around with the syntax, and a full C API for parsing and manipulating ASON values (python bindings to come soon... I hope). I intend for this to grow an on-disk database module, thus fulfilling my needs, and possibly have a proper DB server built over it, much like tokyotyrant sat atop tokyocabinet. There's also some more syntax to specify, mostly around updating values.

For now, I have something roughly complete, and interesting. The API documentation is shipped as the manpage ason_values(3) and some others which should be linked by association. Hopefully I can persuade a few curious souls to play around with it. Drop your bugs at github.

Read and Post Comments

Next Page ยป