Mike Hewner's Homepage

This is a post not really about what I did, but what 3 of the high-school students I worked with did. I met Gil and Claus in my automata class at GHP this summer. As part of the summer research project, they worked on developing their own kind of automata and ended up with something similar to a Neural Network. This fall, Gil, Claus, and Anirudh contacted me and proposed continuing that research but expanding the neural network connection for the Siemens Science Competition. Although I agreed to advise them where I could, this project was basically completely done start-to-finish by Gil, Claus, and Anirudh.

What eventually came out of this project was an attempt to build a Neural Network capable of optimizing the parameters of other neural networks. As the paper says:

The fundamental goal of this project is to create a backpropagation neural network that can determine the optimal training parameters to train another neural network with a different goal based on only characteristics of the training data. Such a neural network has the capability of improving the accuracy and speed of the training process for backpropagation networks used for any application.

In order to accomplish this Gil, Claus, and Anirudh researched neural networks, programmed their own network framework from scratch, and attempted to design a neural network that could help optimize other neural network parameters. Unfortunately the result they got was that the intermediate network wasn’t able to optimize properly. My opinion is their approach still has merit, and whether or not they get back to this particular project I hope they weren’t too disappointed with the negative result. This was a very cool project that required a great deal of both math and Computer Science to accomplish.

(Contains 2 attachments.)

I know, I know…I’m supposed to be revising important papers. But this is a cool distraction:

http://azarask.in/projects/algorithm-ink/#803de9d0

1 362892 hewn-Val-Hum-Fem-Law died in The Quest on level 19
[max 26]. Killed by a death ray. 119 [119]

Nethack is the ultimate is computational masochism. One second your Valkyrie is rocking through the Dungeons of Doom like a hurricane of Spolier-pumped pain, taking all comers with your painstakingly tricked out Mojinir. Then 1 random fire giant with a wand of death ends it all in an instant.

My UW class starts in June so if you want to party, earlier is slightly better than later.

So the always-wise Big Easy pointed me to this particular article:

http://unqualified-reservations.blogspot.com/2007/08/whats-wrong-with-cs-research.html

I’m no particular friend of the heavy-formalism crowd he’s poking fun at, but I think there is perhaps another explanation for the fact that a lot of research is not particularly useful.

I sort of think of research as a government funded place to produce really great ideas. Pretty great ideas like websites that let you build other websites, for example, are supposed to be accomplished by enterprising people in industry. Really great ideas might be something like a brain scanning device that let you build websites.

The bad news about these ideas is that they are probably impossible. But of course it’s difficult to be 100% sure until you build a brain scanner and hook it up to Microsoft FrontPage. Actually even then you wouldn’t be 100% sure because maybe you need a slightly different version of FrontPage, or a slightly different kind of brain scanner.

What this means what academics tend to seem to produce, from an external perspective, is bad ideas. If an idea has to be 99% unlikely to work before industry won’t touch it, that implies that 99/100 academic papers are going to be describing things that don’t work.

Of course, you can’t really say on a grant application that you’ve done 10 things and produced 10 uniformly uninteresting results. So some of your work is even more boring stuff where you study things, to attempt to reduce the risk of producing bad ideas. Like if you’re an AI guy studying natural language, you might painstaking analyse 100 lunchroom conversations for pronouns, just to see if the many potentially great language processing algorithms that can’t handle pronouns well might actually great idea. And then you might publish a paper who’s exciting finding is “pronoun usage of category X occur in less than 5% of conversations”. Woo! But most likely, you’ll find that people use category X 55% of the time, which is the same thing as saying that natural language processing is just as hard as everybody thought.

Researchers are always looking for huge wins, which means that even ideas that turn out to be pretty good are discarded. Like say you are curious about type-checking, so you build some better type checking for C. And it turns out to make C a little bit better. From an academic’s perspective, that’s pretty much the same as a failure. So it’s no wonder you don’t spend a lot of time trying to publicize this, even though this might actually be a helpful worthwhile idea. In some sense, this is not a mistake: your job is to search the idea space quickly, not capitalize on every OK idea you run across. So you go on and find 5 other type-checking things that are even less useful. Or maybe you develop some formalism of your first OK idea on the theory that the formalism might reveal some ground breaking abstraction you could use to reliably produce neat type-checking ideas.

So that’s how I think the system is supposed to work. Research produces craploads of bad ideas in the hopes of someday producing some good ideas. Now is this a good plan? I dunno.

One thing that is obvious is that it is a profoundly depressing thing for the people involved. The freedom to come up with your own ideas is very much tempered by the knowledge that with all likelihood that idea will be a failure (where failure is defined as a lack of insane success). And once you get the failure, you’ve got the academic damage control of spinning the failure to imply that it implies a huge success is literally around the next corner (and of course, your job depends on this lie, even though everyone realises that you have no idea if success is really coming).

Another good question is in a field where you don’t need fancy stuff like supercolliders, isn’t industry and hobbists sufficiently incentivized to do this idea search without the government kicking on money. Well, I suppose that likely depends on what you think the structure of idea-space looks like which is surely a difficult question to characterize with confidence.

No contracts have been signed or anything, but I just got an email from a guy at digipen saying that my name was going to be on the Operating Systems course this summer. So barring unexpected complications, I guess I’ll be around!

I got pointed to this post about how to learn programming. My favourite section was where he talked about programming languages:

Learn at least a half dozen programming languages. Include one language that supports class abstractions (like Java or C++), one that supports functional abstraction (like Lisp or ML), one that supports syntactic abstraction (like Lisp), one that supports declarative specifications (like Prolog or C++ templates), one that supports coroutines (like Icon or Scheme), and one that supports parallelism (like Sisal).

I’ve got some stuff to learn.

And by the way, I was referred to this by this essay by Eric S. Raymond. Which says a few neat things – but like most people I find that ESR just bugs the heck out of me half the time.

Should I want to get into a big argument at CIGCSE this year and cement my reputation as a frothing nerd code freak, I need only reference this blogpost which makes the provocative claim that Computer Science is about programming. I’m pretty sure most of the BuffaloBlog audience is inclined to agree with him – and myself too insofar as I think attempts to say “CS is not about programming” are essentially counterproductive (as the weeks go by, students are eventually going to notice they are writing an awful lot of C). But I want to just toss 2 counter-arguments out there for you to chew in the spirit of multiple perspectives:

1. As someone who works in a giant HCI building, I’m more familiar with HCI than maybe this guy is. Some HCI is definitely more coding-oriented than he’s thinking: building gloves that allow recognition/translation of sign language for example. And some is completely not: finding out how religious organizations conceive of and use technology. One important factor in HCI’s presence in CS departments he’s likely not considering – if HCI is under design, designers don’t usually talk to computer scientists very much.

   Anyways, just a few slightly pragmatic thoughts that suggest that maybe it isn’t unreasonable for there to be some parts of computer science that do not immediately involve programming. Is P/NP programming? Circuit design? As much as I really do like programming, I don’t see that it somehow needs to be the gold standard for what’s in Computer Science as a field.

2. Being a nerd is actually pretty cool in this century.

   Confession time: I am a big nerd. I know you’re all shocked. And I will freely admit than when I came to Georgia Tech, it wasn’t without some thought that maybe CS ought to return to its “roots” of being a haven for nerds like me and my friends. Nobody wants to code ’cause its nerdy? Who cares – we’ll just teach nerds and anyways nerds write better software. Nerds have their own little field. Society has awesome programs. Everybody is happy.

   What turned me around was maybe the . This book talked about CMU CS education back before they started to try and be more gender balanced – it was *really* nerdy back then.

   The conclusion I came to after reading this book is that in a lot of key ways, “nerd culture” is referring to a set of shared experiences that non-nerds really cannot go back and access. Yes, you can go watch The Princess Bride and Hackers. But you can’t watch it when you were 12. And you can’t retroactively be dateless at promtime or ostracized for you coding skills or install OS/2 on your 486 with 16 megs of RAM. And if we make being a computer programmer about these things, we’re basically talking about playing golf at the country club to a black student in 1960. We’re referencing a shared experience that some people won’t ever be able to be part of no matter how hard they work. And that seemed very inappropriate to me, so I’ve been a little down on the “CS is for nerds” angle ever since.

This started as post in response to the wise and talented Josh , but eventually got so unwieldy I split it off into this post. Josh’s post – and the post he was responding to – are worth reading but all the context you actually need to know is that somebody made the claim that the top 5% of programmers are more 20x more productive than the other 95%.

1. So as to this 5% business, I’ve actually heard a ton of various numbers along these lines. Now that I’m a honest-to-goodness academic who is supposed to know about this kind of stuff, I feel pretty confident in saying that these numbers are made up. Though if anybody has actual references to studies or something, I would really seriously appreciate if you’d post them up here because I could really use them.

But from an academic perspective, just having some semi-viable metric of evaluating 1 programmer against another basically doesn’t exist. There is someone in my lab who’s currently working on a test to compare CS1 students against one another, and if that works out it’ll be a Ph.D. for her. And if you think about how you would try and compare a CS1 student trained in Java to one trained in Scheme, it ain’t easy.

2. So whether it’s 10x or 100x, I agree that the numbers we hear kicking around are talking about a phenomenon that people really do see out there in the real world. And I wouldn’t say that it’s likely unique to programming – programming is just an obvious example because it has something to do with quantity as well as quality. That is, maybe the best writer can crank out a screenplay that’s 100x better than a average writer but it’d be pretty hard to know that. With a programmer you might be able to really write something in 3 days that would take someone else a year.

So I’m not disputing the 5% thing, except to say that 5% is just a catchy way somebody came up with to describe “some programmers are waaaaay better than others”.

3. The real million dollar question in my mind, Josh, is what are these super-programmers doing that others are not? Even if there’s just something irreproducible in their genes – shouldn’t this be the sort of thing we could detect somehow? Or if there are strategies involved, could we use them to make the 90 percentile 5x better than everybody else?

Because I think that all of us serious programmers, on our darker nights, think about those 5%ers and wonder if we are really making the cut. And we shake it off in the morning and say well “I got thus-and-such on my SATs, and I went to this school, and didn’t I get a 5 on my last performance review?” “Surely,” we think, “surely that must mean I’m on the list.” It’s a lot like Calvinism, when you think about it.

But I think those doubts afflict students most of all, and especially those who are not white male with a lifetime of hacking experience and tons of free time. I think it’s likely (though unproven) that once you decide you can’t make the 5%, you’re likely not going to put in the time and effort to get there even if it was possible.

Which of course brings me around to my research – say you had a group of the 50 cs students who enter an average program around the country. And say your goal was to get absolutely as many of them as possible into that 5% and screw everything else. What would you say to them? What would you do with them?

Comment, you slackers!

This post has no particular purpose…I just ended up writing a few paragraphs about being a grad student in an email and thought you might be amused:

Once the whole pirate/ninja internet meme plays itself, I think graduate student positioned to be the next sexy job of the decade. A graduate student is like ninjas and pirates all mixed together. We’re the hard-drinking hard-fighting lords of the night and we never sleep. We look snappy in longcoats, black long underwear and oversized hats, though it is difficult to tell because we are indeed masters of disguise. We know tons of secrets man was not meant to know, a fact which we will drukenly brag about when we have even the tiniest amount of rum. We sail the high seas of academe, seeking whores and honor and grant money (which for various grant accountancy laws must be buried on deserted islands).

The only thing that grad students really lack in an enemy. But technology will soon enter the stage when we can again build evil robotic nuns, the ancient sacred prey of the graduate student. How do I know this? Indisputable paleontological evidence proves that what we think of as “dinosaurs” were actually a race of evil robotic nuns created and subsequently destroyed by insect graduate students who started their degrees in the late-Paleozoic. And you can be sure that when the nuns come back they will itching for a rematch.