Dave's Data

Over the past year, I've spent some time working on improving the robustness of TensorFlow.  As I mentioned earlier, one of my goals for my time at Google was to dive into industry best-practices for writing good code.  At Google, writing good code starts with careful programmers, requires good tests that get run on a fantastic internal testing infrastructure, is improved through code review, and makes use of several code quality tools and linters.

One part of that testing that's been gaining more visibility recently is fuzzing - throwing random inputs at programs or libraries to try to cause them to crash.  John Regehr has been fuzzing compilers for a while now - very effectively.  (That link has a nice taxonomy of the types of fuzzers.)  Google's Project Zero has been fuzzing the FreeType library for the last 4 years, and has found a tremendous number of security vulnerabilities in all sorts of programs.   (This isn't to suggest fuzzing is new - it's been used f…

Daniel Lemire and Maxime Chevalier just had a great exchange on Twitter about the state of compilers and being able to automatically vectorize code, such as a scalar product.  Of course, hoping the compiler can correctly vectorize your code can be a bit fragile, and as Maxime points out, writing raw intrinsics results in an unreadable pain in the editor, and a GPU-style implementation in CUDA or OpenCL might be more scalable and maintainable.

A few years ago, some folks at Intel wrote a compiler called ISPC, the Intel SPMD Program Compiler.  A possibly unfairly simple way to describe ISPC is that it's OpenCL-style programming for x86 vector units.  You can write code that looks like this:
export uniform float scalar_product(uniform float a[],                                     uniform float b[],                                     uniform int count) {     float a_times_b = 0.0;     foreach (i = 0 ... count) {         a_times_b += a[i] * b[i];     }     return reduce_add(a_times_b); }
Afte…

As you may have guessed from my recent post analyzing the TCO of my former automobile, and my post on finances for CS Ph.D. students, I've been thinking about finance a bit lately.  After a thought-provoking discussion with a senior colleague in another department, I've come to the conclusion my financial satisfaction graph looked something like this -- and I bet it's similar for many other no-kids-at-completion academics who end up enfamilied (can I make up that word?).

(In case I seem too negative about kids, don't get me wrong.  As the absolutely awesome book All Joy and No Fun notes, being honest about the myriad costs of having a child isn't at odds with also being very glad that the little wriggling worm is in your life.  Great book.  If you haven't read it and you do or may have kids, read it.)

The Taulbee survey suggests that the median assistant professor in Computer Science has a 9mo salary of about $96,055.  Assuming you pay your summers, that's …

Almost two years ago, I posted on Google+ that I'd purchased an Amazon Echo.  In that post, I wrote: It's surprisingly cool.  But the best part is what it makes me want, because it also sucks.  It can't recognize my 2.6 y/o daughter's voice, for example -- even though she can say to it fairly clearly "Alexa, play Puff the Magic Dragon"  (it will, if I ask it).  What an awesome enabler for kids, if it worked.  A little dangerous, too, but hey. :-)  It can't turn my remote-enabled lights on or off for me.  It can't even send me an email to remind me about something.  Boo.But - these are all current limitations.  Its speech recognition, albeit within a slightly narrow domain, is really solid.  It's happy with me, it's happy with my wife, and it's happy listening to us with the microwave running and a toddler running around.  The convenience is awesome.  I suddenly want to control more of my life by talking to it. But I can't. Yet.It'…

In discussions on Hacker News, I repeatedly see comments about "nobody ever implements sorting algorithms" -- or, generalized, that it's a waste of time studying up on basic data structures and algorithms, because they're all already implemented.

While it's rare, it's not been my experience to never re-implement the basics.  I decided to data-mine my and my research group's past and current projects to identify the number of places we've implemented elementary data structures for good reasons.
Sorting In the Cuckoo Filter, we implement a custom sort to sort the four items that can go in a bucket.  Because this is a high-performance context, and the size of the set to be sorted is fixed and small, it's much faster to do this inline.  This sort only cares about the least-significant byte of the key.

    inline void SortPair(uint32_t& a, uint32_t& b) {          if ((a & 0x0f) > (b & 0x0f)) {              std::swap(a, b);          }     …

This post is based upon a few recent conversations I've had with my own Ph.D. students.  Its intended audience is Ph.D. students at mid-to-upper-end computer science programs in the United States, who are either US citizens/permanent residents, or plan to remain and retire in the U.S.

Welcome to graduate school in computer science, where not only do we not charge you tuition, but we shower you with so much money that you can afford to eat, have a house over your head, and wear shirts that have fewer than five holes in them!

In fact, you can probably do better than that, and get a nice boost to being financially independent.  But it takes some advance planning.

Why Bother?
Mostly, because flexibility -- or, more crassly, "FU Money."  (You can google that in case the meaning isn't obvious.)  Getting started early on the path to financial independence lets you be in charge of your life.  You may also discover halfway through your program that there's this crazy nonp…

A recent Wired article breathlessly predicted the end of code:
Soon We Won’t Program Computers. We’ll Train Them Like Dogs Of course, this is the same magazine that declared in 2010 that The Web is Dead.  So perhaps we should step back and think before throwing in the towel.  Have you looked at a self-driving car recently?

In this simplified diagram blatantly stolen from Google, there's a laser scanner, a radar, a compass, speed sensors - missing are the cameras, the engine computer, the onboard computers, the cellular uplinks, the data recorders, the
onboard entertainment system (which will hopefully get even more use when the driver gets to play also).  And the backup systems, which are often redundantly engineered and even separately programmed to avoid coordinated failure.  Each of these devices has substantial embedded firmware controlling them, and per-device processing in order to control them and make sense of the data they generate.

Oh, and that car depends on Google'…