What is Big Data, and why you could care

January 8, 2011

That’s the title for my talk this coming Thursday¬† night (January 13th, 2011), at the first “Tech Talk” sponsored by Sierra Commons. Details are at http://sierracommons.org/2011/local-business/2108.

Erika Kosina has done a great job of setting this up, and I’m looking forward to meeting more of the local tech community.

Some of the things I’ll be touching on are:

  • Why big data is the cool new kid on the block, or why I wished I really knew statistics.
  • Crunching big data to filter, cluster, classify and recommend.
  • How big data + machine learning == more effective advertising, for better or worse.
  • The basics of Hadoop, an open source data processing system.

Hope to see you there!


Big Data talk for technical non-programmers

December 22, 2010

I’ve going to be giving a talk on big data for the newly formed Nevada County Tech Talk event – a monthly gathering at Sierra Commons.

Unfortunately most of the relevant content I’ve got is for Java programmers interested in using Hadoop. Things I could talk about, based on personal experience:

  • A 600M page web crawl using Bixo.
  • Using LibSVM to predict medications from problems.
  • Using Mahout’s kmeans clustering algorithm on pages referenced from tweets (the unfinished Fokii service).

I’m looking for relevant talks that I can borrow from, but I haven’t found much that’s targeted at the technically minded-but-not-a-programmer crowd.

Comments with pointers to useful talks/presentations would be great!


Local mines near Nevada City

August 25, 2009

While searching (in vain) for a cool new domain name, I stumbled upon the CaliforniaMaps.org web site. As you can see from the snapshot below, there’s lots of local places to go looking for empty mine shafts to fall down:

Nevada County Mines

Check out the interactive version here.

Slide + Snow = Simply Fun

January 4, 2009

Our friends Matt & Susan (plus family) came to visit Nevada City shortly after a rare “big snow” day.

Matt posted a video of the kids having fun at a local school.

Wish I knew how to embed Facebook videos in a WordPress blog posting.

Direct from Nevada City – Captains of Crush!

October 26, 2008

My friend Ron recently sent me an article from his Stanford magazine on Randall Strossen and the “Captains of Crush gripper” that his company (IronMind Enterprises) sells.

Captains of Crush Gripper

Captains of Crush Gripper

No, I’m not thinking of a new career in strength training – the tie-in is that Randall & IronMind are based in my small town of Nevada City.

The case of the curious power drain

October 4, 2008

I took my MacBook to the Nevada City 4th of July parade, to save some spots at Wisdom Cafe. This is your classic small-town event, with fire engines, jazzercisers and The German American Friendship Club in full lederhosen.

After about an hour of work, my battery life was down to less than an hour, instead of the 3-4 hours I was expecting. Hmm, maybe my battery is dying. But I can hear a faint humming noise, which means the hard disk is spinning continuously. And I had trouble putting my Mac to sleep before I left for the parade.

I fired up Activity Monitor, and saw that something called “ditto” was using up 95% of my CPU. Firing up the Terminal, I executed “ps auxwww | grep ditto” and got:

kenkrugler 15193  98.2  0.0    75624   1024   ??  Rs   Tue02PM 3942:21.74 /usr/bin/ditto -xk - /Users/kenkrugler/Desktop/.BAHO26HV

Executing “man ditto” in the Terminal tells me it’s used to “copy directory hierarchies, create and extract archives”. What’s odd is that it’s been running for 3942 minutes, or almost 3 days.

Then I remembered that a few days ago, I’d renamed a .jar file to .zip and tried to expand it, to show somebody the structure of a jar file. But the Archive Utility hung, or rather it refused to finish or quit. So I had to force-quit it.

Which apparently left the underlying “ditto” process running, and what looks like a fully expanded version of the .zip file in a hidden “.BAHO26HV” directory on my desktop. I restarted my Mac, deleted the directory, and everything returned to normal (less 3 hours of battery).

With friends like these…

September 14, 2008

Last Friday our friends Cindy & Schmed arranged a late afternoon surprise for us, since we’d just celebrated our 15th wedding anniversary. No, it wasn’t the solar powered heart rainbow maker with the genuine heart-shaped Swarovski crystal, even though I thought I’d dropped plenty of hints.

Instead, Jenna came home from school with a sealed envelope they’d left with her teacher. This directed us to a sundial Schmed had secretly constructed on our property.

At 4pm the shadow gave us the digit to use to construct a phone number, which we called for our next clue. Hmm, where can you find the most wheels in Nevada City that aren’t attached to cars? They’d cleaned up that illegal tire dump near Old Downieville Highway. Of course – the Tour of Nevada City bicycle store! After a fun hour of digging up buried hints, figuring out what a Pikachu evolves into (good thing Jenna was there) and reading “Go Dog Go” for clues…

...behind the fire hydrant... ...find the animal on page 12...

…we found ourselves at the final destination:

Banner Mountain is the highest point around Nevada City & Grass Valley, at 3899ft. During fire season the tower at the top is manned by volunteers.

Jenna & Schmed on the tower stairs

Jenna & Schmed on the tower stairs

It turns out Cindy & Schmed had been practicing “pair fire-spotting” for the past few months…

Cindy demonstrates good technique

Cindy demonstrates good technique

and they’d set up a lovely anniversary party for us at the top. At the end of their 6pm shift, Jenna got to sign off, which pretty much made her day.

Talk about raising the bar – I’ll have to start planning their next anniversary event soon, since it’s only 8 months away.