What is Big Data, and why you could care

January 8, 2011

That’s the title for my talk this coming Thursday¬† night (January 13th, 2011), at the first “Tech Talk” sponsored by Sierra Commons. Details are at http://sierracommons.org/2011/local-business/2108.

Erika Kosina has done a great job of setting this up, and I’m looking forward to meeting more of the local tech community.

Some of the things I’ll be touching on are:

  • Why big data is the cool new kid on the block, or why I wished I really knew statistics.
  • Crunching big data to filter, cluster, classify and recommend.
  • How big data + machine learning == more effective advertising, for better or worse.
  • The basics of Hadoop, an open source data processing system.

Hope to see you there!


Big Data talk for technical non-programmers

December 22, 2010

I’ve going to be giving a talk on big data for the newly formed Nevada County Tech Talk event – a monthly gathering at Sierra Commons.

Unfortunately most of the relevant content I’ve got is for Java programmers interested in using Hadoop. Things I could talk about, based on personal experience:

  • A 600M page web crawl using Bixo.
  • Using LibSVM to predict medications from problems.
  • Using Mahout’s kmeans clustering algorithm on pages referenced from tweets (the unfinished Fokii service).

I’m looking for relevant talks that I can borrow from, but I haven’t found much that’s targeted at the technically minded-but-not-a-programmer crowd.

Comments with pointers to useful talks/presentations would be great!


Local mines near Nevada City

August 25, 2009

While searching (in vain) for a cool new domain name, I stumbled upon the CaliforniaMaps.org web site. As you can see from the snapshot below, there’s lots of local places to go looking for empty mine shafts to fall down:

Nevada County Mines

Check out the interactive version here.

Slide + Snow = Simply Fun

January 4, 2009

Our friends Matt & Susan (plus family) came to visit Nevada City shortly after a rare “big snow” day.

Matt posted a video of the kids having fun at a local school.

Wish I knew how to embed Facebook videos in a WordPress blog posting.

Direct from Nevada City – Captains of Crush!

October 26, 2008

My friend Ron recently sent me an article from his Stanford magazine on Randall Strossen and the “Captains of Crush gripper” that his company (IronMind Enterprises) sells.

Captains of Crush Gripper

Captains of Crush Gripper

No, I’m not thinking of a new career in strength training – the tie-in is that Randall & IronMind are based in my small town of Nevada City.

The case of the curious power drain

October 4, 2008

I took my MacBook to the Nevada City 4th of July parade, to save some spots at Wisdom Cafe. This is your classic small-town event, with fire engines, jazzercisers and The German American Friendship Club in full lederhosen.

After about an hour of work, my battery life was down to less than an hour, instead of the 3-4 hours I was expecting. Hmm, maybe my battery is dying. But I can hear a faint humming noise, which means the hard disk is spinning continuously. And I had trouble putting my Mac to sleep before I left for the parade.

I fired up Activity Monitor, and saw that something called “ditto” was using up 95% of my CPU. Firing up the Terminal, I executed “ps auxwww | grep ditto” and got:

kenkrugler 15193  98.2  0.0    75624   1024   ??  Rs   Tue02PM 3942:21.74 /usr/bin/ditto -xk - /Users/kenkrugler/Desktop/.BAHO26HV

Executing “man ditto” in the Terminal tells me it’s used to “copy directory hierarchies, create and extract archives”. What’s odd is that it’s been running for 3942 minutes, or almost 3 days.

Then I remembered that a few days ago, I’d renamed a .jar file to .zip and tried to expand it, to show somebody the structure of a jar file. But the Archive Utility hung, or rather it refused to finish or quit. So I had to force-quit it.

Which apparently left the underlying “ditto” process running, and what looks like a fully expanded version of the .zip file in a hidden “.BAHO26HV” directory on my desktop. I restarted my Mac, deleted the directory, and everything returned to normal (less 3 hours of battery).

With friends like these…

September 14, 2008

Last Friday our friends Cindy & Schmed arranged a late afternoon surprise for us, since we’d just celebrated our 15th wedding anniversary. No, it wasn’t the solar powered heart rainbow maker with the genuine heart-shaped Swarovski crystal, even though I thought I’d dropped plenty of hints.

Instead, Jenna came home from school with a sealed envelope they’d left with her teacher. This directed us to a sundial Schmed had secretly constructed on our property.

At 4pm the shadow gave us the digit to use to construct a phone number, which we called for our next clue. Hmm, where can you find the most wheels in Nevada City that aren’t attached to cars? They’d cleaned up that illegal tire dump near Old Downieville Highway. Of course – the Tour of Nevada City bicycle store! After a fun hour of digging up buried hints, figuring out what a Pikachu evolves into (good thing Jenna was there) and reading “Go Dog Go” for clues…

...behind the fire hydrant... ...find the animal on page 12...

…we found ourselves at the final destination:

Banner Mountain is the highest point around Nevada City & Grass Valley, at 3899ft. During fire season the tower at the top is manned by volunteers.

Jenna & Schmed on the tower stairs

Jenna & Schmed on the tower stairs

It turns out Cindy & Schmed had been practicing “pair fire-spotting” for the past few months…

Cindy demonstrates good technique

Cindy demonstrates good technique

and they’d set up a lovely anniversary party for us at the top. At the end of their 6pm shift, Jenna got to sign off, which pretty much made her day.

Talk about raising the bar – I’ll have to start planning their next anniversary event soon, since it’s only 8 months away.

Optimal solar panel angle

March 16, 2008

Back in 2002, I was working on getting a PV solar panel system installed on my house.

My initial calculation for the pay-back period was 15 years, due to slightly less-than-optimal roof orientation and some shade from large cedar trees around the house. This was based on the “easy” approach of having a local company handle the entire project, including the rebate.

By paying for consulting to design the system, buying the equipment myself, hiring a contractor to install it, and dealing with the California state rebate program directly, I managed to get the break-even time down to about 7 years. Though at the expense of significant hassle and a few close calls, like the fact that my original inverter couldn’t deal with the (lower) voltage gain from the Kyocera panels I wound up buying.

Solar Panels

But back to the title of this post – while working on the design, I searched the net to see if somebody had a way of calculating the optimal angle for the panels on the roof. I found exactly what I was looking for here. Then I noticed that the author of this page is Charles Landau, somebody I had worked with briefly while consulting at Palm. And then I found out that he lives in Nevada City, about a mile away from my brother-in-law.

Now we’re both members of the Nevada City tech lunch group, so we get to talk about solar panels, environmental testing, and open source projects like his CapROS operating system.