Software to extract meaning out of life's trivia

Here's a very interesting article in Washington Post, that ties in to some degree with Clive Thompson's article on ambient awareness (discussed in my previous blog post):

Bytes of Life: For Every Move, Mood and Bodily Function, There's a Web Site to Help You Keep Track

It reflects a lot of my own thoughts about what part data plays in our lives, and how it could let us get much more out of life.

"In San Diego, statistics student David Horn [...] is working with his engineer girlfriend, Lisa Brewster, to develop an all-encompassing life tracker, under the working title of "I Did Stuff."

I would like to have these guys' job. They want to track and record everything -- everything that happens in their lives, down to (or especially) the most mundane events.

It's been known for a long time, and a recent study confirmed, that keeping a diary recording every bite they ate helped people to lose weight. And therapists recommend people who have trouble sleeping to record what they ate, drank, and did before sleep, to see if a trend emerges that shows a correlation between certain foods / activities and insomnia. Also, keeping track of your time minute-by-minute -- writing down all activities, no matter how mundane -- may allow you to see where all your time goes, if you feel you have no time for anything in your life. So there is a well-established practical use for navel-gazing, that predates the internet. And the internet made it infinitely easier to record your daily events, both the kind you do consciously (Brightkite for tracking your location, MyMileMarker.com for driving habits, Fitday.com to map food intake and calorie expenditure, Last.fm for listening habits, and even BedPost for sex life), and the kind your body does autonomously (sites for tracking heart rate and blood glucose levels, or the self-explanatory MyMonthlyCycles.com :-))

But these two researchers want to take it much further.

Tracking not just what you did, but what you got out of it

[...] David Horn already belongs to BrightKite, Last.fm and Wakoopa.com, which tracks his Internet usage. He's also experimented with Fitday.com to map food intake and calorie expenditure. It was satisfying for a while, but now he wants something bigger -- something simultaneously broader and more nitpicky -- to fill in the gaps that individual sites don't currently track.

Horn is working with his engineer girlfriend, Lisa Brewster, to develop an all-encompassing life tracker, under the working title of "I Did Stuff."

"I'd like to track the people I talk to," says Brewster, "and how inspired I am six hours later. And definitely location history -- where I am, what time -- "

"Correlated with weather history," interjects Horn. "And allergy data, pollen and mold in the air."

Plus, "Web sites I read and their effect," says Brewster. "If I spend a long time reading a blog, like TechCrunch, but I don't get noticeable output from it."

At first the author of this article is boggled by this level of self-indulgent navel-gazing, but then she seems to understand what it is about. The usefulness of tracking is of course not in the raw data (who would have the time to re-read their life at the same pace as they are living it? :-)) but in extracting trends that would help you correlate perceptions with facts.

Has it really been a month since you last had sex, or does it just feel like that? Did you really floss five times last week, or was it more like twice? Now that you realize that, are you a little less angry at your dentist for that painful last appointment?

Analysis of mundane events reveals profound trends in one's life

Self-tracking [...] is partly about the recording, but also as much about the analysis that goes on after the recording.

The apparent meaninglessness of data recorded over time is actually what makes it profound.

The problem with diaries and blogs, trackers say, is that people use them to record the events they think are meaningful. What they forget is that meaningful events are often a result of months of insignificance, a cause and effect not readily visible to the human eye but easily detected with the help of a computer program.

"Things that happen over time can lead up to bigger events," says Horn. "They may seem small by themselves, but looking at them as a whole I can see how they lead to a bigger theme or idea."

"I was always a terrible self-journaler," says Messina. "Every once in a while I'd write in a journal, but it was always a major, momentous event. 'Got to college.' 'Broke up with girlfriend.' You lose a lot of the nuance that caused that situation to come about."

Tracking can "zoom out over my entire life," he says. It could, for example, help him better understand the aforementioned breakup. "When you've self-documented the course of an entire relationship, trivia that doesn't seem like much could, over time," help him understand exactly what went wrong, and when.

Maybe, to extrapolate on Messina's idea, your weekly date night had been Friday. And maybe you were always in a tetchy mood on Fridays because you'd just come from chem lab, which you hated. Maybe the whole relationship could have been saved by switching date night to Sunday, after your endorphin-boosting yoga class. Maybe you just didn't realize the pattern, because you weren't tracking it. All the answers could be right there, in your life data.

We can extrapolate even further. Perhaps the tracking software, if it was sophisticated enough, could notice increasing frequency and viciousness of arguments between you and your significant other, increasing frequency and length of time spent apart, and things like that. The software could flag it to you as a warning sign that the relationship is in danger. Then you could take steps to get it back on track. You might say most people don't need software to tell them when their relationship is off track; however, I think people often ignore warning signs -- sometimes wilfully, sometimes out of inertia. Inertia certainly plays a huge part in everything we do. We would rather keep a mental image of things as they were at their most comfortable, or downplay the significance of worrisome events, than acknowledge the truth that something is going astray. Life-tracking software could point out discrepancies between our partner's words and actions. It could force us to pay attention to those signs before it is too late.

The software could also give us tools to defuse certain recurring arguments which, if unexamined, tend to pick up destructive strength like a hurricane crossing the Gulf of Mexico. :-) You could look at the software and say: "we've had this discussion before; here is what was said; here is the conclusion we have reached. Do you have any new information that would give us a reason to revisit this issue?"

Of course, there are a lot of people -- most people, perhaps -- who would hate the idea of having their every word or phrase recorded, and of those records being resurrected as evidence (even by people they trust). I'm sure some people might think it diminishes their relationship somehow. But how could truth diminish it? Anyway, that's a social engineering problem, though those are often harder than computer engineering. Among the latter, a major problem would be to find a way to structure the data so as to capture its essential qualities. For example, how would you compute the intensity of the four horsemen of Apocalypse (made famous by John Gottman): Criticism, Contempt, Defensiveness and Stonewalling? How do you quantify formless, deeply subjective data? How do you even decide what to measure? It would be a tough task, but one I would gladly spend years working on, if I didn't have to worry about making a living. :-)

In fact, if I had come of age at the time of Web 2.0., I would seriously consider going to grad school so that I could do this project as my thesis / dissertation. I would probably find a professor somewhere in some university who could get interested in this idea enough to serve as my advisor. (I've seen people in computer science departments doing stranger projects than that. Or if not in computer science, then surely in the interdisciplinary studies. :-))

Bookmark This on Delicious

If real-life people cared as much as netizens

From Washington Post:

The Impassive Bystander: Someone Is Hurt, in Need of Compassion. Is It Human Instinct to Do Nothing?

A woman fell on the floor, convulsed and died half and hour later in a hospital's waiting room while the staff walked by, watched, and did nothing. (Fellow patients didn't call for help either.)

A 78-year-old man tries to cross a street with a carton of milk. To quote the article, "He steps off the curb just as two cars that appear to be racing swerve on the wrong side of the street. The first car swerves around the man. The second car hits him and throws him into the air like a doll, then speeds away. What follows is even more chilling: People walk by. Nine vehicles pass him lying in the street. Some drivers slow down to look but drive away."

The article questions how it could have happened, and whether we are actually wired for indifference.

If no one else is moving, individuals have a tendency to mimic the unmoving crowd. Although we might think otherwise, most of us would not have behaved much differently from the people we see in these recent videos, experts say. Deep inside, we are herd animals, conformists. We care deeply what other people are doing and what they think of us. The classic story of conformist behavior can be found in the 1964 case of Kitty Genovese, the 28-year-old bar manager who was slain by a man who raped and stabbed her for about half an hour as neighbors in a New York neighborhood looked on. No one opened a door for her. No one ran into the street to intervene.

We seem to be more caring on the internet

At the same time, most of us in the blogosphere have probably heard of cases of people rallying around a virtual friend who's having a real life emergency -- e.g. calling police upon suspicion that a fellow blogger is about to commit suicide -- even if they have never met that person before. It can make you wonder: can we engineer the real world to be more like the internet in that respect? Would we be more likely to help a stranger on the street if we knew he was just a few friends away from being connected to us on Facebook?

Can we engineer real life to be more like the net?

Among humans, negative example is apparently contagious. But perhaps our machines could show some positive example for us, if we programmed them to do so. Let's say, a victim's mobile device notifies his/her Facebook friends, Twitter followers, or other social networking contacts; they in turn could alert their friends on their mobile devices based on location; and so the bubble of alerts could propagates to those who happen to be relatively close to the victim at the moment. (In the 6-degrees-of-separation world this should not take long.) Then perhaps those people within the alert wave might be motivated to help, not the least because they know there is an "audit trail" making them accountable to their friends. (Of course, there are a few technical problems here, but I'm talking about the concept.)

So in a sense the users would be Big-Brother'ing each other. But at the same time social networking applications could prompt us to do the right thing by making it appear as if somebody is doing the right thing. :-)

An authentication scheme cooked up by a pointy-haired boss

There was this article in Washington Post that reminded me of a conversation I had with a friend via his blog about authentication questions, the kind you have to answer in order to login to online banking systems and such. In this post he pointed out that security questions based on users' wishes, preferences and hobbies were useless. (My, has it been almost a year? It feels like we had this conversation just yesterday.) Well, things have only gone downhill in the security world since then, if this article is any indication.

This Is Your Life*... *As Determined by Confounding Identity-Protection Safeguards

Old authentication systems, as imperfect as their questions were, at least allowed you to select a question and an answer that suited you best (or that was less useless to you than others). But how would you like an authentication system that does not even let you choose the questions? Instead, it asks you your biographical facts that you are "supposed" to know. For example, your great-grandmother's birthday. I wish it was a joke, but it isn't.

How does the authentication system know your great-grandmother's birthday in the first place? To quote the article, "unlike traditional shared knowledge authentications, in which the user picks the test and the answer and regurgitates it with each sign-on, Verid [the company that makes this uniquely egregious kind of authentication software -- E.] vacuums public records for factoids, then tosses them at the user at random."

The birthdays of long-gone relatives are not even the most obnoxious example of authentication questions. Others are, for example, "what was your high school mascot" or "the name of your homecoming queen". So what do you do if you come from a country like mine, where not only high schools don't have mascots or homecoming traditions, but the very concept of high school does not exist (all grades from from first to twelfth are taught under the same roof. You don't have to change schools when you transition from primary to secondary education.) Well, OK, a non-existent homecoming queen of a non-existent high school would not appear in public records, and would not serve as a basis for an authentication question. But if you dared to forget her majesty's name, you are screwed. :-)

The article does address the issue that some people's lives don't follow a typical middle class American route, thus some people don't have, or don't remember, the biographical facts enabling them to answer security questions. What the article does not address, is the invasion of privacy committed by a company that "vacuums up the public records" and collects all the knowledge about you, up to your great-grandparents' names and birthdays.

Only a pointy-haired boss could have come up with this kind of authentication scheme. I just hope that the company I work for -- which happens to write software for online banking -- never comes up with something like this.


The best quote from the article:

"Computers are like very dumb people, but they're very fast at being dumb," says Jason Hong, a professor at Carnegie Mellon's Human-Computer Interaction Institute (HCII).

eBay as a giant museum

Many words have been spilled on Web 2.0 and how it's supposed to change the world -- from Pulitzer prize-winning pundits to rank-and-file bloggers, everybody seems to have weighed in. Still I haven't read many observations as sharp or fresh as this one from William Gibson. This quote is from William Gibson's interview in Washington Post. Yes, this article is a couple of weeks old, but, as I mentioned before, I'm a leisurely blogger, not a professional one. :-)

"Google is the piece de resistance of weird [stuff] finding," he says. "One of the things I've been doing in the eBay era -- I've become a really keen observer of the rationalization of the world's attic. Every class of human artifact is being sorted and rationalized by this economically driven machine that constantly turns it over and brings it to a higher level of searchability. . . . The tentacles of that operation extend into every flea market and thrift shop and basement and attic in the world. . . .

"Every hair is being numbered -- eBay has every grain of sand. EBay is serving this very, very powerful function which nobody ever intended for it. EBay in the hands of humanity is sorting every last Dick Tracy wrist radio cereal premium sticker that ever existed. It's like some sort of vast unconscious curatorial movement. (emphasis mine -- E.)

"Every toy I had as a child that haunted me, I've been able to see on eBay. [...]

"This is new. People in really small towns can become world-class connoisseurs of something via eBay and Google. This didn't used to be possible. If you are sufficiently obsessive and diligent, you can be a little kid in some town in the backwoods of Tennessee and the world's premier info-monster about some tiny obscure area of stuff. That used to require a city. It no longer does."

The whole article can be found here.

I think I may lobby the FACT reading group coordinator to put Gibson's new book, "Spook Country" on the FACT group candidate list. Even though FACT group on the average didn't like Gibson's earlier novel, "Pattern Recognition", I am quite intrigued by Gibson's perspective.

Odds are stacked against myth debunkers

Here is an interesting article I've read recently. It has depressing implications for Center Of Inquiry and I guess everyone who would like to see the world thinking more rationally.

Persistence of Myths Could Alter Public Policy Approach

Some highlights from the article:

The conventional response to myths and urban legends is to counter bad information with accurate information. But the new psychological studies show that denials and clarifications, for all their intuitive appeal, can paradoxically contribute to the resiliency of popular myths.

[...]

The research also highlights the disturbing reality that once an idea has been implanted in people's minds, it can be difficult to dislodge. Denials inherently require repeating the bad information, which may be one reason they can paradoxically reinforce it.

Indeed, repetition seems to be a key culprit. Things that are repeated often become more accessible in memory, and one of the brain's subconscious rules of thumb is that easily recalled things are true.

But silently ignoring the lies isn't any better, the article says:

So is silence the best way to deal with myths? Unfortunately, the answer to that question also seems to be no.

Another recent study found that when accusations or assertions are met with silence, they are more likely to feel true, said Peter Kim, an organizational psychologist at the University of Southern California. He published his study in the Journal of Applied Psychology.

The article ends with this pessimistic conclusion:

Myth-busters, in other words, have the odds against them.

It really makes you wonder if humanity is doomed to live in perpetual ignorance (and I know a lot of people would answer with a resounding "yes"! ;-() How did people ever stop believing, for example, that whispering incantations or casting an evil glance on someone can cause a person to fall ill? Actually, it may be too optimistic to say people stopped believing it. :-) There are plenty of people who still do, even these days. But at least the majority doesn't... I would hope. Or, OK, at least the majority doesn't think it's a valid reason to accuse someone of casting spells and burn them at a stake. So how did the humankind ever moved beyond these superstitions? Well, this question runs central to a James Morrow novel "The Last Witchfinder". The novel is about one woman's lifelong quest to banish not just the trials and executions of witches, but the very notion of witchery. We discussed this book in the FACT book club, and it lead to some interesting philosophical debates. I will post the report of the discussion in the next few weeks.