TEDx Talk: Digital Death, Online Afterlife

Posted on August 22nd, 2011

Hello, my name is Aaron Uhrmacher, and I’ve always been interested in how we remember people. Today, I specifically want to look at how our digital tools are changing the way that we remember and memorialize people as well as some of these tools that are helping us to do that better. Let’s start by taking a look at how we live.

What’s astounding to me is that without even thinking about it, we’re creating a rich, online archive of our real lives. Now it can be debated as to the extent that our online sharing represents who we are, but the numbers are astounding. I took a look at the average user of several services just to represent an average output for a user on any of these services in a given year. If you multiply that over ten years, or twenty or thirty, you start to see that we are putting a lot of our lives online. And even though the motivation for all of this sharing may be to connect with people today, it’s becomes this valuable record that we can look back on.

There are a few sites out there that prompt you to remember moments from you own life. Here’s a tool that was created by the site Photojojo geared towards photo enthusiasts. When a user signs up, the Photojojo Time Capsule accesses the user’s Flickr account and twice a month sends emails to the user with images they took a year ago.

4square and 7 years ago is another brilliant tool built by fans that allows Foursquare users to receive emails every day with reminders about the places they checked in on the service a year previously. Both these tools are in place to help the user remember and reflect on moments in their own lives.

And while all these services are fun, in the end we’re just adding to this incredible amount of clutter to our digital inboxes. There’s no way to tell which social media updates are important, and it’s difficult to imagine how a family member or descendent might take all this data from multiple sites with differing data structures and organize it to create a meaningful representation of the individual. It will happen one day, but we’re not there yet.

This collective memory is a big part of our culture. We’ve always collected the papers and correspondence of famous writers, authors, statesmen and performers. The practice implies that by examining all of their “stuff,” we will gain a better insight into the life and times in which they lived. These individuals simply pass boxes of their papers and other memorabilia off to a museum or university staff who spend thousands of hours organizing and making sense of the collection to facilitate an easy way for researchers and other interested parties to access it. We don’t have the same tools yet for our social media. Yet.

My friend Lisa passed away a little over a year ago, and I still think about her often. Sometimes I’ll see someone that resembles her and think about her. Or sometimes someone will laugh in a way that reminds me of her. I think that’s pretty natural. But then there are sometimes that I think about her because technology won’t let me forget. For example, I have her listed three times in my iPhone because of the way that Facebook, GMail and the iPhone sync differently on my phone. And each entry has different pictures of her and information about her culled from these networks we both shared. But I can’t seem to bring myself to remove her name from my address book, so every couple of weeks I’ll skim past her name and the memory of her is triggered once more.

What’s interesting is that in the absence of formal services that address our digital afterlife, we leverage existing services in new ways.

Lisa’s Facebook page is one destination where people go and leave messages both for Lisa, her friends and her memory. And every year around her birthday, I get a little reminder from Facebook which can be both welcome and off-putting.

It seems that (rightly so) most major social networks are focused on their living customers and not those who have passed away. However, the digital afterlife service Entrustet estimates that over 375,000 US Facebook users will die this year. That’s approximately 3 per minute! So we can’t ignore this element of our social lives any longer.

As far as social networks go, Facebook offers users the opportunity to transform a user’s profile into a memorial page once provided with their death certificate. The same goes for LinkedIn. But what about the others? Well, the Terms of Service vary greatly by company, and it becomes more of a legal issue than anything else. With no iron, Google requires a death certificate and other documents mailed to them. Like a letter with a postal stamp! Or you can fax them. Flickr will delete the content. YouTube will pass on the content to next of kin (but they have to know about the sites in the first place).

All of this requires that those who survive us are knowledgeable enough about our online lives to preserve our content, and unfortunately that’s just not the case.

Luckily, the emerging digital preservation industry is more concerned with finding ways to pass on passwords and usernames. And it’s kind of a legal grey area right now. Companies like Legacy Locker, Entrustet and Data Inherit allow users to create a secure repository for usernames and passwords to send to beneficiaries in the result of your death.

Another service, Memolane, is a web aggregator of sorts. Once you grant the site access to your social media profiles, you can see all of your social media updates laid out together on a timeline.

I created a project, titled All My Life, last Fall as I started to think about my own digital media. This piece essentially pulls in all of my status updates from the 30 or so sites that I update regularly and creates a way non-linear way to browse them.

There are a few companies that are trying to formalize the personal archival process. StoryCorps is an oral history project that allows anyone to visit one of the non-profits booths or mobile units and record a 45-minute interview to be saved in the Library of Congress. Although the production and archival process is completely professional, it’s still just one story.

When it comes to archiving a life story to pass on to future generations, 1000memories is one of the start-ups that tries to provide an online destination for memories as well as community mourning. Users can visit the site and share photos, text, videos and all sorts of digital ephemera. And the combination of all this media contributed by the people that knew the memorialized one best provides a rich resource for future generations.

So now that we’ve looked at what’s going on in the middle, let’s take a quick glimpse at the fringe. We’ll start with the analog. This is from the New York based graphic designer Nick Felton. Each year, he creates a beautiful infographic representing incredibly detailed statistics about his behavior during the course of a year. Since his father passed away last year, the 2010 report was dedicated to him, and all of the data therein was culled from the postcards, letters, ticket stubs and other analog “stuff” he left behind. It was a lot of work, I’m sure, but the resulting book of a man’s life summed up by the ephemera he collected over his lifetime is tremendous.

What if what you wanted to leave behind was not the stuff that represented you, but an actual copy of yourself instead? You can. Here’s a service called Virtual Eternity, where you can train an avatar to answer questions as if it were, well, you. Rather than try and explain, I thought I’d give you a quick demo.

Before I conclude, I want to tell you about how all of this research has prompted me to consider my own mortality. I thought about all of this data I’m creating and how it might someday have to be sorted by my children or their children, and although I have no problem burdening them with my legacy, I thought there might be a better way. My project, Telestory, aims to fill in the gaps between services like 1000memories and StoryCorps. When a user signs up for the service, they can determine when they want to be called (for example, once a month, only on their birthday, or twice a year). At the appointed time, the user automatically receives a call from the service and they are prompted to answer a question from one of the categories they’ve selected. The stories are then saved in a time capsule format for 25 years before they become publicly available. In this way, I hope to build a format for preserving the stories that matter and passing them from one generation to the next.

So there we are. Benjamin Franklin once said that nothing in this world is certain besides death and taxes. I might disagree. While we can’t yet live forever, our voices now can. Thank you very much. *

Pop N Scream

Posted on July 29th, 2011

Built by Aaron Uhrmacher and Steve Aquillano

Synopsis

Pop N Scream is a fun circus like game where participants use their mobile phones to pop balloons.

Description

It’s loud! It’s fun! In Pop N Scream, users compete against each other by making sounds into their phones to keep their respective balloons from popping on the ceiling. As the players attempt to match the volume pattern generated by the computer, observers watch the balloons rise and fall in a virtual tug-of-war, until one player’s balloon finally POPS!

User Scenario

  • Two users step up and dial the respective numbers on the Pop N Scream display.
  • Once connected, each player can watch how his/her voice reacts to the light meter.
  • When the game starts, players try to match the volume of their voice to the middle light bar, which is generated by the computer.
  • The computer checks to see how many seconds each user has hit the correct volume level. Whoever is correct the most times in a given period will see their balloon move downward while their competitor’s balloon rises towards the ceiling.
  • The object of the game is to keep the balloon from rising too high and getting popped.

Implementation

  • Pop N Scream is built using java as well as proprietary interactive phone service software developed and graciously shared by MegaPhone Labs.

Exhibitions

Mad Lips

Posted on May 14th, 2011

I recreated the Mad Libs from my youth for the video age, which I titled Mad Lips.

Mad Lips works like this:

  • The application starts up and asks the user a question, such as “what do you do if your waitress brings the wrong order?”
  • The program then prompts you to answer the question and records your reply.
  • Once your response is recorded, the program plays a famous movie scene and, at the appropriate time, substitutes your lips reciting your response to the question with a key actor during the crucial moment in the scene.

The final version looks something like this:

and like this:

Technically, here’s what’s happening:

The program uses the Minim library to record the audio and the native Processing MovieMaker library to record the video. I’m also calling the OpenCV library to calculate face detection and then approximating the location of the mouth using some basic math, which is the only part of the video that’s recorded. Once the audio and video are recorded, it uses the GStreamer library to playback the movie clip and then inserts the recently recorded video in the middle, substituting the user’s lips for the actor’s at a pivotal scene.

Here’s the Processing code:

// Comp Cameras Final Project
// Aaron Uhrmacher
// May 2011

import codeanticode.gsvideo.*;
import hypermedia.video.*;
import processing.video.*;
import java.awt.Rectangle;
import ddf.minim.*;

// AUDIO
Minim minim;
AudioInput in;
AudioRecorder recorder;
AudioPlayer player;

// VIDEO
GSMovie movie;
GSMovie movie2;
OpenCV opencv;
Rectangle bestRect = new Rectangle(0, 0, 0, 0);

// VARIABLES
boolean playBack = false;
boolean recordingLoaded = false;
PImage baby;
int state = 5;
PFont font;
int startingTime;
boolean movieNotStarted = true;

// recording movie: objects and variables
MovieMaker mm;
boolean readyToRecord = false;
boolean recordingStarted = false;
String title = "Baby";
int m, h;
int newFrame = 0;
int newFrame2 = 0;

void setup() {
  size(640, 352);
  baby =  loadImage("baby.png");
  opencv = new OpenCV( this );
  opencv.capture( 320, 240 );                   // open video stream
  opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );  // load detection description, here-> front face detection : "haarcascade_frontalface_alt.xml"

  font = loadFont("Apollo.vlw");
  textFont(font); 

  // VIDEO RECORDING VARIABLES
  m = minute();
  h = hour();
  println("Press 'E' to enable recording");

  // AUDIO
  minim = new Minim(this);
  in = minim.getLineIn(Minim.STEREO, 2048);
  recorder = minim.createRecorder(in, "data/myrecording.wav", true);

  movie = new GSMovie(this, "baby.mov");
  startingTime = millis() - 1000;
}

void movieEvent(GSMovie movie) {
  movie.read();
}

public void stop() {
  opencv.stop();
  in.close();
  if ( player != null ) {
    player.close();
  }
  minim.stop();
  super.stop();
}

void draw() {

  if (state == 0 && movieNotStarted) {
    movie.play();
    movieNotStarted = false;
  }

  else if (state == 5) {
    background (0);
    fill (255);
    text ("Mad Lips", 20, 50);
    text("What's your favorite", 20, 120);
    text("bumper sticker slogan?", 20, 150);
    text ("Press 'A' To Record Your Answer", 20, 250);
    if (key == 'a') {
      state = 1;
      println(state);
    }
  }

  else if (state == 0) {
    if (1 < movie.width && 1 < movie.height) {
      if (newFrame != movie.frame()) {

        if (movie.time() >=15.5) { // BEGIN MOVIE PLAYING
          movie.pause();
          movie.jump(457);
          playBack =true;
          state = 6;
          println("NEW STATE: " + state);
        }
      }
      image(movie, 0, 0, width, height);
    }
  }

  else if (state == 1) {
    if (!playBack) {
      opencv.read();
      // proceed detection
      Rectangle[] faces = opencv.detect( 1.2, 2, OpenCV.HAAR_DO_CANNY_PRUNING, 40, 40 );
      // display the image
      PImage frame = opencv.image(); // get the video
      if (faces.length > 0) {
        float lower_3 = (faces[0].height / 3) *2;
        float x_offset = (faces[0].width * 0.25);
        // the mouth PImage comes from the video, so it is essentially a mask
        PImage mouth = frame.get((int)(faces[0].x + x_offset), (int)(faces[0].y + lower_3), (int)(faces[0].width - x_offset), (int)(faces[0].height /3));
        mouth.resize(110, 47); // resize the mouth video
        image(mouth, 240, 205);  // place the mouth under the png image
      }

      image(baby, 0, 0);
    }
  }

  else if (state == 3 ) {
    if (movie.available()) {
      //movie.pause();
      // movie.jump(294);
      //movie.read();
      movie.play();
      println("playing movie");
    }
    image(movie, 0, 0, width, height);
  }

  if (readyToRecord == true) {
    mm.addFrame();
  }

  if (state == 6) {
    if (playBack) {
      if (!recordingLoaded) {
        //  println("loading movie");
        movie2 = new GSMovie(this, "Jack.mov");
        player = minim.loadFile("data/myrecording.wav", 2048);
        recordingLoaded = true;
        movie2.play();
        player.play();
      }
      if (1 < movie2.width && 1 < movie2.height) {
        if (newFrame2 != movie2.frame()) {
          // println("ready to playback");
          // movie2.play();
          image(movie2, 0, 0, width, height);
          if (!movie2.isPlaying()) {
            state = 3;
            println("NEW STATE: " + state);
          }
        }
      }
    }
  }
}

void keyPressed() {
  if (key == 's') {        // start or stop the record head when the ’s’ is pressed
    if ( recordingStarted == true ) {
      if ( readyToRecord == true ) {
        readyToRecord = false;
        recorder.endRecord();
        minim.stop();
        println( "Cut" );
      }

      else if ( readyToRecord == false ) {
        readyToRecord = true;
        recorder.beginRecord();
        println( "Rolling" );
      }
    }
    else {
      println( "Press 'E' to enable recording" );
    }
  }
  if ( key == 'e' ) {          // create new movie for saving
    if ( recordingStarted == false ) {
      recordingStarted = true;
      // compression and frame rate options
      mm = new MovieMaker( this, width, height, "data/"+title+".mov", 15, MovieMaker.ANIMATION, MovieMaker.HIGH );
      println();
      println( "Press 'S' to start and stop the record head" );
      println( "Press 'X' to save the movie and exit the sketch" );
    }
  }
  if (key == 'x') {                      // Finish the movie if the ‘x’ is pressed
    if (recordingStarted == true) {
      mm.finish();
      player = recorder.save();
      if (readyToRecord == true) {

        println("Cut");
      }
      println("Print!");
      state = 0;
    }
  }
  if (key == 'p') {
    playBack = true;
  }
}

Showetry

Posted on May 9th, 2011

Chris and I collaborated on our final presentation for Reading and Writing with Electronic Text, which we titled, “Showetry.”

There were two parts to the performance. For the first, we programmatically searched the movie “Revenge of the Nerds” using the subtitle text file to find any referenced to ‘nerd’ or ‘nerds.’ Then, using a combination of Popcorn.js and HTML5, we essentially created a real time clip reel of each instance and played it sequentially (video coming soon). Since the titled of the evening’s performance was “Hello Word,” we also changed the text from ‘nerd’ to ‘word’ in all of the subtitles as they appeared.

The second component of our performance were a series of poems that we read.

Our Python script generated these poems automatically by analyzing the subtitles of 50 YouTube videos based on the keywords that we provided.

When it found the word or phrase, it returned the line in the closed captioning from where it was found. What I love about these programs, in addition to the randomness achieved through pulling the same words from videos in a range of contexts, is the imperfections of the transcriptions themselves. Here are the texts of the two poems that I read:

I NEED

I need you to survive.
And I’m like, “Oh UGH! Why did I need coffee now?”
That way to I can actually go
ahead and carve out that area that I need.
Since I need this to kind of come up in an angle
I’m going to start at an angle.
I’m fall and then I’m going to see where I need
to adjust my line. I’m not going to use any.
I realized I had a lot of very difficult memories
that I need to let go of.
Is what I need to do right now while listening
to good music. You don’t even wanna see.
I love you, I need you, yeah.
I love you, I need you, oh, babe.
And I need you like a heart needs a beat.
That’s why I need to search for you.
I need a new car.
Charles Please marry me, Fiona.
I want you, I need you, I love you.
I need you.
I’ll lend an ear.
To find out what I need to repair.

======================

I THINK

I think in this economy these voters.
I think maybe those are the fastest guys we got.
WTF? I think I’m having an stroke,
Suffering a white guy overload.
Speed of light in a vacuum, and actually,
now that i think about it.
And I think maybe I should faint. But I don’t. NO.
And I think maybe I should faint. But I don’t. NO.
Making juice at home a pain.
Well I think that’s why the Easy Go Juicer…
I think Iraq will be a dominant
issue. It is the issue of our era.
Ron Paul: Well, I think the party has lost
its way, because the conservative wing…
Ron Paul: Well, the lower the taxes the better,
and I think cutting taxes would be beneficial.
No I think the…
Yeah I…
Yes, I think so.
Guys without the capital.
I think this guy in the dark so it’s pretty good
they came too,
Well kid that’s my analysis.
And I think that that’s…
I mean I-I think that that’s I think that
now that the show is over,
I think we can put our differences behind us…
Arsenio: I think you were his inspiration.
Ali: Do I think??
He…I think he likes it.
Hank, I thin that’s why, in the end, all
we can really do is be kind to each other.
Joe the plumber,
I think he’s really shiny.
I think this is an ignoramus statement.
Umm, I was even a person who thought…
I think we should grow
That’s…No, but I think
that’s exactly the problem.
I think that goes against
the idea of American exceptionalism.
Actually, Liz, I think you
wanna jump up to Robert.
yeah if I had you, ya, ya, ya you, ya, ya, ya,
you, ya, ya (I think he’s having a seizure)
I think I am going to leave now.
Wait, aren’t you going to try to
scheme?
I think a lot of people just hope these people will die.
I think you’re right. -But, you know, worst case scenario
It provides a very important
distraction.
I think talking about you don’t need God for
existence? I think it is too audacious a remark.
I think I’ll press the iBlend button.
Nah dude, I think this is clear enough.
It’s been really fun to watch, a this point,
I think I ought to clarify a few things. First
I think I have to puke again.

The audio of the performance is here:

Final Performance by aaronu