Hello, my name is Aaron Uhrmacher, and I’ve always been interested in how we remember people. Today, I specifically want to look at how our digital tools are changing the way that we remember and memorialize people as well as some of these tools that are helping us to do that better. Let’s start by taking a look at how we live.

What’s astounding to me is that without even thinking about it, we’re creating a rich, online archive of our real lives. Now it can be debated as to the extent that our online sharing represents who we are, but the numbers are astounding. I took a look at the average user of several services just to represent an average output for a user on any of these services in a given year. If you multiply that over ten years, or twenty or thirty, you start to see that we are putting a lot of our lives online. And even though the motivation for all of this sharing may be to connect with people today, it’s becomes this valuable record that we can look back on.

There are a few sites out there that prompt you to remember moments from you own life. Here’s a tool that was created by the site Photojojo geared towards photo enthusiasts. When a user signs up, the Photojojo Time Capsule accesses the user’s Flickr account and twice a month sends emails to the user with images they took a year ago.

4square and 7 years ago is another brilliant tool built by fans that allows Foursquare users to receive emails every day with reminders about the places they checked in on the service a year previously. Both these tools are in place to help the user remember and reflect on moments in their own lives.

And while all these services are fun, in the end we’re just adding to this incredible amount of clutter to our digital inboxes. There’s no way to tell which social media updates are important, and it’s difficult to imagine how a family member or descendent might take all this data from multiple sites with differing data structures and organize it to create a meaningful representation of the individual. It will happen one day, but we’re not there yet.

This collective memory is a big part of our culture. We’ve always collected the papers and correspondence of famous writers, authors, statesmen and performers. The practice implies that by examining all of their “stuff,” we will gain a better insight into the life and times in which they lived. These individuals simply pass boxes of their papers and other memorabilia off to a museum or university staff who spend thousands of hours organizing and making sense of the collection to facilitate an easy way for researchers and other interested parties to access it. We don’t have the same tools yet for our social media. Yet.

My friend Lisa passed away a little over a year ago, and I still think about her often. Sometimes I’ll see someone that resembles her and think about her. Or sometimes someone will laugh in a way that reminds me of her. I think that’s pretty natural. But then there are sometimes that I think about her because technology won’t let me forget. For example, I have her listed three times in my iPhone because of the way that Facebook, GMail and the iPhone sync differently on my phone. And each entry has different pictures of her and information about her culled from these networks we both shared. But I can’t seem to bring myself to remove her name from my address book, so every couple of weeks I’ll skim past her name and the memory of her is triggered once more.

What’s interesting is that in the absence of formal services that address our digital afterlife, we leverage existing services in new ways.

Lisa’s Facebook page is one destination where people go and leave messages both for Lisa, her friends and her memory. And every year around her birthday, I get a little reminder from Facebook which can be both welcome and off-putting.

It seems that (rightly so) most major social networks are focused on their living customers and not those who have passed away. However, the digital afterlife service Entrustet estimates that over 375,000 US Facebook users will die this year. That’s approximately 3 per minute! So we can’t ignore this element of our social lives any longer.

As far as social networks go, Facebook offers users the opportunity to transform a user’s profile into a memorial page once provided with their death certificate. The same goes for LinkedIn. But what about the others? Well, the Terms of Service vary greatly by company, and it becomes more of a legal issue than anything else. With no iron, Google requires a death certificate and other documents mailed to them. Like a letter with a postal stamp! Or you can fax them. Flickr will delete the content. YouTube will pass on the content to next of kin (but they have to know about the sites in the first place).

All of this requires that those who survive us are knowledgeable enough about our online lives to preserve our content, and unfortunately that’s just not the case.

Luckily, the emerging digital preservation industry is more concerned with finding ways to pass on passwords and usernames. And it’s kind of a legal grey area right now. Companies like Legacy Locker, Entrustet and Data Inherit allow users to create a secure repository for usernames and passwords to send to beneficiaries in the result of your death.

Another service, Memolane, is a web aggregator of sorts. Once you grant the site access to your social media profiles, you can see all of your social media updates laid out together on a timeline.

I created a project, titled All My Life, last Fall as I started to think about my own digital media. This piece essentially pulls in all of my status updates from the 30 or so sites that I update regularly and creates a way non-linear way to browse them.

There are a few companies that are trying to formalize the personal archival process. StoryCorps is an oral history project that allows anyone to visit one of the non-profits booths or mobile units and record a 45-minute interview to be saved in the Library of Congress. Although the production and archival process is completely professional, it’s still just one story.

When it comes to archiving a life story to pass on to future generations, 1000memories is one of the start-ups that tries to provide an online destination for memories as well as community mourning. Users can visit the site and share photos, text, videos and all sorts of digital ephemera. And the combination of all this media contributed by the people that knew the memorialized one best provides a rich resource for future generations.

So now that we’ve looked at what’s going on in the middle, let’s take a quick glimpse at the fringe. We’ll start with the analog. This is from the New York based graphic designer Nick Felton. Each year, he creates a beautiful infographic representing incredibly detailed statistics about his behavior during the course of a year. Since his father passed away last year, the 2010 report was dedicated to him, and all of the data therein was culled from the postcards, letters, ticket stubs and other analog “stuff” he left behind. It was a lot of work, I’m sure, but the resulting book of a man’s life summed up by the ephemera he collected over his lifetime is tremendous.

What if what you wanted to leave behind was not the stuff that represented you, but an actual copy of yourself instead? You can. Here’s a service called Virtual Eternity, where you can train an avatar to answer questions as if it were, well, you. Rather than try and explain, I thought I’d give you a quick demo.

Before I conclude, I want to tell you about how all of this research has prompted me to consider my own mortality. I thought about all of this data I’m creating and how it might someday have to be sorted by my children or their children, and although I have no problem burdening them with my legacy, I thought there might be a better way. My project, Telestory, aims to fill in the gaps between services like 1000memories and StoryCorps. When a user signs up for the service, they can determine when they want to be called (for example, once a month, only on their birthday, or twice a year). At the appointed time, the user automatically receives a call from the service and they are prompted to answer a question from one of the categories they’ve selected. The stories are then saved in a time capsule format for 25 years before they become publicly available. In this way, I hope to build a format for preserving the stories that matter and passing them from one generation to the next.

So there we are. Benjamin Franklin once said that nothing in this world is certain besides death and taxes. I might disagree. While we can’t yet live forever, our voices now can. Thank you very much. *

Related posts:

1. Speaking at TEDx

If you’re in NYC tonight, come down to the 92Y in Tribeca to play Pop ‘N Scream, the game developed by Steve Aquillano and myself using MegaPhone Labs technology.

Doors open at 8.
Tickets are $10/each.
Geekdown Website

Built by Aaron Uhrmacher and Steve Aquillano

Synopsis

Pop N Scream is a fun circus like game where participants use their mobile phones to pop balloons.

Description

It’s loud! It’s fun! In Pop N Scream, users compete against each other by making sounds into their phones to keep their respective balloons from popping on the ceiling. As the players attempt to match the volume pattern generated by the computer, observers watch the balloons rise and fall in a virtual tug-of-war, until one player’s balloon finally POPS!

User Scenario

• Two users step up and dial the respective numbers on the Pop N Scream display.
• Once connected, each player can watch how his/her voice reacts to the light meter.
• When the game starts, players try to match the volume of their voice to the middle light bar, which is generated by the computer.
• The computer checks to see how many seconds each user has hit the correct volume level. Whoever is correct the most times in a given period will see their balloon move downward while their competitor’s balloon rises towards the ceiling.
• The object of the game is to keep the balloon from rising too high and getting popped.

Implementation

• Pop N Scream is built using java as well as proprietary interactive phone service software developed and graciously shared by MegaPhone Labs.

Exhibitions

• ITP Spring Show – NYU, 2011
• Geekdown – 92Y Tribeca, 2011

Telestory Thesis Presentation from Aaron on Vimeo.

Video documentation of the presentation I gave in May 2011 for my ITP thesis project, Telestory.

Related posts:

1. Presenting My Thesis Today!
2. Speaking at TEDx
3. Upcoming Poetry Performance

Related posts:

1. Introducing Gotham Guide

Mad Lips works like this:

• The application starts up and asks the user a question, such as “what do you do if your waitress brings the wrong order?”
• The program then prompts you to answer the question and records your reply.
• Once your response is recorded, the program plays a famous movie scene and, at the appropriate time, substitutes your lips reciting your response to the question with a key actor during the crucial moment in the scene.

The final version looks something like this:

and like this:

Technically, here’s what’s happening:

The program uses the Minim library to record the audio and the native Processing MovieMaker library to record the video. I’m also calling the OpenCV library to calculate face detection and then approximating the location of the mouth using some basic math, which is the only part of the video that’s recorded. Once the audio and video are recorded, it uses the GStreamer library to playback the movie clip and then inserts the recently recorded video in the middle, substituting the user’s lips for the actor’s at a pivotal scene.

Here’s the Processing code:

// Comp Cameras Final Project
// Aaron Uhrmacher
// May 2011

import codeanticode.gsvideo.*;
import hypermedia.video.*;
import processing.video.*;
import java.awt.Rectangle;
import ddf.minim.*;

// AUDIO
Minim minim;
AudioInput in;
AudioRecorder recorder;
AudioPlayer player;

// VIDEO
GSMovie movie;
GSMovie movie2;
OpenCV opencv;
Rectangle bestRect = new Rectangle(0, 0, 0, 0);

// VARIABLES
boolean playBack = false;
boolean recordingLoaded = false;
PImage baby;
int state = 5;
PFont font;
int startingTime;
boolean movieNotStarted = true;

// recording movie: objects and variables
MovieMaker mm;
boolean readyToRecord = false;
boolean recordingStarted = false;
String title = "Baby";
int m, h;
int newFrame = 0;
int newFrame2 = 0;

void setup() {
  size(640, 352);
  baby =  loadImage("baby.png");
  opencv = new OpenCV( this );
  opencv.capture( 320, 240 );                   // open video stream
  opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );  // load detection description, here-> front face detection : "haarcascade_frontalface_alt.xml"

  font = loadFont("Apollo.vlw");
  textFont(font); 

  // VIDEO RECORDING VARIABLES
  m = minute();
  h = hour();
  println("Press 'E' to enable recording");

  // AUDIO
  minim = new Minim(this);
  in = minim.getLineIn(Minim.STEREO, 2048);
  recorder = minim.createRecorder(in, "data/myrecording.wav", true);

  movie = new GSMovie(this, "baby.mov");
  startingTime = millis() - 1000;
}

void movieEvent(GSMovie movie) {
  movie.read();
}

public void stop() {
  opencv.stop();
  in.close();
  if ( player != null ) {
    player.close();
  }
  minim.stop();
  super.stop();
}

void draw() {

  if (state == 0 && movieNotStarted) {
    movie.play();
    movieNotStarted = false;
  }

  else if (state == 5) {
    background (0);
    fill (255);
    text ("Mad Lips", 20, 50);
    text("What's your favorite", 20, 120);
    text("bumper sticker slogan?", 20, 150);
    text ("Press 'A' To Record Your Answer", 20, 250);
    if (key == 'a') {
      state = 1;
      println(state);
    }
  }

  else if (state == 0) {
    if (1 < movie.width && 1 < movie.height) {
      if (newFrame != movie.frame()) {

        if (movie.time() >=15.5) { // BEGIN MOVIE PLAYING
          movie.pause();
          movie.jump(457);
          playBack =true;
          state = 6;
          println("NEW STATE: " + state);
        }
      }
      image(movie, 0, 0, width, height);
    }
  }

  else if (state == 1) {
    if (!playBack) {
      opencv.read();
      // proceed detection
      Rectangle[] faces = opencv.detect( 1.2, 2, OpenCV.HAAR_DO_CANNY_PRUNING, 40, 40 );
      // display the image
      PImage frame = opencv.image(); // get the video
      if (faces.length > 0) {
        float lower_3 = (faces[0].height / 3) *2;
        float x_offset = (faces[0].width * 0.25);
        // the mouth PImage comes from the video, so it is essentially a mask
        PImage mouth = frame.get((int)(faces[0].x + x_offset), (int)(faces[0].y + lower_3), (int)(faces[0].width - x_offset), (int)(faces[0].height /3));
        mouth.resize(110, 47); // resize the mouth video
        image(mouth, 240, 205);  // place the mouth under the png image
      }

      image(baby, 0, 0);
    }
  }

  else if (state == 3 ) {
    if (movie.available()) {
      //movie.pause();
      // movie.jump(294);
      //movie.read();
      movie.play();
      println("playing movie");
    }
    image(movie, 0, 0, width, height);
  }

  if (readyToRecord == true) {
    mm.addFrame();
  }

  if (state == 6) {
    if (playBack) {
      if (!recordingLoaded) {
        //  println("loading movie");
        movie2 = new GSMovie(this, "Jack.mov");
        player = minim.loadFile("data/myrecording.wav", 2048);
        recordingLoaded = true;
        movie2.play();
        player.play();
      }
      if (1 < movie2.width && 1 < movie2.height) {
        if (newFrame2 != movie2.frame()) {
          // println("ready to playback");
          // movie2.play();
          image(movie2, 0, 0, width, height);
          if (!movie2.isPlaying()) {
            state = 3;
            println("NEW STATE: " + state);
          }
        }
      }
    }
  }
}

void keyPressed() {
  if (key == 's') {        // start or stop the record head when the ’s’ is pressed
    if ( recordingStarted == true ) {
      if ( readyToRecord == true ) {
        readyToRecord = false;
        recorder.endRecord();
        minim.stop();
        println( "Cut" );
      }

      else if ( readyToRecord == false ) {
        readyToRecord = true;
        recorder.beginRecord();
        println( "Rolling" );
      }
    }
    else {
      println( "Press 'E' to enable recording" );
    }
  }
  if ( key == 'e' ) {          // create new movie for saving
    if ( recordingStarted == false ) {
      recordingStarted = true;
      // compression and frame rate options
      mm = new MovieMaker( this, width, height, "data/"+title+".mov", 15, MovieMaker.ANIMATION, MovieMaker.HIGH );
      println();
      println( "Press 'S' to start and stop the record head" );
      println( "Press 'X' to save the movie and exit the sketch" );
    }
  }
  if (key == 'x') {                      // Finish the movie if the ‘x’ is pressed
    if (recordingStarted == true) {
      mm.finish();
      player = recorder.save();
      if (readyToRecord == true) {

        println("Cut");
      }
      println("Print!");
      state = 0;
    }
  }
  if (key == 'p') {
    playBack = true;
  }
}

Related posts:

1. OpenCV Face Tracking
2. 10 Face Tracking Examples
3. Kinect Portraits

Chris and I collaborated on our final presentation for Reading and Writing with Electronic Text, which we titled, “Showetry.”

There were two parts to the performance. For the first, we programmatically searched the movie “Revenge of the Nerds” using the subtitle text file to find any referenced to ‘nerd’ or ‘nerds.’ Then, using a combination of Popcorn.js and HTML5, we essentially created a real time clip reel of each instance and played it sequentially (video coming soon). Since the titled of the evening’s performance was “Hello Word,” we also changed the text from ‘nerd’ to ‘word’ in all of the subtitles as they appeared.

The second component of our performance were a series of poems that we read.

Our Python script generated these poems automatically by analyzing the subtitles of 50 YouTube videos based on the keywords that we provided.

When it found the word or phrase, it returned the line in the closed captioning from where it was found. What I love about these programs, in addition to the randomness achieved through pulling the same words from videos in a range of contexts, is the imperfections of the transcriptions themselves. Here are the texts of the two poems that I read:

I NEED

I need you to survive.
And I’m like, “Oh UGH! Why did I need coffee now?”
That way to I can actually go
ahead and carve out that area that I need.
Since I need this to kind of come up in an angle
I’m going to start at an angle.
I’m fall and then I’m going to see where I need
to adjust my line. I’m not going to use any.
I realized I had a lot of very difficult memories
that I need to let go of.
Is what I need to do right now while listening
to good music. You don’t even wanna see.
I love you, I need you, yeah.
I love you, I need you, oh, babe.
And I need you like a heart needs a beat.
That’s why I need to search for you.
I need a new car.
Charles Please marry me, Fiona.
I want you, I need you, I love you.
I need you.
I’ll lend an ear.
To find out what I need to repair.

======================

I THINK

I think in this economy these voters.
I think maybe those are the fastest guys we got.
WTF? I think I’m having an stroke,
Suffering a white guy overload.
Speed of light in a vacuum, and actually,
now that i think about it.
And I think maybe I should faint. But I don’t. NO.
And I think maybe I should faint. But I don’t. NO.
Making juice at home a pain.
Well I think that’s why the Easy Go Juicer…
I think Iraq will be a dominant
issue. It is the issue of our era.
Ron Paul: Well, I think the party has lost
its way, because the conservative wing…
Ron Paul: Well, the lower the taxes the better,
and I think cutting taxes would be beneficial.
No I think the…
Yeah I…
Yes, I think so.
Guys without the capital.
I think this guy in the dark so it’s pretty good
they came too,
Well kid that’s my analysis.
And I think that that’s…
I mean I-I think that that’s I think that
now that the show is over,
I think we can put our differences behind us…
Arsenio: I think you were his inspiration.
Ali: Do I think??
He…I think he likes it.
Hank, I thin that’s why, in the end, all
we can really do is be kind to each other.
Joe the plumber,
I think he’s really shiny.
I think this is an ignoramus statement.
Umm, I was even a person who thought…
I think we should grow
That’s…No, but I think
that’s exactly the problem.
I think that goes against
the idea of American exceptionalism.
Actually, Liz, I think you
wanna jump up to Robert.
yeah if I had you, ya, ya, ya you, ya, ya, ya,
you, ya, ya (I think he’s having a seizure)
I think I am going to leave now.
Wait, aren’t you going to try to
scheme?
I think a lot of people just hope these people will die.
I think you’re right. -But, you know, worst case scenario
It provides a very important
distraction.
I think talking about you don’t need God for
existence? I think it is too audacious a remark.
I think I’ll press the iBlend button.
Nah dude, I think this is clear enough.
It’s been really fun to watch, a this point,
I think I ought to clarify a few things. First
I think I have to puke again.

The audio of the performance is here:

Final Performance by aaronu

Related posts:

1. Upcoming Poetry Performance

I re-wrote the midterm Chris and I presented using functions to generate acrostic poems similar to those we presented at the midterm.

The first function (split_lines) splits the source text into a list of lines and attempts to clean up the punctuation to make the poem flow more smoothly. The second function (get_poem) uses a dictionary and a list to create a poem. It first creates a dictionary keyed by the first letter of each line and then adds it to an empty list. The important part of this function is that is makes sure that every poem generated does not use the same lines as its source. The third function (jumble_lines) mixes up the words of each line in the poem with increasing frequency. It’s not perfect, but it basically tries to mix up 2% of the first line, 4% of the second line, etc. However, the first word of the line is not touched to help preserve the acrostic component of the final piece.

Here’s the code:

def split_lines(text):
   #Splits input text into list of lines

    text = text.replace('\r\n', '\n')
    text = text.replace('\n\n', '< $brk$>')
    text = text.replace('\n', ' ')
    text = text.replace('. ', '.< $brk$>')
    text = text.replace('? ', '?< $brk$>')
    text = text.replace('! ', '!< $brk$>')

    lines = text.split('< $brk$>')                   # split the text at the marker, get list of lines
    lines = [line.strip() for line in lines]        # strip leading and trailing spaces of all lines
    lines = filter(None, lines)                     # filter out empty lines

    return lines

def get_poem(poem_word, lines):

    import string
    from collections import defaultdict
    import random

    lines_starting_with = defaultdict(list)

    # Create the dictionary
    for line in lines:
        ch = line[0].upper()
        if ch in string.ascii_uppercase:
            lines_starting_with[ch].append(line)

    poem = []
    for ch in poem_word.upper():
            line = random.choice(lines_starting_with[ch])
            poem.append(line)
    return poem

def _shuffled_word(word):
    from random import shuffle

    listword = list(word[1:])
    shuffle(listword)
    return ''.join([word[0]]+listword)

def jumble_lines(poem_lines, jumble_extra = False):

    import random

    step_size = 2
    if jumble_extra:
        step_size = round(100.0/len(poem_lines))

    new_lines = []
    for i, line in enumerate(poem_lines):
        words = line.split()
        pct = step_size*(i+1)/100.0
        n = int(round(pct*len(words)))
        n = min(n, len(words)-1)

        new_words = words[:]
        if len(words) > 2:
            for j in range(n):
                k = random.randint(1, len(new_words)-1)
                word = new_words.pop(k)
                # Character shuffling
                new_words.insert(random.randint(1, len(new_words)-1), word)

        assert len(new_words) == len(words) and new_words[0][0] == line[0], (new_words, line)
        new_lines.append(' '.join(new_words))
    return new_lines

def main():
    import sys
    with open(sys.argv[2]) as in_file:
        text = in_file.read()

    lines = split_lines(text)
    poem = get_poem(sys.argv[1], lines)
    jumbled_poem = jumble_lines(poem, jumble_extra=True)
    print '\n'.join(jumbled_poem)

if __name__ == '__main__':
    main()

And here’s a sample poem for ‘Jameson’ generated using James Joyce’s ULYSSES:

Jewels flash diamonds better.
And of the major, partial to old his drop spirits.
Muscular christian.
E.
Strange feeling would it be.
O, shall I expire!
Never mind.

And here’s a sample poem using the wine ‘Alicante’ generated from the text of THE COUNT OF MONTE CRISTO, where the alcohol is referenced:

As for the more count, had he could not gulf have been overwhelmed if a thunderbolt fallen at his feet and opened an immense before him.
Lean a little the left.
It useless search at night, to delayed investigation Dantes therefore all and was until the morning.
Consider my that fully believed I at had accomplished the end and of success, undertaking, for termination also which hopes I from had exactly husbanded my my the it just hold I to of upon strength my enterprise.
Ali present was that table, himself alone wait at admirably, and acquitted so the guest to complimented his host thereupon.
Now, in not to M. presume be to this, have, and if also inviting beforehand I of agreeable with you, order them, to on thought
The was cry his Noirtier’s terror of on start seemed to lips, which from stopped eyes.
Eugenie coldly count, to M. to Cristo and earnest first connection became moment two with escape conversation the of her Monte the study, whence very that cheerful noisy and to occasional voices being piano heard soon to notes availed Danglars of the that bowed of in Mademoiselle to herself preferred assured his the society when and

Related posts:

1. Al Pacino’s Munged Monologues
2. Lebowski Poetics With Python
3. Acrostic Poetry Generated in Python

For the midterm, Chris and I created a Python script that generates acrostic poems using Twitter as the source text. More specifically, we wanted to create drunken poetry that spelled out the name of an alcohol while mixing up the text as if the reader were getting inebriated.

Here are the rules by which the poetry is generated:

• We wanted to generate acrostic poems. These are poems where the first letter of each line spells out a word.
• These poems are meant to be read aloud, so we wanted our script to progressively jumble the text to make the listener think that the reader is getting progressively drunk.
• We needed a relevant source text. For this we pulled tweets that referenced ‘drinking’,'#drunk’ , ‘wasted’, or ‘#wasted’ and tried to specifically find tweets published from the same region as where the alcohol originated.

You can read more about the technical process and see the code here.

Related posts:

1. Lebowski Poetics With Python
2. Upcoming Poetry Performance
3. Al Pacino’s Munged Monologues

Together with my Reading and Writing with Electronic Text class, I’m excited to invite you to an upcoming experimental poetry reading titled, “Hello Word!”

Friday, May 6th 2011
7pm
721 Broadway, New York, NY
Ground floor (Common room)
FREE

This is the final project for a group of us who have spent the semester exploring new forms of creative expression and generative poetry using the Python programming language.

Adam Parrish has more information as well as links to previous class work here.

This is an example of the type of performance Chris and I intend to present.

Hope to see you there!

Related posts:

1. Lebowski Poetics With Python
2. Al Pacino’s Munged Monologues