Skip navigation

Category Archives: Uncategorized

The Curator

Polo Chau

François Chu

Sue Ann Hong

Patrick Gage Kelley

Art That Learns 2009.


This installation explores the act of acceptance and rejection, of the artist, and the influence of institutions such as museums and galleries on artists’ directions as well as that of artists on such institutions.

Here, a computer automatically decides whether to add to its collection simple drawings submitted by the audience. While various criteria are used in judgment and reception of art in our society, “The Curator” simulates a simple aspect: originality.

It uses a machine learning algorithm based on anomaly detection to determine the acceptance of pieces, rejecting the ones similar to those already seen. (Also, to simulate the concept of revival, after a time, older pieces are forgotten by the algorithm.)

Hence the computer adapts to respond to the artistic ideas given by the audience, while we hope that the decisions made by the computer shape the submissions as the audience attempts to please the machine, and avoid brutal rejection.

The Process

Our design was an iterative process from the beginning. The idea, a cousin to that of an early concept presented to the class referred to as “Lack,” incorporated children by allowing them to create their own artworks which would be appraised.

Early on we decided to have two end states, an acceptance, and a rejection, where work would be destroyed. However the physical incarnations changed, from clear acrylic boxes to the idea of the shreds simply tumbling to the floor.

The best example of our attempts to build simplicity into our design come from the submission module. We knew there needed to be a way for the children to enter their work into the machine, however the specifics of this mechanism changed frequently. From a fed document strip scanner, to a vertical slot, a horizontal slot, a deconstructed flatbed scanner, we in the end went with a design that was a single slot in a piece of acrylic.

Our physical construction involved laser cutting, wood work, bending acrylic, priming, painting, the deconstruction of a mouse, the deconstruction of a shredder, and finally electrical work.

The final circuit used was simple, an arduino powered two servos, one for acceptance, the other for rejection, and additionally a relay, through a transistor, powered by a 9V battery, to control the timing of the shredder (whose automatic mechanism & safety switch were removed).

The shredder itself was deconstructed and rebuilt inside of a clear acrylic box.

For the learning algorithm we use a simple anomaly detection algorithm. We keep around the last n (we use 30) drawings in the form of a orthonormal basis: every time we get a drawing, we compare its feature vector (of pixels) v to the existing basis vectors (except the oldest one, which is what we’re replacing), and store the component of v orthogonal to them. Before creating the new basis vector, we test the drawing for acceptance by projecting onto the space of the orthonormal basis, then compute the reconstruction error. If the reconstruction error is greater than our threshold for acceptance, then the drawing is accepted. Intuitively, if the current drawing can be described well by the last 30 drawings (and hence not “original”), we do not accept it. Note that since the feature space is much larger (300×200=60,000?) than the basis space, the basis should not span the space of drawings.

We also learn the threshold for acceptance over time to maintain the specified acceptance rate (e.g. we used 40%). After deciding the fate of each drawing, we set the threshold such that the threshold would have accepted 40% of the last 30 drawings.

The Installation.

The installation was reasonably straight forward. We late in the process found out about the light up wall that would be the backdrop of our work, added frames to compensate for the space, which increased the overall impact of the artwork, and added a second stage of “acceptance” (albeit, a human stage, not one controlled by learning).

We made a few changes during the process including changing the timing of the button (adding a delay), adjusting the acceptance threshold, and fixing one of the servos which was overdrawing power from the arduino.


One success is that we were actually able to get children’s attention and keep it, possibly for too long. Many parents had to take their children away from the exhibit. Children were pleased with acceptance and rejection, the clear box was a sign of winning, but the shredder was fun, loud, and visceral. As everything was a reward, they wanted to keep drawing, sometimes to the point of mass producing scribbles…, artwork.


We are going to call this a success, children liked it, no one cried, no one got hurt, it was able to stay in the museum, the learning worked (well enough), and it photographed well. The biggest failure is that it likely did not truly accomplish it’s intended reflection on rejection in society, but with six year olds, this may be a near impossible task to accomplish.

Photos & Videos


We’re interested in the dynamics of pollination, what that means for each individual plant and what it means for an ecosystem.  In the natural world, plants continuously adapt to pollinators in their environment like birds and insects, seeking a balance between attracting the biggest number and keeping them moving. Some morphological aspects that change are: changing shape, color, line patterns, and scent. They also explore a variety of attractants from the promise of or actual food or sex (mimicking the appearance of the other gender).single-plant

In this installation, we can explore the point of view of an individual plant or the whole ecosystem. Plants would be able to change to become more attractive to visitors. The plants would be able to vary using color, rhythms of pulsing light. In a sense, our plants would have the opportunity to evolve a plant song in light. An ecosystem would be interested in maximizing its life by using all the plants.

Input: proximity of people

Black box: optimizing the number of people around, and/or their movement between plants

Output: the most attractive plant song

SCENARIO 0: One Plant

Only one plant is installed. Visitor can interact with it by feeding it different tags and/or talking to the plant. The plant learns from how many people it can attract (proximity) to be more attractive.  The information received via tags or sound influences the strategies it attempts.

SCENARIO 1: Ecossystem

A series of glowing “plants” are in a dark room. Each plant has tags in it. People would be able to grab tags from one and drop in the next (pollination). The objective of the plant is to spread its information by attracting the most people.Potentially we could compare competition with collaboration, if the plants are allowed to work as an ecosystem that tries to learn how to maintain the largest number of people in the area and moving between plants.

SCENARIO 2: Pollination “battle”

A series of 4 glowing “plants” are in a dark room. Between the four plants is a 3 ft x 3 ft LED board. From time to time, each plant may pollinate and the LED board lights up near the plant. The nearby presence of visitors triggers pollination, maybe can also influence the pollination pattern.  Plants will try to “cover” the LED board with their seeds. In a sense the four plants could be playing a game against each other.

Potentially, pollination can occur if seeds of one plant reach another plant.

SCENARIO 3: Similar to scenario 2, but everything happens on the LED board, including the plants which become a clump of LEDs.


 – How to create enough interesting variance in the plants? Each plant can deal with sound and light, patterns have a small set of LEDs. Each plant can detect light, sounds..

– How to pass info between plants? Using RFID tags

– How to visualize next generations?



Pollination is a means of sexual reproduction in the plant world, the “male” gamete is released (pollen) to attempt to reach the ovaries of another plant. The gametes carry genetic information. The fusion of the two gametes creates a seed that can originate a new plant.

I keep thinking about gene splicing with things that aren’t living, that is, how can we give living objects the ability to breed traits involving kinematics, computational thought, etc.     How would they use these new abilities to collect food, defend themselves, develop relationships with their environment, etc.

One idea is a plant that can use sensors to know how many people are near it then attract or drive away people as its need for light, warmth, etc change.  As an example, if it’s getting too much sun, it needs to attract people to give it shade,  but if it’s very cloudy, it would try and drive them away.   A couple of ways to attract/repel people are to play sounds they like/dislike, or to display interesting patterns of light, so this plant might have controllable LEDs and hidden speakers to attract/repel people, and sensors hidden in the cabinet and “dirt” to determine how many people are nearby or how much light it’s receiving.

To answer Carlos before he asks, the machine learning component is what patterns of light and sound work best to attract/repel people of different heights at different times of the day.

rough sketch of attract/repel plant

rough sketch of attract/repel plant


I think I got video of everyone’s projects. If you want the raw .dv file, let me know and I’ll post it later this week.

Well let’s be honest, we are certainly figuring this out as we go.

We have stripped out the generative music (and thus, the pipe organ) from the project. Probably mostly because we thought this would help us distinguish us from the other generative music group, and also because we weren’t committed to that learning anything very interesting. That being said we are still working on defining what part of our project learns – and also, if it is art.

We are keeping the monsters, and the buttons. People will be able to push the buttons to make the monsters do something. Also we will be tracking people as they move, and translating that into monster movement. Very reactive. The plan is to save all of these movement patterns and create generated monsters that interact with the real people-based monsters in the animation.

Things to implement

  • People (blob) tracking
  • Masking the blobs maybe into some sort of particle system that the monster can be drawn as.
  • Building a box to house projector, laptop
  • Constructing button interface
  • Animating the scene with monster (maybe 8-bit)
  • Storing the movement patterns so that they can be recreated
  • Bonus: customizable monster creation?

If you are not on the mailing list, you can subscribe here.

Some more things to ponder.

Art, Installation, New Media (overview)


Hi everyone,

I use a tablet PC in class.  I’ll try to post my slides without annotation before any lecture I give.  I will then post the annotated version after the lecture.

intro to ML

intro to ML annotated slides

See you in class,