dpp
Week 7
Read
Although these toolkits are powerful, they cannot help in most new prototyping environments because they are not available.
-Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes
Make: Zen Tile
Tile a surface by sketching primitive shapes.
| Game | Recording 1 | Recording 2 |
Notes
- I studied the $1 gesture recognizer algorithm and visualized how it works.
- I’m interested in simple visual grammar. Can we sketch shapes to tile a surface? The idea even led to Nobel Prize in Chemistry!
- I also came across this fresh best paper from UIST ‘26. I like its demo of fitting shapes with automatic snapping.
- Let’s combine these ideas: sketch to produce a shape, and use automatic constraints to snap shapes for tiling.
Process
First, I got the Declarative Dragging library to work. The author provided a good AGENTS.md file, so my coding agent did the job without much effort.
Next, I want to expand the visual grammar but make it procedural. Let’s “unlock” a new shape if it can fit into the current tiling pattern. (later I added a rule that 2 a tile unlocks when two connected edges are shared with existing tiles)
And then I added sketching, roughly based on $1 gesture recognizer, but additionally use position and orientation on the canvas to get a boost on accuracy.
Lastly, add visual design, sound design, and polish the experience. AI is pretty good at this kind of work, I just need to design a coherent visual style - I like the b/w high contrast vibe - and the sound track/effects. I went for a zen/relaxing sound track. The sound effects depend on modes: sketching mode plays crayon sound, drag-drop mode plays a thud when you drop.
Week 6
Watch
- Line interpolation
- Spring
- Particle
Read
I wanted to impress people, but I wanted to do it with style!
When I was a high schooler, I didn’t think much about art. Until I fell in love with this Art School Girl.
“Is this an art project or a tech demo?”
Make: Broken Record
Prompt: Create a code based drawing system that animates, and brings drawings to life.
Strobe Light Warning Live app (with sound) | Video recording
Notes
- Started with video tutorial, make some squiggly lines and crawls like a worm. It was gross! You try it.
- I tried to spinning things and use the motion itself as paint.
- Tried coupling the stroke to the path of the animation.
- How about I speed it up, indefinitely?
- I start to see strobe light effect. Reminded me of John Whitney
- I really liked the strobe light effect, let’s put it on a disc and spin it.
- The vinyl record metaphor is salient:
- You scratch on the disc, it’s a broken record that repeats what you did.
- Spinning accelerates, creating a strobe light effect
- Lastly, add sound effects. Used AI to generate different “Hollywood” sound effects and picked two tracks that I liked to mix together.
Week 5
Watch
Read
The pre-camera, pre-computer Chinese artist took a life-time of understanding in order to make one meaningful ink filled brushstroke. It may take a life-time to develop a computer program to make one new communicating pen line which is meaningful for us.
-Computer Grass Is Natural Grass by Colette S. Bangert and Charles J. Bangert
There is no privileged place in any picture; on top, below, to the left, to the right, in the middle-every place is of the same importance and has the same role.
-My Mother’s Letters: Simulation by Computer, by Vera Molnar
Make: TimesDraw
| Live app | Video recording |
Prompt: Create a code based generative drawing that models something in nature, such as handwriting, grass, clouds, etc.
Notes
- Time-series models predict the periodicity, patterns, and trends in data over time. In nature, we have trees swinging in wind, water ripples in a pond, high/low tides, and temperature cycles across four seasons, all caused by underlying periodicity.
- Recently, Mistral released a new frontier model for time series prediction, named Timestral. The model takes data seen in the past and predicts the future.
- What if we use this model to predict the hand motion of an artist?
- Conceptually, the artist will see the line they are about to draw, 2 seconds into the future. Today’s LLM is predicting the next word we will say, will tomorrow’s machine predict the next thing we will do?
Process
First, a minimum system that predicts the most likely trace, the least likely trace, and the medium trace. Just to check if the model is working. The black trace is made by me. The red trace is model prediction. The semi-transparent red traces are less likely trajectories.
Then trying to visualize all possible trajectories. The model provides up to 21 quantile predictions, from least to most likely. The higher the likelihood, the darker the color.
What if we make it realtime? I’m curious about the feedback loop between the user and the model. Everyone has the experience of pointing a camera at its own screen. This is where things get weird. We also introduce some color coding to distinguish the trajectories. Also, fade them out so the more distant the future, the less visible they are. And if we keep all the trajectories on the canvas, to preserve the history of the drawing, it becomes a beautiful mess.
Based on my observation, the model is not smart enough to understand the “big picture” behind the drawing, but can follow circles and sine curves quite well. It struggles with sharp angle turns, which probably don’t occur that much in the time-series datasets.
Week 4
Watch
Read
One of the most obvious characteristics that distinguishes our model of the computer artist is an emphasis on process rather than artifact. The commercial paint system is a tool optimized for minimal user involvement and maximum production output; by contrast, computer artists treat the computer as a generalized medium within which to develop facile tools over which they exercise intimate control
The ultimate product of in-depth human-computer interaction is the human transformed. How many interfaces design for this product?
Make: Zoom Draw
| Live app | Video: Way in | Video: Way out |
Prompt: Create a code-based drawing tool — can you extend gestures in an interesting way or “complicate” the act of drawing?
Process notes:
- Complicate the pen:
- Experimented with different brush styles and a focus on geometry, but they didn’t feel compelling enough.
- Complicate the paper:
- Complicate the interaction between pen and paper:
- Restricted movement to 90-degree angles.
- Explored having each new stroke create gravity.
- Final direction: Zooming while drawing:
- Zooming is a digital-native capability impossible on physical paper.
- Coupled active drawing with zooming, giving the user a sense of control alongside a feeling of disorientation.
Week 3
Read
Form came to be seen as imposed by an agent with a particular design in mind, while matter, thus rendered passive and inert, became that which was imposed upon
Make: 7 Days of Drawing
Make the smallest and largest drawing you possibly can. (TBD)
Create a unique instrument for making a drawing. (TBD)
Make a drawing come to life - Animating the snake. I feel bad that the snake has to die before I run out of paper
Make a collaborative drawing with someone else - Quincy, Lucy and I are visiting Paris. We did this drawing together at a cafe.
Do a memory drawing - Institute of Future Technology in Paris.
Make a detailed drawing of your computer, can be realistic or imaginary - This is my laptop running vscode. It’s really hard to get the precise proportion of the keys on the keyboard.
Make a drawing about drawing. (TBD)
Week 2
Read
Whenever the main purpose of data visualization is to open people’s eyes to fresh knowledge, it is impractical to avoid a certain level of visual complexity.
Authorship, after all, is not only what is created but also what is selected.
Make: Data Postcard
- The project responds to the prompt to visualize “battery level”.
- It visualizes both device battery levels and their owners’ human energy levels.
- The design exploration began with concrete representations and progressed toward abstract forms.
Artworks
Week 1
Read
Distances and Drawings - Four Letters From A Correspondence between James Elkins and John Berger
Art is there the most fundamental reminder of our capacity to find what is lost and bring it back where we can - almost - touch it.
Drawing is all the more powerful because it is both present to the eye and - nearly - absent to the touch.
The more the figures have “shrunk”, the more the space around them is charged with their presence.
To draw something well is to touch its resistance.
Make: Search Engine Optimization
- Instructions by Sun Chuanqi.
- Wall drawings by Eitan Wolf, John Chung, Sun Chuanqi.
- This project is a participatory wall drawing inspired by Sol LeWitt (example).
- The drawing process challenges participants to efficiently locate information by recursively dividing the space.
- It teaches the computational concepts of binary search and, more fundamentally, recursion.
Artwork
Process and findings
- The project began with the goal of teaching the concept of optimization.
- Participants were instructed to start with a naive approach: divide the space in half repeatedly using alternating horizontal and vertical lines, then progressively refine their approach with greater freedom in line placement to minimize the number of lines required to locate the target word.
- Testing with an initial participant revealed several key observations:
- The concept of a “cell” was too abstract, so the terminology was changed to “section”.
- Panel 2 required each line to always originate from the most recently drawn line.
- Freehand drawing on the third panel did not improve efficiency.
- The three-panel process was too long.
- Consequently, the process was simplified into a single-panel drawing.
- The final test with two participants yielded success.
- Participants understood the need to divide information rather than physical space.
- Participants ignored the semantics of the words and focused purely on executing the search process, raising the question of whether humans and machines share more in common than commonly assumed.
Test session
