“A painter paints pictures on canvas. But muscians paint their pictures on silence ~Leopold Stokowski

Friday, December 9, 2011

Circuitree 2.0

Published by Amy at 12:23 PM

Team: Oscar Camorlinga

Oscar: Unity, Scripting, Kinect Implementation, Texturing, Rigging
Amy: Concept Art, Modeling, Texturing, Rigging, Animation


We decided to create a dynamic and immersive environment in which the user will gain an interesting and new perspective of a computer virus.  The user is in a surreal forest where they are left to heal infected trees.  We expanded on our original idea by adding additional levels.  We originally were going to go with three levels + a boss level, but we decided to stick with three levels (third level being the boss level).  It was necessary to obey the Rule of Thirds!


Kinect Implementation - For Circuitree 1.0, there was a large separation between the user and the environment because of the navigation system (computer).  It was only natural to pursue the kinect since it involves the body.

Larva- Besides the computer worm, the larva are also responsible for spreading the virus. The larva interact by infecting trees and attacking the user.  When the user gets attacked by the larva, the camera shakes.  

Environment - The environment was expanded to three levels.  

1st level: The user encounters environment and explores.  The user notices green (healthy) and red (infected) trees in the environment.  The user can heal trees in this level.  The user also encounters a few larva in the level.  Larva infect the healthy trees.  The user can kill the larva by "stomping" on them.  Once the user has healed a certain amount of trees, the gate opens and they can proceed to the next level.  

2nd level:  The user encounters more larva and continues to heal infected trees.  Once the user has healed a certain amount of trees, the gate opens and they can proceed to the final boss level.

3rd level:  The user has to heal four columns around the computer Worm.  The worm can attack the user.  After they heal the columns, they heal the alter which is the central computing processor.   

Technical Background
Used the Kinect in conjunction with Unity to create a more immersive experience when playing the game. It proved easy to install and run in Unity, but the real challenge came with gathering the Kinect data and transferring that data to the controller mechanics. There was a vast array of scripts that were all communicating with each other while Unity was running, making it rather difficult to sift through thousands of lines of code to find only certain pieces of the script that were needed for implementation. Thanks to Krista Murphy and her successful implementation of the kinect in Honey, I shrunk something! (see research below) she was able to guide me through the process of gathering the data ans using it for player control.

Kinect controls were researched, such as how best to use the body to control the movement of the character in a 3D space. Should we control the character using the movement of our upper body like a joystick? Perhaps the head movement could be tracked so that you could simply lean forward and backward? How will you turn the camera to look at the world around you? perhaps by rotating your shoulders? Or using your hand as a kind of "guide" for the camera? Or possibly using both your hands as a kind of steering wheel as if you were in a vehicle? We ended up polling people their opinion on past Kinect experiences with games similar to ours, as well as researched online the best methods of Kinect character control. Ultimately it was decided that the head would be tracked to control the movement of the character on the X & Z plane, as if you were a giant joystick, and then the shoulder rotation would determine the rotation of the camera on the Y axis. Below are two videos on Kinect movement implementation that we used extensively as a reference.

Honey, I shrunk something!
Call of Duty FPS on Kinect Hack (FAAST) with Wiimote (GlovePIE)

Unity & Kinect:
We first decided to get the hardest part out of the way, which was the Kinect implementation in Unity for player movement, though this proved to be the least time consuming and easiest part of the project on the technical side. Once the Kinect was working, scripting began on the game. First, the initial game was re-scripted up to before the boss battle (tree healing), and then the addition of the Larva AI introduced. (see challenges below for more details) 

As Oscar implemented the Kinect with Unity, I was off to model a new feature to the game: Larva.  Thankfully, I had saved previous versions of the worm model and was able to construct the larva from separate pieces.  In the levels, the larva infect healthy trees and also come after the user.  They can bite at the user which causes the camera to stumble.  The user can kill the larva by lifting their leg and "stomping".  The larva has two attack and three death animations.  I also rigged, textured, and animated the larva.


Unity & Kinect:
Several issues arose, which are listed below, and were widely researched through Unity's amazing reference library and support forum.

Determining the current level and using that information to change game states and larva activeness
Particle systems for tree healing and final pedestal
Particle system lightning effect for the worm boss
Correcting the larva orientation to match the terrain using ray casting

Larva AI

Infecting trees
Attacking player once a certain "closeness" was reached
Spawning from the mother worm during the boss battle
Worm AI
Spawning larva
Following the player
Attacking the player once a certain "closeness" was reached
Receiving "power" from the pillars (electricity going to its body and following it)

Generative level creation
Kinect sensitivity changing depending on the distance of the player
Kinect not being able to re-calibrate users once they went off screen or calibrate a different user
Kinect shoulder data becoming corrupt and "reversing" the rotation
Kinect shoulder data needed to update rotation dynamically (the more you rotate your shoulders, the more the character turned)
Kinect stomp being extremely delayed at times
Everything else was either already implemented from the last version or didn't take too long to implement.

The worm rig was supposed to be updated in order to provide better control for the animation of the boss, but the spine IK setup proved to be problematic. Controls were not being updated with the movement of the spline (ie. leg/arm controls, head joint and jaw controls) Trying to fix the issue by parenting and/or constraining control movement to the spline caused double influence on joints and interesting rotation problems.

Future Improvement

Health would be an essential characteristic for the game.  Right now, the larva attacks only cause the camera to stumble or shake.  The larva would have more purpose (besides infecting trees) if they actually did damage to the user.  

Wiimote:  The wiimote would be used for camera orientation.  This would allow the user to look up and down.  

User Experience

1st person: Users felt as if they were in the environment.  They thought the game was enjoyable and wanted to explore.  (Screen was lowered to make the ground plane parallel to the actual ground).

2nd person:  People enjoyed watching users play the game.  They encouraged and helped the user through the environment when navigating for infected trees or larva.   

3rd person:  People seemed to be actively involved in the game, whether they watched the user or actually played Circuitree.


Presentation Slides

Published by Amy at 12:21 PM

Sunday, December 4, 2011


Published by Amy at 10:03 AM

We had extra models for Circuitree that were not implemented for the first project.  I had to check each model (5 rocks, 2 plants) to see if their normals were correct.  For the first project, Circuitree's trees normals were inverted.  That's why the the models were missing a few faces.  Oscar also had to fix the normals for each tree.  He separated them into different parts.  I reconstructed them by putting each of the tree parts together in an irregular manner.  

As for textures, Oscar created the new texture for the worm, larva, and trees.  I did the textures for the rocks and the plants.  The plant texture wasn't mapped correctly, so I plan to change it before the VIST show.  

Storyboard of game

Thursday, November 17, 2011


Published by Amy at 9:43 AM

Using segments and parts of the original worm model, I created the larva!  Oscar made some minor alterations in the body, face, and teeth.  I can't wait to do the animation for these little guys.  

Monday, November 14, 2011

Circuitree expanded!

Published by Amy at 10:12 PM

Here's an update on our project, I know it's been a while:

Hwaryoung suggested that Oscar and I should continue broadening Circuitree since it has potential.  Our previous idea for our final project would have been difficult to achieve in the limited amount of time.  This time, we hope to expand the game along with immersion and interaction.

navigation: wiimote
interaction: kinect

Three levels + Boss

1st level: The user encounters environment and explores.  The user notices green (healthy) and red (infected) trees in the environment.  The user can heal the trees in this level.  Once all the trees are healed, the user can proceed to the next level.

2nd level:  The user discovers a reason behind the infected trees.  The environment is very similar to the first level, but there are larva infecting the trees.  The user can "stomp" to kill the larva.  Once all the trees are healed and the larva eliminated, the user can proceed to the next level.

3rd level: The user is brought through a neutral zone before they encounter the boss.

Boss: The user has to heal the infected columns.  Larva will serve as distractions for the user and will regenerate.  The Worm will also lash out at the user.

Wednesday, November 2, 2011

Final Project Research

Published by Amy at 10:08 PM

Oscar and I finally decided on our idea for the final project this semester.  We are going to have a small ensemble perform on stage.  The conductor is going to conduct with a wiimote.  The conductor's movements will be visually represented on a screen.  We have not decided on what the visuals will be yet since we need to figure out our piece of music first.

I contacted the TAMU Orchestra instructor, Paul Sikes, yesterday and we are going to meet today to discuss our idea to see if he is interested.  He is usually pretty busy at the end of the semester with band and orchestra concerts, but he may be able to suggest a short and appropriate piece for our project.  If we decide to involve the audience using a kinect as Hwaryoung suggested, we will need a happier, more interesting piece.  Here is some research I did relating to conducting with wiimotes:

Tuesday, October 25, 2011

Japan Media Arts Festival

Published by Amy at 6:21 AM

Japan Media Arts Festival

The Japan Media Arts Festival is an event that promotes the creation and development of the media arts such as comics, animation, games, and media art.  Currently there are four divisions that are awarded prizes: art, animation, entertainment, and manga.  The Japan Media Arts Festival currently accepts submissions from 48 nations and regions.  

Art Division: Artwork created with computers or other electronic devices.  Interactive art, media installations, video works, digital photographs, graphic art, website works, etc.

Entertainment Division: Entertainment created with computers or other electronic devices.  Games (including web games and social network games), video works (music videos, publicity films including TV commercials, digital special effects videos), toys (including figures), character merchandise, website works (including advertisements, etc.), application software, etc.

Animation Division: Theatrical animation, TV animation, other original animation, animated short films, etc.

Manga Division:  Manga published in book form, in magazines (including works still being serialized), produced independently and in small private magazines, on websites (PC/mobile), etc.

For each division, one Grand Prize, four Excellence Awards and three New Face Awards will be awarded on the basis of artistic quality and creativity.  In addition, a Special Achievement Award will be awarded on the recommendation of the jury to a person or group having made a special contribution to media art in the four specified divisions.  Potentially 9 awards in each of four divisions.

Grand Prize
Certificate from Minister of Education, Culture, Sports, Science and Technology Trophy
600,000 Japanese Yen

Excellence Award
Certificate from Minister of Education, Culture, Sports, Science and Technology Trophy
300,000 Japanese Yen

New Face Award
Certificate from Minister of Education, Culture, Sports, Science and Technology Trophy
200,000 Japanese Yen

Special Achievement Award
Certificate from Minister of Education, Culture, Sports, Science and Technology Trophy

Application Period 2011
The receipt of entries for the 15th Japan Media Arts Festival closed on September 22, 2011.

Judge, Announcement of Results
October through November: Jury Judge
1. Notification of winners
2. Announcement of Results of the Japan Media Arts Festival website and in the mass media (TV, Newspapers, Magazines)

Awards Ceremony
Tuesday, February 21, 2012
Tokyo Midtown

Exhibition of Award-winning Works
Wednesday, February 22 through Sunday, March 4, 2012
The National Art Center, Tokyo

Previous Winners:


Wednesday, October 19, 2011

Beware of the Shadows

Published by Amy at 1:28 PM

Team: Laura Murphy, Cameron Coker

Laura: VVVV, Shadow Creature Rig, Compositing
Cameron: Man Model, Man Rig, Animation, Compositing
Amy: Shadow Creature Model, Texturing, Animation

Laura's Blog
Cameron's Post

Our project idea sprouted from our first projection mapping exercise several weeks ago.  Laura liked incorporating shadows, so we continued to work with them for our second project.  We initially had issues finding our "space" to project onto, but thankfully Hwaryoung suggested an interesting place on Building A of the Architecture school.  She suggested at first that we project from our studio down onto the space.  Unfortunately, the glass created too much reflection.  We re-arranged ourselves on the breezeway.  The space is the second story on Langford A, above Azimuth.  The space works well because there is not any interior lighting that interferes with the projection.  It is also visually interesting at the angled perspective.  

Laura and Cameron originally played with the idea of positioning doors on each of the panels/slats of the building.  We continued with the idea for our project, but provided two interesting characters: the man and the shadow creature.  The three of us decided it would be interesting to make the project interactive with movement (webcam) even though it was not required.  Through quick discussion, we decided on our final concept:

A silhouette of a man walks through a series of doors.  From the shadowed areas, a shadow creature stalks the man though reveals itself gradually.  If there is no movement in the area of the web cam, the man will get eaten by the shadow creature.  If there is movement, the shadow creature spooks and darts back into the shadows.  

Technical Background
We used the program VVVV for the projection mapping.  Laura created the transformations for the walls using a function that takes the desired output corners.  VVVV has easy functionality for displaying videos.  Unfortunately, it only reads certain file types with certain codecs.  Laura created functionality to randomly generate the action queues for the characters. For more details on how Laura created the program, see her blog:

Laura's Blog

Since I decided to use ZBrush for modeling and 3D Coat for retopologizing (two programs I was unfamiliar with), I had to do some research.  I looked up several tutorials that gave me guidance on each of the programs.  Thankfully, Jarrod Simpson is rather familiar with ZBrush.  He was also able to help me immensely and lead me in the right direction. 

Laura, Cameron, and I did additional research on the movement of our shadow creature.  Randall from Monster, Inc. was used as a reference for the movement of the creature.  Laura found many, many videos of lizards.  She has a post dedicated to her reference videos:

Shadow Creature

For look development, one of references was Limbo.  Cameron felt the silhouette of the man needed eyes so that the audience could connect with the character.  My design for the shadow creature was influenced by Insanely Twisted Shadow Puppets.  

Insanely Twisted Shadow Puppets

Cameron, Laura, and I worked in a partial non-linear path.  Cameron and I were responsible for the models and animation while Laura created the program for our interactive projection mapping.  Cameron and I followed the general pipeline by starting with modeling, rigging, and animation.  Cameron modeled, rigged, and animated the man.  He also did the eat animations.  We have four different types of animations for the shadow creature, with additional variations of those four types.  There are peaks, advances, eats, and spooks.  These are videos that are randomly generated through Laura's program.  By the time Cameron and I were animating, Laura had already completed majority of the program.    We just needed to implement our rendered videos into VVVV.  Laura also did the rig for the shadow creature after she created the program.  She made dual offset SINE Modifiers for the rig so the shadow creature could move like my original concept drawing.  She alternated working between the rig and the program a couple of times.  After we finished majority of the program, we had to render and composite the animations in Adobe AfterEffects.  Laura created a calibration image so we could properly align and measure the panels when we composited.  The calibration image was originally created to make sure they were sized correctly on each panel.  

Image used for compositing arrangement

The three of us ran into several issues, each individual and different.  I ran into several issues since I decided to use ZBrush to model.  As of two weeks ago, I had no experience working with the software.  It was a bold move on my part to attempt to use ZBrush for a two week project.  It was challenging, but I partially now know an additional program I could potentially use in the future.  I found 3D coat's auto-retopology function the most difficult challenge for me.  I was even more unfamiliar with the software and re-topologizing a mesh.  The function was able to partially retopologize the mesh, but it wasn't exactly acceptable for animation purposes.  Laura and I combined two separate meshes for the final model.  The low resolution base mesh from ZBrush was used for the body, and the next highest resolution head was used.  Laura skinned the separate meshes together.  

Emergent Narrative
The narrative becomes important when the audience recognizes their role in the interactive project.  Their role is first not evident until they recognize that their movement has an impact on the animation.  If they stand still, the man will get eaten.  If they create movement, they will spook the shadow creature and save the man from death.  At this point, the audience becomes part of the narrative - they determine the man's fate.  Interesting enough, we found that the audience desired to see the man get eaten by the shadow creature.  

Future Improvement
In the future, we would like to improve several characteristics.  I would definitely like to do additional peak, advance, eat, and spook animations.  We have on average about three different animations for each currently.  We would also like to improve the interaction between the audience.  We would need additional lighting so the web cam could recognize movement better.  A more fluid spook system would enhance our project.  Though it is interactive, we all agreed that audio or sound effects would make the project increase suspension of disbelief.  The audience also did not immediately understand their role in the narrative.  My team and I would need to create a more obvious role for interaction.

I wish we could have had more time on the project, but I still found it exciting and different.  This was the first time for me (and majority of us), to do projection mapping.  This project has broadened our knowledge and portfolio/demo reel.  I am very happy I partially know ZBrush now.  I plan on doing some personal work in the near future with ZBrush.  


Sunday, October 16, 2011

Shadow Creature Cont.

Published by Amy at 8:15 AM

It's been a while since I've updated.  Here's what's new:

I finished the shadow creature model last Sunday and used 3D Coat to retopologize the mesh.  3D Coat has an auto retopology function that uses manually created guides on the mesh to help with edge flow. For the most part, Auto-retopology works.  I ended up redoing the function about 10 times just to get a decent mesh I could work with.  I took the mesh into Maya and fixed what I could.  I found several tris and holes within the mesh, which were difficult to fix.  
Laura and I decided to use the lowest subdivided mesh from zbrush for the body, and the auto-retop creature's head.  Laura is finishing up the rig at the very moment I believe.  I will be creating most of the animations for the shadow creature.  I created some blend shapes that will be utilized in our animation as well.

Using guides in 3D coat
What beautiful geometry
Blend Shapes

Sunday, October 9, 2011

Shadow Creature

Published by Amy at 12:17 PM

I decided to create the shadow creature in ZBrush.  Even though we do not have much time to complete the project, I feel this project is a good opportunity to experience a new program.  I started with Zspheres to create the basic frame of the creature.  My first model did not include the toes, so it was rather difficult trying to sculpt toes with not enough geometry.  I had to go back to a previous file and add the toes, along with the spikes on the body before I could sculpt.

Zbrush has several different materials available to the mesh when sculpting.    To check the creature's silhouette,  I tried out the Outline material. 
ZBrush also has some fun brushes to experiment with.  I tried out the Snake Hook Brush on the creature's spikes.

Monday, October 3, 2011

Project 2 Concept

Published by Amy at 1:15 PM

On Saturday night, Cameron and Laura tried out the suggested space on Building A (southwest side, by TRC).  Instead of projecting from our studio space (as discussed last Thursday), it was easier to set the projector on the breezeway's bridge area.  Laura and Cameron did a test projection of doors on the angled space.

On Sunday afternoon, Laura, Cameron and I established our concept.  Laura really liked the idea of including shadows, as we did in our previous exercise. We continued with the door idea between each panel/slat, but we decided to incorporate a shadow creature along with interaction.  

Projection: There will be a silhouette person walking between doors (located on each panel) as a shadow creature stalks him.  The creature will appear from the shadows and will try to engulf the person, unless there is movement around the area.  The creature will spook and hide back into the shadow/shadows.  As an audience, their goal is to prevent the person from being eaten by the shadow creature by creating movement. 

I got to design the shadow creature.  As of now, I plan to create and animate the creature with AfterEffects using the Puppet tool.  It was inspired by two main references: Randal from Monsters, Inc. and Insanely Twisted Shadow Puppets.

Laura, Cameron and I tried out our projector for the second time last night.  Laura made a diagram of our concept, so we decided to project her image on our wall to check perspectives and shadows.  We originally were going to project on the opposite side (left), but I noticed that the TRC's lights from the windows may interfere with our projection.  I decided to record our mayhem.  You'll have to excuse Laura.  She had a little too much coffee. 

Here is a quick animatic of our concept: