Assignment 1

Due: Monday March 4th at 9am.

In this assignment, you will create a simple experience of some form that uses vision base tracking in Argon.  You will use one image target and at least 2  frame markers.  The experience will be fixed on the image target (i.e., you should write the experience assuming the image target does not move), and the markers will be movable.  You should imagine an interesting vision-AR-based experience, and then define a simple fragment of an experience that is evocative or indicative of the full experience.  You only need to implement the fragment that satisfies the technical requirements below.  The goal of this assignment is to give you practical and technical experience working with vision-based AR.

You should envision an experience, possibly a very very simple one, that takes place on a fixed 2D image target.  You could imagine this image being on the wall, the floor, a desk, a map, a book, etc.  The experience “lives” on that image target, in 3D, situated on and above it.   The image must be sized such that it can be printed on an 8.5 x 11 inch piece of paper.   The image must be appropriate for your experience, and you must not use one of Qualcomm’s samples.  You should choose an image target that achieves reasonable (3+ stars) tracking performance, and has features reasonably distributed over the whole image.

Your experience should use a mix of HTML5/CSS content (displayed via Argon’s Three.js CSS object) and WebGL-based content (either models, or simple shapes, or whatever is appropriate).  A few constraints:

  • There must be both CSS and WebGL content displayed in the space of the image target (e.g., in addition to anything attached to the markers).  This can include labels, elements used for content, and elements used for feedback.
  • You may use 2D elements attached to the screen, that depends on your experience.
  •  The content on the image target should react and change based on the movement of the camera in some compelling way.  For example, as you move closer to some content, perhaps it changes to reveal more detail.   Or, have content move to arrange itself based on the layout of the content relative to the camera (e.g., have a label move around so it does not occlude the thing being labeled).  Or, have a character (e.g., a monster) turn to face the camera when it is close enough.

The additional 2 or more markers should be sized such that they can be held easily, such as being smaller than playing cards.  You should use the interior of the frame markers to put whatever graphics or text you need to make it clear what these markers do, and what they are for.  They should be used in at least two different ways.

  1. as a token or trigger of some form.  Technically, what this means is that when a marker of this form is seen, it’s appearance is recognized by your code and some state changes as a result.  For example, in a game, it could be a power-up or result in the addition of a new ability to a game character.  In a advertising application, it could act as a coupon or purchasing code.  No content needs to be registered continuously with this kind of marker (one would expect it’s behavior to be drawn on the paper marker itself, which you could do inside the frame maker), but the experience on the image target should visibly react to the appearance of the token marker.  Whether or not there is graphics is up to you, as long as the combination of what you put in the frame marker and what you render virtually makes sense.
  2. as a continuous interactor.  Technically, this kind of marker should have 3D content on it (either Three.js CSS content in 3D, or Three.js WebGL content), and both this content and the content on the image target should be visible at the same time, and visibly react to the movement or location of the interactor marker as the user moves this marker relative to the image.  You will need to structure the physical setup of your experience such that the interaction works reasonably (given the fact that the marker needs to be fully visible, but also might occlude part of the image given the field of view of the camera, etc).

You will submit the following via a blog post:

  • A URL to your experience.
  • A brief description of your experience (both what you imagine the “real” experience being, and what you did), as well as step by step instruction for the professor to fully experience what you did.
  • A short video of your experience in action.

You must also submit

  • physical copy of the physical markers (sized as required, with any decoration you want to including on them) for your experience to the professor
  • a PDF file (again, set up so that if it is printed on 8.5 x 11 inch paper, everything is the right size) so they can be printed again if necessary via t-square.
  • In the description part of the t-square submission, include a direct link to your blog post

Grading will be as follows:

  • Display non-trivial 3D content (HTML and WebGL) relevant to your experience, in the space of the image target.  (3/10)
  • Handle tracking gracefully, in particular when the image target or a marker is lost for a small amount of time due to transient occlusion or computer vision errors.  Feedback and content should reflect such losses appropriately. (1/10)
  • Have the content change in compelling ways based on the movement of the camera and it’s relationship to the image target (2/10)
  • Have content on fixed marker affected by trigger marker(s). (1/10)
  • Have non-trivial 3D content on interactor marker(s). (1/10)
  • Have content on the image target and the interactor marker(s) react to relative movement or location of the interactor relative to the image target. (2/10)
  • bonus:  use the orientation of your device to determine the real-world orientation of the image target to the real world, and use this knowledge in your experience in a compelling way (1/10).