SDN - Comp 150-TUI

Reflections on Project

2008-05-07T17:07:00.000-07:00

Overview:

This was a very interesting project and I learned a lot from it. It turned into a bit of a nightmare for me in terms of the amount of time I needed to put in, but I enjoyed doing it and it was very satisfying to get it to work in the end, at least in the limited fashion that it did. The main issue was that there weren’t enough computer scientists on the team to handle the amount of serious development that was required. Courtland and Juno were both very helpful with the tasks that fell into their skill set (building things for Courtland, more basic programming for Juno), but this project required a lot of difficult programming (graphics, advanced object-oriented programming with complicated interaction between objects, etc.) that only someone whose discipline was computer science would be able to do effectively. Again, I enjoyed the project a lot, but having to put several hundred hours of work into a project is a bit too much.

Development Challenges:

The main development challenges, in order from most difficult to least difficult, are as follows:

1. Implementing the network of roadway objects and car objects with all the necessary interaction between various roadways, cars, connectors, etc. This required a lot of advanced programming time.
2. Hard-coding the graphics for the cars. This was tedious work that was also not easy to do because graphics are tricky and without taking a course in it, concepts like Affine Transformations are hard to grasp. This also required a lot of programming and caused us to scale back the number of roadway types we would support.
3. Getting the cars to drive in the right direction and on the right side of the road in all possible configurations. This required a bunch of additional testing and debugging to get right.
4. Basic mapping of physical tokens to exact digital replicas of them in the system. It wasn’t too hard to draw the tokens’ shapes, but getting them all to appear in world space with the correct position, orientation, and size took some thought.
5. Getting Mike’s custom webcam driver to work with our system. This was important for getting better results from the topcode reading, and required some time to get to run on the computer we were using and integrate into our code.
6. Dealing with lighting and glare with the webcam to get the topcodes reading consistently, even when the projector was running. This got a boost from using Mike’s webcam driver, and the projector issue was solved by placing window tinting film on the underside of the glass and directly lighting the topcodes from below.
7. Calibrating the output image of the program to the physical tokens on the table. This was tedious work – viewing the image and making small changes to the global position and zoom values to move the image closer to where it needed to be on the table. This was also dependent on the positioning of the projector and the camera.

Things that would have helped:

Really there is only one here – more computer scientists. The skill set of the group was nice and diverse, and I felt we were prepared for everything involved except pure software development. The project required too much advanced programming to have only one person with a software background.

Team Work Process:

a. The main tasks were:
-Building the table (not sure, Courtland knows)
-Building the start tokens (not sure, Courtland knows)
-Putting the labels and topcodes on the roadway tokens (10 hours)
-Coding the topcode reading (5 hours)
-Coding the graphics (150 hours)
-Coding the system’s interaction and the cars (150 hours)
b. Work was divided based on what people were capable of doing. We divided everything as evenly as possible, but there were certain tasks that some of us knew how to do and others didn’t, and that pretty much dictated who did what.
c. The creation of tokens with fixed positioned topcodes was done first, because it was necessary to have a permanent topcode position and orientation to work with on each token. Then we coded the topcode reading and basic graphics followed by integrating those two parts together. After that we started developing the system-wide interaction and the cars. The building of the table and the start tokens was done towards the end.
d. The integration wasn’t really an issue. The only two times we needed to integrate separate components were integrating the topcode reading with the basic graphics, in which case the two components were designed to work well together, and integrating the final software component with the table and physical objects, which wasn’t too difficult outside of some last-minute calibration.
e. Testing was done within each task to confirm that that component worked as it should before integration. Once everything was put together, more testing was done to work out issues such as cars driving in the wrong direction and on the wrong side of the road. Testing of the camera to deal with lighting and glare was also done.
f. We had to scale back the project quite a bit. In order to save time, we didn’t implement functionality for 3 types of tokens that we had originally planned to support. We also didn’t get the cars to work to the point we had hoped to. We got single cars to move through any system that the user created, but we didn’t get to making the cars aware of each other and having their speeds be affected by the tokens they are traveling through so that traffic can jam up in the system. The useful traffic modeling was too much to implement in the time frame we were working with.
g. Our team dynamics were quite good. Even though Juno and Courtland didn’t have the programming experience to help with much of the development, they contributed where they could and tried to help me as much as possible. I understood that it wasn’t their fault that they didn’t have a programming background. So even though there were difficulties with breaking up the work, I thought the team dynamics were good.

Summary of Development Process

2008-05-07T14:04:00.000-07:00

Here is a summary of the development process for the Traffic Sim project:

I started by learning the basic drawing capabilities of Java2D, like how to draw a rectangle with certain dimensions in a certain location, etc. Once I got that stuff down, I created exact to-scale replicas of each road piece using these basic shapes, with their centers at 0,0. Eventually I was able to draw each piece to scale.

Once the basic graphics was implemented, I needed a way to put it all together. Therefore, I made java classes for each type of road, each with a draw function that would do the graphics stuff I had previously developed for that piece. Each class also had a way to internally store that piece's position, orientation, and size.

I also created a thing called an interface that acts like a generic roadway which each of the other classes implement. This allowed me to have the global system accept any type of roadway without having to specify a specific class.

After all of the pieces had classes, I needed a global picture that could draw many pieces in their correct position, orientation, and size. For this, I used things called Affine Transformations, one of which each piece's class contained. To draw a piece, I would first remember the original transformation of the global picture, then transform into the space of the piece I wanted to draw. Once the drawing was done, I would return to the original global space, which resulted in the piece being in its correct position, orientation, and size.

At this point, I could have a list of tokens and iterate through the list, drawing each token in its correct position on the global image. I then integrated the Webcam code into this, fetching the list of topcodes that the webcam read and transforming them into a list of token classes that could then be drawn. Each topcode had a position, orientation, and size that I used for the corresponding tokens.

Here I had a bit of an issue getting the webcam code and my code to work together, as the webcam code wasn't really designed to continuously give data to another part of the program. I got past this by having the webcam code run continuously in a different thread, and periodically asking it for its current topcode list.

Now the system could run continuously and keep an updated digital representation of a traffic system that the user had placed on the table. It was time to turn my attention to the cars. For this, I came up with a scheme where a car would have a set number of possible locations on each token, each with a hard-coded red rectangle that would be drawn at that location. Since the locations were relative to the tokens, the hard-coded values would move with the tokens as they were rotated and shifted around the table. The car would know which location it was at, and when it was time to move to the next location, it would simply update its current position variable and draw the rectangle for the new location.

Juno and I then started the incredibly tedious process of hard-coding graphics code for a rectangle at each possible position on each type of token. This took quite a while, but we got it done for nearly every token type. I now needed to implement the cars moving along the positions and drawing the rectangles as they went.

The first tricky part was handling when the car got to the end of a token and had to move to the next token. I had to create a network of connections, so that the connector of one token could tell the car what token it was connected to. I had the webcam read the locations of the connection points of each token, and if one was close enough to another, it would connect them. This meant creating a pointer from one to the other and back. Now, once a car got to the end of a token, it could ask the connector for the next token, set that to its current token, and set its position to 0. It would then move along in the next token until it got to the end, where it would repeat the process until it reached a connector which wasn't connected to another token.

The next tricky part was handling different possibilities for a car on a certain type of token. Depending on how a token was oriented and which direction the car or cars were coming from, the cars would have to go through the positions in different orders and be on different sides of the road. Fixing this was quite difficult and required some tricky math with various multiplication by -1 and different position offsets in each possible situation. Eventually I got it so the tokens could be placed in every possible way and the cars would drive in the correct direction and on the correct side of the road.

The final major programming tasks were making the system work with multiple cars at once (which created a whole additional bag of headaches) and getting the cars to move automatically with a timer, so that the user could press a button once and the cars would run through the system. Previously the user had to press a button for each car movement.

In addition to all the programming outlined above, Juno and I spent quite a bit of time trying to optimize the performance of the topcode reading, counter the interference the projector was causing, and align the projection with the physical objects. The performance of the topcode reading got a big boost by Mike and I putting a couple of hours into getting his custom webcam driver to work with our system, giving us full 2 megapixel resolution. I was able to solve the projector interference pretty well by using window tint film on the bottom of the glass and directly lighting the topcodes from below.

There were also countless smaller issues that would sidetrack me for a few hours here and there, but that was the major stuff. Hopefully this gives you a basic idea of my process throughout the extensive development that went into this project.

Reflection on project design process

2008-03-24T10:07:00.000-07:00

My group (consisting of myself, Juno, and Courtland) chose a traffic system simulator for our project. We have begun the process of designing the system and refining our design to a point where it can be implemented without ambiguity. Our design is as follows:

A finite number of road pieces that can be connected together to form a traffic system
Traffic flow tokens which start a flow of traffic into the system, each of which has an adjustable amount of traffic
Traffic displayed onto the system and interacting with the system in real time
Traffic simulation that starts when a traffic flow token is attached to another token, and stops when no flow tokens are attached to another token
Behavior of the simulation so when traffic reaches the end of a road, it fades out as to indicate it continuing on in that direction indefinitely
Traffic that is modeled as a flow with a set speed such that each type of road piece has an effect on the speed of the flow moving through it

Other possible design choices we could have made are:

Using a start button to begin the simulation instead of having it start automatically when the traffic flow tokens are attached to other tokens. We chose to avoid this as it doesn't seem to fit with the goals of a tangible user interface.
Behavior such that traffic would react to an end of the road, either as a destination or as a blockage. Also perhaps requiring the system to have sources and sinks on all ends. We chose to indicate continuation to avoid the complexity of requiring sources and sinks and to convey the idea that traffic systems are not self-contained, but rather they connect to other traffic systems in a larger continuous network.
Traffic modeled at the individual car level, or with a more complex and realistic representation than a simple flow. We feel that even with this very simplified model, this project will be a lot of work to implement. Therefore, we avoided more complex traffic representations to limit the possibility of having the project be too difficult to complete within the time constraints of this class.

Difficulties we encountered while designing the project were mainly confined to coming up with the ideas listed above, and deciding between the various options we had. There were many different ways to go with this project, so coming up with the basic assumptions was the most difficult part of the design process.

Here are some of the design sketches we came up with:

FAQ for the OBID i-scan RFID Reader/Writer

2008-02-27T17:34:00.000-08:00

Here are some FAQ for anyone trying to use the OBID i-scan:

Q: The hardware is connected correctly and the software is completely installed, but when I run TestRFID.bat it tells me it can’t find the reader. What is wrong?

A: You must have a tag on the reader in order for it to be detected. Place any tag on the reader and try again. Keep this in mind when you are using other programs as well.

Q: I created my application using and IDE like NetBeans or Eclipse and it generated a dist folder with an executable jar file, but when I try to run my program using this jar, it tells me it can’t find any COM ports. Why is this happening?

A: Executables created in this manner will not see the COM ports correctly. Try writing a build.xml script and running your program using the script and Apache ant.

Q: I copied the comm.jar and rfid.jar files into a folder that my program looks in to find them, but when I run my program it tells me it can’t find any COM ports. What is wrong?

A: You need to also copy the javax.comm.properties file into the folder with comm.jar and rfid.jar. If javax.comm.properties is not included with the libraries, the program will not be able to see the COM ports.

Hopefully these will be helpful for anyone trying to use this tool. Along with many more instructions, this FAQ is also contained in my Quick Start Guide for the OBID i-scan.

Technology Demo

2008-02-25T10:13:00.000-08:00

Recently I have been working on a technology demo assignment. For this assignment, we each had to take a piece of essential TUI technology, figure out how to use it, and design a simple system that would allow us to demo the technology to the class.

My technology was an OBID i-scan RFID reader/writer. This is a very cool piece of technology that allows you to identify objects by placing RFID tags on them. These tags come in the form of small, square stickers. Once an object is tagged, it can be identified simply by placing it on the RFID antenna attached to the reader. Tags can also be written to simply by placing them on the antenna.

For my demo, I wrote two programs. The first is a Virtual Hardware Store application, which can read tags for 5 different tools: a hammer, an adjustable wrench, a tape measure, a philips-head screwdriver, and a flat-head screwdriver. These have RFID tags on them with DSFID numbers 1-5. When a tool is identified, my software shows a picture of that tool on the screen, displays the item's name and price, and adds the price to a running total. The second program I wrote is a command-line utility that allows the user to write the DSFID number on an RFID tag. The user simply places the tag on the RFID antenna, inputs a number to write to the tag's DSFID, and removes the tag.

This assignment was fairly straightforward, but I had a bit of trouble with the portability of my applications. I found that when I compiled my code into executable jar files, my programs would not be able to access the computer's COM ports correctly. I therefore had to use Apache ant to run my programs. This created a portability issue as many computers do not have ant installed and it is not the easiest thing for a user to configure properly. I therefore decided to package ant with my software. I also had an installation issue where my software was packaged with several images, Java libraries, files associated with ant, etc. I had to then create an installer batch file to copy these files to a user's computer and create a batch file on the desktop which will run my two programs using ant. This portability portion of the assignment took my quite some time, but I believe I have my software to a point where it is very easy to install and use on nearly any Windows XP or Vista machine.

Assignment 1C

2008-01-28T10:09:00.000-08:00

For this assignment, I am asked to define a tangible user interface and to describe a TUI that I like.

I define a TUI to be a human-computer interface in which physical objects are used such that these physical objects represent underlying digital information. The objects can be interpreted and manipulated by the user as a method of interpreting and manipulating the underlying digital information.

As for a TUI that I like, I will use one from the presentations we saw in class. I liked the Urp urban planner interface because it offered a tool that can be useful to professionals, not a dumbed-down interface with limited functionality. I also liked how people with no specific training in computer tools can use Urp. When using tools like a CAD program, training would be required for manipulating the tool. With Urp, however, all stakeholders in the project can participate, as no knowledge is required other than the ability to understand a scale model. Everyone involved can stand around the table and manipulate the interface.

TUI Analysis Presentation

2008-01-16T19:06:00.000-08:00

Today I gave my TUI Analysis presentation on the Tangible Video Editor, a project of a previous TUI class at Tufts. This interface allows people to edit video using plastic tokens with small embedded computers. The goal is to combine the benefits of traditional, physical film editing with the advantages of digital, non-linear editing.

The main components of the interface are video clip-holder elements, transition elements, and a play controller. The clip-holders are contain small screens and have a "puzzle-piece" shape which makes connection obvious. The transitions use shapes which illustrate their function, with 3 transitions implemented: fade, rotate, and minimize. The play controller contains a play button and a jog wheel, and connects to a desktop PC which shows the output. The user can arrange the clips and transitions on a table, connect them in various combinations due to their puzzle-like shapes, anf play back different combinations by connecting them to the play controller.

Overall, I thought this was a neat idea. This interface does encourage group interation more than GUI software like Premiere or Final Cut, and it can create a more enjoyable editing experience. However, its functionality is severely limited compared to the software that is available today, and it would be very difficult to implement such functionality for this interface. For that reason, I can see this interface being a good fit for children and for adults who do not require advanced functionality and just want to create a simple edit. It would be a good solution for anyone who is not particularly computer literate.

An additional aspect of this interface that I liked was that the user can physically arrange the clips and transitions on the table, including preliminary and intermediate ordering and sorting. The user can immediately start to arrange the clips, and then continually re-order and refine the arrangement until the final arrangement is achieved. I think that this ability to continuously arrange and re-arrange is a huge advantage over GUI software like Premiere and Final Cut.

Inaugural Post

2008-01-16T19:00:00.000-08:00

Welcome to the blog of Stephen Normandin, Computer Science student at Tufts University. This blog fulfills a requirement for and serves as a record of my progress in the Comp 150-TUI Tangible User Interfaces course. I hope the information contained in this blog will be helpful to those who are interested in my work for this course.