Skip to content

Mapping GPS Data

When I last left the data logger project off I was having quite a few issues with managing the size of the data in the allotted EEPROM space on board the Arduino. I built up an SDCard writer based on several specs I found in the messageboard and it worked flawlessly. I successfully captured RPM data from my bicycle commute. The problem was, I had no way to correlate one collection of data to the other. Different routes, days, time, weather could all affect the results of the dataset.

It was never on my list of things to do since I knew the battery usage scenario would be so high, but I figured I’d bite the bullet and go the GPS route anyways. I picked up LadyAda’s GPS Sheild and have had a wonderful time working with it ever since.

I had been working on a way to parse the NMEA string data from the GPS device delivers when I figured it would be a perfect way to try out the Google Maps Flash API (my previous experience had only been using the JavaScript API, and Yahoo Maps API) and Flash Player 10. Flash Player 10 offers a number of new features, one of which being the FileReference API which allows you to browse for a file on your local machine and load it into the Flash Player without making a round trip to the server. Once loaded I split the text file based on the carriage return and cast all of the GPS Points to a Vector (another new feature — typecast Array. The speed difference is quite staggering!).

The end result is plotted on a map and colour coded based on velocity, which is recorded in Knots and translated to Km/h. The end result is a colourful chart of the route I commuted that particular day.

In the screenshot above, note the big red blotch at Allen Gardens — flat tire. The other red marks are stop lights or streetcars.

An example of the map in action requires some coordinates, so save these text files to your machine first (contains a couple hundred lines of NMEA data), and hit the load button in the map application. Once loaded the map should recentre on the coordinate boundary.

I’m currently working on some new modifications for the map so I should have more available soon. But first I’m putting the finishing touches on a wooden case I’m hand crafting as a shell for the GPS device. I’ll have photos up from that project soon.

A Maker’s deep-dark secret

I’m just going to put this out there, I feel its something that isn’t talked about often and can be an influencing factor when making things. Building things yourself will almost certainly cost you more than if you buy the object in the store (if it exists already). On almost every single project I’ve built I’ve gone over budget… not that I’ve had a budget, but if my budget was set it would probably be at or less than the cost of buying something in the store.

  • LED Lighting system for my bike = $75 store bought lights
  • Solenoid project = less than the professional units, but more than I was planning on spending
  • GPS unit = equivalent to a Garmin device
  • Multitouch table = less than Microsoft’s Surface, but can I call it completed?
  • Wooden toys for my daughter = more than it would cost from a mass factory in China (but minus the lead additives)

The point is, we shouldn’t be choosing to build these projects to save money. Looking back on things, I did think I could do it cheaper by doing it myself. Chances are you will get a less polished result, and more expense trying to fix things that went wrong.

But buying something in a store won’t get you the experience, the inspired creative ideas, or abilities gained out of doing it yourself. And in my case those reasons far outweigh the immediacy of running out to the store and adding an item to your collection.

Conversation Clouds with Physics

So after posting the previous entry I went to sleep, only to find myself thinking about improving the visual aspect of the Conversation Cloud. I decided that adding physics would be the way to go, it would allow words to deflect off of each other, grow and gain mass, and constrain themselves to a container.

I’ve used APE for the physics engine, and Tweener for some of the elasticity when the mass increases.

Conversation Clouds

Word of mouth is one of those impossible things people like to track. The best people seem to be able to do is to ask the question on a registration form, “Where did you hear about us?”

Is it possible to track word of mouth any better? This is the premise of my latest project, Conversation Clouds.

Imagine a conference on multiple floors, with thousands of people. You stand at a screen visualizing all the conversations going on simultaneously. And there on the screen is the general location of a topic you’re interested in, so you venture down a floor and join in.

Now, privacy concerns aside, the system works as such:

  • Microphones are placed around a physical space. Each is marked with an id which relates to its geographic location.
  • The audio is transmitted to a server which performs voice to text translations in small batches.
  • As each sample is translated it pushes to the visualization script which then displays on a central monitor.

Get Adobe Flash player

In this early proof of concept I’m using recorded conversations left through answering machines which are transcribed into textual form. Once a new message arrives the script parses it through a custom UIComponent in Flex. Words are added to a hashmap and as repetition occurs the visualization increases the scale and opacity. With future versions I hope to add in a decay parameter as well, so as the conversation flows from one topic to the next so does the visualization. (In the demo above I’m using the content from a previous post.)

Drawdio - Cool Toy of the Day

The cool toy of the day has to go to Drawdio. You draw an picture and it connects a circuit through the graphite in the pencil. It then plays back the sound through the speaker.

The only complaint I’ve heard is that it sounds like a dentist drill. I fully intend on using this all day Monday at the office.

Really straightforward kit, I’d recommend it to anyone that has a budding interest in electronics. At $20 it’d sure make a unique gift.

Nuit Blanche 2008

What struck me about the pieces shown during last night’s Nuit Blanche was that all the ideas were entirely doable by any person. In fact I feel that way about most art pieces. Its the idea that matters, and frankly, it doesn’t have to be an original idea to make an impact. Ideas and memes spread so quickly, to credit them with someone’s name seems almost impossible. This isn’t even a “in this day and age” concept, we still argue about whether the Wright brothers invented the airplane, or Bell invented the phone first. Watch Blinkenlights in the video above for example. What is the critical difference between this presentation and a LED matrix controlled by a computer?

The critical difference is the wherewithal to attempt a project this big. The scope of this project was aweinspiring but the technology (in concept) was simple. The mechanics and performance where anything but simple, but it was done so effectively it left you with that same feeling you get watching elite sports atheletes perform. Like magic.

This piece was inspiring, not because I found the artistic statement to be profound, but because I saw the look in the people’s faces as they watched the presentation. That amazement is something I want to strive for in all my works.

Data Logger — Next Steps

The Arduino EEPROM is a useful thing to have available for those times when you need to store small samples of data. In my initial data logger prototype I managed to record the Hall Sensor data into two bytes worth of space per minute interval. This gave me around 4.25 hours of recording data.

What if you wanted to record more data, or at an increased resolution? Adding another sensor would further reduce your available space depending on how many bytes the recorded data would require. Its time to look at other methods of storing data.

  • An I2C EEPROM with greater storage capacity connected to the Arduino using the wire.h library - couldn’t readily find an IC from any local vendors
  • USB Stick - got these sitting around, but the Arduino won’t act as a host device. Viniculum makes such a device, but it would require an additional purchase and possibly a bigger form factor
  • SD Card - readily available, just had to rip apart a cheap card reader for the port
SD Card Writer

I’m coming to the realization that the more difficult phase of this project is making sense of the data I’m collecting. I’m currently working on an application which monitors the data and charts and maps the correlations, but the collection of the data needs to be more seamless. I’m currently looking at a number of services, so I’ll be sure to write about them later.

Presenting at the Toronto Flex User Group

Just a quick note that I will be presenting at the Toronto Flex User Group on Thursday October 23rd, 2008 at the Oakham House on the Ryerson Campus, starting at 6:30pm.

Oakham House
Room: Oakham Lounge
55 Gould Street
Toronto, Ontario
M5B 1E9

I’ll be presenting: ActionScript or Flex Project? Development lessons learned building client projects.

Data Logging to the Arduino EEPROM

Data Logger Schematic

I made the first successful run with the data logger yesterday. I’ve wired up a hall sensor to the wheel of my bike and recorded 60s intervals of my ride into work and back. Ideally I’m looking for increased resolution over a typical bicycle computer; where a low cost CatEye gives you average speed over the course of a total ride, I’d like to see how that average speed changes in increments. I recognize I wouldn’t be able to compete with a device such as a Garmin, but I could definitely improve on the $30 devices you get at bike stores.

The first issue was how to deal with the intermittent pickups of the Hall Sensor. If the sensor fired too rapidly and the loop method wasn’t in the right position you wouldn’t pick up the digital I/O change. The solution, offered by Blalor, was to use the Arduino Interrupt commands. The Interrupt fires whenever a digital I/O signal is received. In this case I receive a 1 every loop. I’d be looking for a 0, so I’d set the Interrupt to look for a FALLING attribute.

The EEPROM was the next issue. 512 bytes is a serious limitation. Some math was required to see if I what wanted to do was even possible. If I were to poll every minute over a ride I’d be able to record 8.5 hours of one byte readings. That’s pretty good, seeing a 200km ride would take approximately 6 hours.

One byte is capable of storing 256 possible values. Is that enough, or would I need more values? If my revolutions were 255 with a wheel circumference of 2110mm I’d be traveling an average speed of 32km/h. Clearly not enough. I’d need to peak at 70km/h, with an average at maybe 30-40km/h.

Is it possible to compress a value of over 256 into a byte? Neil sent along an interesting piece of code on Octet Packed Integers. This code is a little beyond my comprehension so I gave it a pass. Instead it was decided to use a second byte. This allows a max value of 65535 rotations to be recorded — and at a circumference of 2110mm a max speed of 8257km/h. Well over the speed of sound! That should give me ample room to accelerate.

Problem is the second byte reduces my overall recording time to 4.25 hours. I also decided to remove 2 bytes from the array so I could track my counter when the device was turned off. When you start it back up it starts recording at the next address from where you left off. One possible solution to the space issue is to externalize the EEPROM and use the wire.h library to communicate. This would also reduce the issue of each address of the EEPROM only be written (and guaranteed) 100,000 times.

Neil also helped with the splitting of the rotation counter across two bytes. We took the full rotation count and masked it out with a hex value of 0x00FF and stored that as an int low. Then the full rotation count was bit-shifted right ( << 8 ) and stored as int high. When the report is generated after the ride the values are recombined and reported back to the screen.

I also added in three indicator LEDs for the Hall pickup, reset button, and report button. The reset button when held down for three seconds deletes all the data on the EEPROM,  anything less than that will generate a report of the current EEPROM memory.

I’ve included the code below, any suggestions on improvements would be much appreciated.

#include 
 
#define PIN_SENSOR_HALL 2 //Hall Sensor
#define PIN_LED_HALL 13 //Hall indicator LED
#define PIN_LED_RESET 6 //Reset indicator LED
#define PIN_LED_RESET_COMPLETE 5 //Reset complete indicator
#define PIN_BUTTON_RESET 7 //Reset/Report Button
#define TIME_INTERVAL 60000 //EEPROM write interval (60s)
#define MEM_SIZE 510 //EEPROM memory size (remaining 2 bytes reserved for count)
 
long countReset = -1;
long countCurrent = -1;
int countEEPROM = 0; //EEPROM Address Count
volatile int countRotation = 0; //Rotation Counter
 
void setup()
{
  pinMode(PIN_SENSOR_HALL, INPUT);
  pinMode(PIN_LED_HALL, OUTPUT);
  pinMode(PIN_LED_RESET, OUTPUT);
  pinMode(PIN_LED_RESET_COMPLETE, OUTPUT);
  pinMode(PIN_BUTTON_RESET, INPUT);
 
  Serial.begin(115200);
 
  countEEPROM = EEPROM.read(512); //Set counter to last known free space
 
  reportEEPROM();  //Report all EEPROM entries
}
 
void loop()
{
  digitalWrite(PIN_LED_HALL, LOW);
  digitalWrite(PIN_LED_RESET_COMPLETE, HIGH);
 
  //Write to EEPROM at specified interval
  if ((millis() % TIME_INTERVAL) == 0)
    writeEEPROM();
 
  //If Hall Sensor triggered then init the test
  attachInterrupt(1, incrementRotation, FALLING);
 
  resetEEPROM();
}
 
void reportEEPROM()
{
  Serial.println("EEPROM Report: ");
 
  Serial.print("[");
  for (int i = 0; i &lt; (MEM_SIZE - 1); i = (i + 2))
  {
    int h = EEPROM.read(i) &lt; &lt; 8;
    int l = EEPROM.read(i + 1);
 
    Serial.print(h + l);
 
    if (i != (MEM_SIZE - 1))
      Serial.print(",");
  }
  Serial.println("]");
}
 
void incrementRotation()
{
  countRotation++;
  digitalWrite(PIN_LED_HALL, HIGH);
 
  Serial.print("Rotations: ");
  Serial.println(countRotation);
}
 
void writeEEPROM()
{
  byte low = countRotation &amp; 0x00FF;
  byte high = countRotation &gt;&gt; 8;
 
  Serial.print("EEPROM Address: ");
  Serial.print(countEEPROM);
  Serial.print(", High: ");
  Serial.print(high, BIN);
  Serial.print(", Low: ");
  Serial.print(low, BIN);
  Serial.print(", Stored: ");
  Serial.print(low + (high &lt; &lt; 8), DEC);
  Serial.println(";");
 
  EEPROM.write(countEEPROM, high);
  EEPROM.write(countEEPROM + 1, low);
 
  if ((countEEPROM + 2) &lt; MEM_SIZE)
    countEEPROM = countEEPROM + 2; //Increment the EEPROM Address
  else
    countEEPROM = 0; //Loop back to start address
 
  EEPROM.write(512, countEEPROM);
 
  countRotation = 0; //Reset the rotation count
}
 
void resetEEPROM()
{
  if (digitalRead(PIN_BUTTON_RESET) == LOW)
  {
    digitalWrite(PIN_LED_RESET, LOW);
 
    if (countReset == -1) //Start Reset counter only if -1
      countReset = millis();
  }
  else
  {
    digitalWrite(PIN_LED_RESET, HIGH);
 
    if (countReset != -1)
    {
      countCurrent = millis();
      if ((countCurrent - countReset) &gt; 3000)
      {
        Serial.println("Reset");
 
        for (int i = 0; i &lt; MEM_SIZE; i++)
        {
          EEPROM.write(i, 0);
        }
 
        digitalWrite(PIN_LED_RESET_COMPLETE, LOW);
        delay(1000);
        digitalWrite(PIN_LED_RESET_COMPLETE, HIGH);
      }
      reportEEPROM();
      countCurrent = countReset = -1;
    }
  }
}

The post ride report looks something like this (values are in rotations per minute):

EEPROM Report:
[-31156,310,332,300,392,306,216,598,228,516,390,282,254,354,
496,198,436,60,254,263,400,382,280,394,362,340,300,306,256,
380,354,297,284,20,72,378,486,450,104,398,522,372,370,133,366,
360,164,336,460,542,488,366,396,384,512,484,300,300,332,346,
344,360,398,226,360,390,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

There appears to be some irrelevant data in the mix (see the array item 0 for example) which I haven’t quite figured out. The numbers don’t quite calculate to the precision I expect from a cycle computer. I have a feeling the Hall count is firing multiple times per revolution — like it has a FALLING state more than once.

Here is a quick JavaScript rendition of the data in a bar chart format. The RPMs are converted using this equation:

(((rpm * wheel circumference) / 1000) / 1000) * 60 = km/h

The next phase is to gather data on multiple days for the same course and comparing the results for analysis. I’m looking at developing an Adobe AIR Project which would allow me to keep an offline/online database. Getting real time data is another big incentive and I’d be looking at the XBee solution for that. Being able to record heart rate data and GPS/Digital Compass are also interesting but the limitations of the EEPROM don’t easily allow for recording of data that complex. So I think hooking up an SDCard is the next most probable option. I’ve already tried a USB adapter but the Arduino does not have USB host capabilities, so that didn’t work. There are options to get the USB host to work, but it involves purchasing another PCB. Finally, I don’t think any of this makes much sense without introducing a mapping API to add context to the data. So there’s still lots more to come!

Data Logger

Been working on a data logging application lately with the intentions of recording my biking sessions and displaying them on a mapping API. The general idea is to use the Hall sensor common in many RPM applications to record revolutions to the EEPROM onboard the Arduino.

I’m having a few issues getting the readings I need, namely the revolutions are not counting with any consistency. I think the issue is with the way Arduino loops through the script. Below is the first iteration of the code I’ve written.

NOTE: Script updated August 31, with Blalor’s suggestions. The RPM reading now works with a high rate of speed.

int pinHall = 2; //Hall Sensor
int pinLed = 13; //Indicator LED
int time = 10000; //EEPROM Write Interval
volatile int rpm = 0; //RPM Counter
 
void setup()
{
    pinMode(pinHall, INPUT);
    pinMode(pinLed, OUTPUT);
 
    Serial.begin(9600);
}
 
void loop()
{
    //Write to EEPROM at specified interval
    if ((millis() % time) == 0)
        writeEEPROM();
 
    //If Hall Sensor triggered then init the test
    attachInterrupt(1, incrementRPM, FALLING);
}
 
void incrementRPM()
{
        rpm++;
        digitalWrite(pinLed, HIGH);
 
        Serial.print("RPM: ");
        Serial.println(rpm);
}
 
void writeEEPROM()
{
    noInterrupts();
    Serial.print("EEPROM: ");
    Serial.println(rpm);
    rpm = 0;
    interrupts();
}

It appears to me like the checkReading() method isn’t firing with any consistency and therefore I’m not incrementing rpm properly. The writeEEPROM() method is firing on target, however I worry that the writing process may be delaying the rpm count.

Below is a Fritzing illustration of the circuit I’ve wired up:

Setup of the Data Logger

Suggestions? I’m open to any hints someone might have on how to structure this a little differently.