final report - university of floridathe single byte data into two bytes that include an op code. low...
Post on 16-May-2020
2 Views
Preview:
TRANSCRIPT
EEL 4924 Electrical Engineering Design
(Senior Design)
Final Report
25 April 25, 2012
Project Name: Keyboard Cam Team Smision: Donavon Bryan and Sydney Greene
Team Members:
Name: Donavon Bryan Name: Sydney Greene
Project Abstract
For this project we made a keyboard that is controlled by the user's hands and feet. We created a
peripheral that connects to a computer via USB. This device handled all of the note generation playback
and outputs through a 3.5mm audio jack. A camera connected to the host computer is also required.
Controlling the virtual keyboard requires a special board that can be strapped to ones shoes. The board
will light a red LED when the shoe makes contact with the floor. The PC runs a program written with
OpenCV libraries to determine what key has been pressed. The PC sends this data to the microcontroller
through USB. The area that the camera sees is divided into sections that represent keys. Depending on
the section you step on, different notes will play. These notes are digitally sampled tones from various
instruments and other sources such as a piano, guitar, pure sine waves, and a celebrity sound board.
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 2/28 Final Report: Keyboard Cam
Table of Contents
Project Abstract………………………………………………………………...…..1
Project Features/Objectives..……………………………………………………….2
Project Architecture…………………………………………………...……………3
Concept/Technology Selection……………………………………………………..6
Cost Objectives……………………………………………………...…………..…7
Division of Labor…………………………………………………………….……8
Project Schedule……………………………………………………………...…….8
Future Work………………………………………………………………...………8
Appendix A: PCB Layouts………………………………………….……….……..9
Appendix B: Microprocessor Code………………………………………………..11
AppendixC: Tracking Code………………………………………………………...
List of Figures and Tables
Figure 1- Block Diagram………………………………………………………….3
Figure 2- Tracking Vision Example……………………………………….………5
Figure 3- Picture of LED Strap……………………………………………………5
Figure 4- Picture of Main Board…………………………………………………..6
Figure 5- Gantt Chart……………………………………………………..……….8
Figure 6- Main Board PCB………………………………………………………..9
Figure 7- LED PCB………………………………………………………….…..10
Table 1- Bill of Materials………………..…………………………………….…..7
Table 2-Member Responsibility table……………………………………………..8
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 3/28 Final Report: Keyboard Cam
Project Features/Objectives
Ability to play music on an oversized keyboard with ones feet and no bulky mat or peripherals
besides a camera and LED shoe strap.
Different instruments selectable via a hand motion using a special colored glove.
Digitally sampled audio files stored on an SD card. Can easily be expanded to support more
instruments.
3.5mm audio output jack. Audio signal can be sent directly to speakers, mixing board,
headphones.
USB powered. No batteries required!
Real time response. Can play songs with no noticeable latency between key presses
Project Architecture
The solution involved 2 major components: the board that handles all of the audio playback and retrieval
and the player tracking system. Figure 1 illustrates the connection between the two systems and their
basic functions
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 4/28 Final Report: Keyboard Cam
Figure 1. Basic Block Diagram of the Entire System
Audio Playback: Computer to Microcontroller through USART
After image processing, the PC sends a two byte code. The bottom bit of the high byte will toggle the
instrument when equal to 1. The low byte contains a key in each nibble. The uC determines which key is
new and plays it. If another key was also just recently pressed (within a fifth of a second) a chord will
play.
Microcontroller SPI Communication with SD Card
Depending on the instrument selected and the key pressed, the uC retrieves a sound pattern from the SD
card. If the instrument is a sine wave, it takes the sine wave file and stores it locally to be looped for a
few seconds. For the other instruments, the uC takes the data from the SD card and sends it straight to
the DAC. We chose a SPI frequency that matched up with our audio sampling frequency so that the
audio would play correctly. The audio files were written raw to the SD card in advance by first
converting them from a .wav to a hex array in matlab and then pasting them into HxD, a freeware hex
editor.
Chords are available for the piano and guitar. A chord only plays if you press the second key very soon
after the first. This gives you the ability to quickly transition to a new key when playing a chord is not
your intention. The uC reads chords the same way as regular keys but they are paged in a different
fashion. They were also stored with an algorithm instead of manually. This method was necessary
because you cannot read two memory locations on an SD card at once and the audio files are too large to
be saved to the uC.
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 5/28 Final Report: Keyboard Cam
Microcontroller USART SPI Mode Communication with DAC
Using the second USART peripheral is Master SPI mode, the uC sends audio to the DAC. It separates
the single byte data into two bytes that include an op code.
Low pass Filter
The data leaving the DAC goes through a low pass 4.77kHz op-amp filter to smooth the jagged edges.
None of our musical notes are about 5kHz so this was a good cutoff.
Audio Amplifier
We used a power op-amp audio amplifier circuit to increase our output. The original DAC signal never
comes near the voltage rails in order to decrease distortion at the filter. The audio amp takes care of the
necessary gain.
Audio Output
We have the option of a through hole speaker and an eighth inch audio jack that you can choose between
with a physical switch.
Player Tracking The first part of this system involves the peripherals for the player’s feet. There is one board for each
foot and they are secured to the player’s foot with Velcro straps. Along the strap is a force sensitive
resistor that should be placed in the middle of the shoe’s bottom. This FSR is connected to one of the
inputs of a comparator circuit that has a set reference voltage on the other input. The resistance of the
FSR is inversely proportional to the force applied and so when it makes contact with the ground the
voltage on the pin will go down. This causes the LED at the output of the comparator to light up. A
diffuser was required for the LED in order to extend the range of play and make it easier for the camera
to detect it at reasonable distances.
The tracking software relies on the OpenCV computer vision library. It first searches the pixel by pixel
and removes all color that is not red. When it encounters a red pixel within the specified threshold it will
place a pixel of max value in that location. The process is illustrated in figure 2.
Figure 2. What the tracking software “sees”
Various smoothing and dilation OpenCV functions are applied to make sure the area of interest is
accentuated.
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 6/28 Final Report: Keyboard Cam
The program then searches the image for any nonzero pixels and places a box around it. The center of
mass of this box is calculated and compared to the division of keys and the note that was pressed is
determined. This information is then relayed to the board via USB-USART communication. For two feet
the process remains basically the same but extra steps must be taken to ensure that you are finding
distinct feet. This is done by searching the image in opposite directions. The algorithms only ever search
the play area in order to increase performance and it is assumed that the player will only ever play in the
actual play area.
Figure 3. Picture of LED shoe strap
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 7/28 Final Report: Keyboard Cam
Figure 4. Picture of Final Board
Concept / Technology Selection
This design was guided by minimizing cost while still providing the quality final product. Each
component and system that was used is cheap, cost effective and exactly what our solution required.
The microcontroller we used was an Atmega324p 8 bit microcontroller. This was the heart of the
audio part of our project and it was exactly what we needed. We required a total of 2 SPI ports
and a single USART. The 324p has 1 SPI and 2 USARTs (each configurable as a SPI in Master
mode). Our running clock frequency was fast enough to receive commands, read from SD card
storage, and play notes in real time which was a major concern during the design phase.
SD card storage was chosen because of their relatively cheap cost. It also enabled us to change
the instruments with very little effort and no special hardware or programmers. It can be done
independent of the microcontroller.
OpenCV afforded us a lot of processing power and tools to get the tracking working. It also
helped teach us a lot about image processing algorithms and standards.
The audio filter was an active filter with a cutoff frequency that allowed the the most common
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 8/28 Final Report: Keyboard Cam
frequencies to pass through.
Our particular LED/shoe board was designed in order to make our system as portable and
convenient as possible. Players can be set up to play the game in a matter of seconds and it does
not hinder the movement or gameplay at all.
Cost Objectives Keyboard CamParts list
Item Quantity Description
Unit Price
Total Cost
LM393 2 dual comparator 2.87 5.74
Atmega324P 1 8 bit microcontroller 5.36 5.36
LT1661 1 10 bit DAC 3.39 3.39
LT1632 1 dual OP-AMP 6.66 6.66
LM386 1 power amplifier 0.91 0.91
Speaker 1 8 Ohm speaker 4.33 4.33
10k Pot 1 resistor 0.92 0.92
10R 1 resistor 0.09 0.09
500R 1 resistor 0.09 0.09
1kR 1 resistor 0.09 0.09
100KR 2 resistor 0.09 0.18
910R 2 resistor 0.09 0.18
Red LED 3 red LED 0.43 1.29
270uF 1 polar capacitor 0.04 0.04
0.047uF 1 ceramic capacitor 0.08 0.08
10uF 3 polar capacitor 0.04 0.12
47nF 1 ceramic capacitor 0.08 0.08
20nF 1 ceramic capacitor 0.08 0.08 SD card Breakout 1 mount/ breakout for SD card 12.95 12.95
FDTI FT232RL 1 serial to parallel chip 4.5 4.5
FSR 2 force sensitive resistor 6.95 13.9
2kR 3 resistor 0.09 0.27
TOTAL
61.25
Table 1. Bill of Materials
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 9/28 Final Report: Keyboard Cam
Division of Labor
Member Responsibility Table
Item Sydney Greene Donavon Bryan
Audio Playback 100% 0%
Video Processing 0% 100%
USB Power 20% 80%
USB Communication 20% 80%
SD Card Communication 80% 20%
Shoe Lights 0% 100%
Altium 50% 50%
Microcontroller Programming 50% 50%
Table 2. Division of Labor
Project Schedule
Figure 5. Gantt Chart
Future Work
The tracking software could be refined a bit to make it more resilient and robust against the environment.
Target acquisition could be refined a bit and mad more accurate. This could allow for the keyboard to be
implemented like an actual digital keyboard with actual black keys instead of just same sized keys for all
notes.
Ideally the design could be optimized and redesigned with an embedded processor that could free the
project from a computer. This could be done with a higher end processor and efficient openCV code or
even an FPGA. This would require major revisions so the board and possible scrapped of the tracking
code if the FPGA route is taken.
The library of instruments/ sounds could easily be expanded with more time. More elaborate hand
controls could be developed that would allow possibly parameter changing of notes, special effects, and
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 10/28 Final Report: Keyboard Cam
other commands.
Appendix A
Project PCBs Main PCB
Figure 6. PCB for the main board
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 11/28 Final Report: Keyboard Cam
LED PCB
Figure 7. LED PCB
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 12/28 Final Report: Keyboard Cam
Appendix B: Microprocessor Code
Main Code:
#include <avr/io.h>
#include <util/delay.h>
#include <avr/interrupt.h>
#include <avr/pgmspace.h>
#include <SDCARD/SD_routines.h>
#include <SDCARD/SPI_routines.h>
#include <SDCARD/UART_routines.h>
volatile unsigned long startBlock;
volatile unsigned long totalBlocks;
volatile unsigned char note1[122];
volatile unsigned long key1, key2, firstDataSector, rootCluster, totalClusters;
volatile unsigned int bytesPerSector, sectorPerCluster, reservedSectorCount;
unsigned char x, option, error, data, FAT32_active, input, commandhi, commandlow, key1command,
key2command;
unsigned int i, j, y, count, instrument, note1stop, note2stop, note1decay, note2decay, stop,
instrument_change;
unsigned char SD_tone(unsigned long startBlock, unsigned long totalBlocks);
unsigned char USART_Receive();
void USART_Transmit( unsigned char data );
//int y;
//int data;
//char x;
//char count;
//int instrument;
void port_init(void)
{
PORTA = 0x00;
DDRA = 0x01; //initialize PortA.0 as an output, this will be the CS
for the DAC
PORTB = 0xEF;
DDRB = 0xBF; //B6 (MISO from sd card) input, rest outputs
PORTD = 0x00;
DDRD = 0x7A; //D0 (RXD) and D2 (MISO_2) are inputs, rest are outputs
}
void SPI_Init_LowSpeed()
{
SPCR = 0x00;
SPCR |= (1<<SPE)|(1<<MSTR)|(1<<SPR1);//MSB first, Master Mode, Sample on Rising,
Leading Edge, Clock Speed 125kHz
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 13/28 Final Report: Keyboard Cam
SPSR = 0x00; //clear flags, clear 2x sck multiplier
}
void TimingInterrupts_Init(void)
{
TCCR1A &= 0xFC;
TCCR1A += 0; //CTC mode
TCCR1B &= 0xE0;
TCCR1B += 9; //CTC mode and /8 pre scale (8 + ps); ps = 1, CLK = 8Mhz, ps =2,
CLK = 1Mhz
TIMSK1 |= 2; //local OCF1A interrupt enable
TIFR1 |= 2; //make sure output compare A flag is initially cleared
OCR1A = 251; //we are consistently using 32,000 hz throughout so I'll just set this
once
}
void USART_Init()
{
/* Set baud rate */
UBRR0H = 0;
UBRR0L = 51;
/* Enable receiver and transmitter and recieve complete interrupt */
UCSR0B = (1<<RXEN0)|(1<<TXEN0)|(1<<RXCIE0);
/* Set frame format: 8data, 1stop bit */
UCSR0C = (3<<UCSZ00);
}
//The SD Card uses the second USART in SPI Mode
//RX1=MISO1, TX1=MOSI1, XCK1=SCK1, PD5=SD_CS
//USART SPI clock rate: 4Mhz
void SD_USART_spi_init(void)
{
UBRR1 = 0; //make
sure ubrr is 0
DDRD |= (1<<4); //set
XCK1 to output
UCSR1C = 0xC0; //select Master SPI Mode, sample on rising edge, clock low when idle,
MSB bite first, sample on rising edge
UCSR1B |= (1<<RXEN1)|(1<<TXEN1); //enable receiver and
transmitter
UBRR1 = 0;
//4MhZ operation
PORTA ^=1;
PORTA ^=1;
}
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 14/28 Final Report: Keyboard Cam
//call this routine to initialize all devices
void init_all(void)
{
cli();
TimingInterrupts_Init();
SD_USART_spi_init();
port_init();
SPI_Init_LowSpeed();
USART_Init();
WDTCSR |= (1<<5)|(1); //set watchdog prescaler to 8s
MCUCR = 0x00;
}
void main(void){
//unsigned char fileName[13];
cli();
instrument = 2;
PORTA |= (1); //Drive CS high
_delay_ms(1000); //wait for stabilization
init_all();
SD_init(); //this method comes from the Dharmani SDcard library, it initializes the sd
card for spi
SPI_MUSIC_SPEED(); //SCK - 62.5kHz
_delay_ms(1);
//set all variables to zero initially
i=0;
key1=0;
key2=0;
note1[0]=0xFF;
note1stop=0;
count=0;
instrument_change=0;
sei();
while(1){ //infinite loop
if(instrument_change){ //this if statement plays a
transition audio clip when an instrument is changed
cli();
if(instrument==0){SD_tone(8300, 105);} //depending on the instrument, the
appropriate declaration is played
else if(instrument==1){SD_tone(8405, 51);}
else if(instrument==2){SD_tone(8456,50);}
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 15/28 Final Report: Keyboard Cam
else{SD_tone(8600, 94);}
instrument_change=0; //clear the instrument change
variable
sei();
}
if(instrument==0 && !instrument_change) { //for the sine waves, note 1 is looped 200
times
if(i==note1stop){i=0; count++;} //reset the position in the array so
that it wraps around itself
if(note1[i]!=0xFF && count<200){ //if note1==0xFF and i=0 there is no note
DAC_SPI(note1[i]);
i++;
}
}
if(instrument==1 && key1!=0 && count<1 && !instrument_change){ //plays a guitar note once
count++;
stop=0;
//clears the stop variable(SD_tone won't play if set)
startBlock = 12 + (key1-1)*400; //find
the appropriate block for this key (guitar notes take up 400 blocks)
SD_tone(startBlock, 400); //play
the note
}
if(instrument==2 && key1!=0 && count<1 && !instrument_change){ //plays a piano note once
count++;
stop=0;
//clears the stop variable(SD_tone won't play if set)
startBlock = 4812 + (key1-1)*100; //find the
appropriate block for this key (piano notes take up 100 blocks)
SD_tone(startBlock, 100); //play
the note
}
if(instrument==3 && key1!=0 && count<1 && !instrument_change){ //plays a George Takei sound
byte once
count++;
stop=0;
//clears the stop variable(SD_tone won't play if set)
startBlock = 6012 + (key1-1)*200; //find the
appropriate block for this key (George Takei sounds take up 200 blocks)
SD_tone(startBlock, 200); //play
the note
}
}
}
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 16/28 Final Report: Keyboard Cam
ISR(TIMER1_COMPA_vect) //only relevant for DAC_SPI function (only used for
sine waves)
{
cli(); //this interrupt is part of the timing control for the
DAC_SPI function
y = 0; //clears y so DAC_SPI can progress
TIMSK1 |=2;
sei();
PORTA ^=(1<<1); //helps for debugging
}
ISR(USART0_RX_vect) //this is where the bulk of the
keyboard work is done
{
cli();
commandhi=USART_Receive(); //receive commands from PC (2
bytes)
commandlow=USART_Receive();
if(commandhi & 1){ //if the low bit of the first byte is
high, we need an instrument toggle
instrument++; //increment instrument
if(instrument>=4){instrument=0;} //reset to zero if it exceeds the max
instrument_change=1;
}
if(instrument>=4){instrument=0;}
key1command = (commandlow & (0x0F)); //commandlow contains a key2 command in
the high nibble and key1 in the low
key2command = ((commandlow & (0xF0))>>4);
if(key1command>12){key1command=0;} //the keys are numbered 1-12
anything higher is invalid
if(key2command>12){key2command=0;}
//sine waves
if(instrument==0){
if(key1command!=key1 && key1command!=key2){ //if key1 is new
key1=key1command; //update the key status
key2=key2command;
if(key1==0){note1[0]=0xFF;} //if key1 is zero no key is
being pressed
else {update_note(key1-1); count=0;} //load note1 with the key indicated by key1,
reset count
}
else if(key2command!=key2 && key2command!=key1){//if key2 is new
key2=key2command; //update the key status
key1=key1command;
if(key2==0){note1[0]=0xFF;} //if key2 is zero no key is
being pressed
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 17/28 Final Report: Keyboard Cam
else {update_note(key2-1); count=0;} //load note1 with the key indicated by key2,
reset count
}
else{
key2=key2command; //otherwise, update the key
status anyway
key1=key1command;
}
}
//every other instrument
else {
if(key1command!=key1 && key1command!=key2){ //if key1 is new
key1=key1command; //update keys
key2=key2command;
count=0; //reset count
stop=1; //enable
SD_Tone
}
else if(key2command!=key2 && key2command!=key1){ //if key2 is new
key1=key2command; //update keys
key2=key1command;
count=0; //reset count
stop=1;
//enable SD_Tone
}
else{
key2=key2command; //else update keys
anyway
key1=key1command;
}
}
i=0; //reset vector
start
sei(); //enable
interrupts
}
//this function is modified from the read multiple block function in the sd_routines library here:
http://www.dharmanitech.com/2009/01/sd-card-interfacing-with-atmega8-fat32.html
//this function sends what it reads directly to the DAC
//a SPI write speed of 62.5kHz yields an output to the DAC for a sampling frequency that is
//very close to 32kHz (which is the sampling frequency we used for all our audio)
unsigned char SD_tone(unsigned long startBlock, unsigned long totalBlocks)
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 18/28 Final Report: Keyboard Cam
{
unsigned char response;
unsigned int i, retry=0;
retry = 0;
response = SD_sendCommand(READ_MULTIPLE_BLOCKS, startBlock); //write a Block command
if(response != 0x00) return response; //check for SD status: 0x00 - OK (No flags set)
SD_CS_ASSERT; //set chip select
sei(); //enable interrupts so that the audio may be
interrupted
while( totalBlocks )
{
retry = 0;
while(SPI_receive() != 0xfe) //wait for start block token 0xfe (0x11111110)
if(retry++ > 0xfffe){SD_CS_DEASSERT; return 1;} //return if time-out
i=0;
while(((data=SPI_receive())!=0xFF) && i<512 && stop!=1){ //stop is set by the USART receive
ISR if a new note is pressed
SD_DAC_SPI(data);
//immediately send data to the DAC
i++;
}
if((i<512) && data==0xFF && SPI_receive()==0xFF){totalBlocks=1;} //the stop token is
0xFF, if you ever see this, the note is over
if(stop==1){totalBlocks=1;}
//end if stop is set
totalBlocks--;
}
SD_sendCommand(STOP_TRANSMISSION, 0); //command to stop transmission
SD_CS_DEASSERT;
SPI_receive(); //extra 8 clock pulses
return 0;
}
//this function is also modified from Dharmani's read multiple block function
void update_note(unsigned long startBlock) //only used for sine waves; loads a note to
ucontroller memory
{
unsigned char response;
unsigned int i, retry=0;
retry = 0;
totalBlocks=1;
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 19/28 Final Report: Keyboard Cam
response = SD_sendCommand(READ_MULTIPLE_BLOCKS, startBlock); //write a Block command
if(response != 0x00) return response; //check for SD status: 0x00 - OK (No flags set)
SD_CS_ASSERT;
while( totalBlocks )
{
retry = 0;
while(SPI_receive() != 0xfe) //wait for start block token 0xfe (0x11111110)
if(retry++ > 0xfffe){SD_CS_DEASSERT; return 1;} //return if time-out
i=0;
while(((data=SPI_receive())!=0xFF) && i<512){
note1[i]=data; //read data and store to note1
i++;
}
note1stop = i; //remember the end position
SPI_receive(); //receive incoming CRC (16-bit), CRC is ignored here
SPI_receive();
SPI_receive(); //extra 8 cycles
totalBlocks--;
}
SD_sendCommand(STOP_TRANSMISSION, 0); //command to stop transmission
SD_CS_DEASSERT;
SPI_receive(); //extra 8 clock pulses
return 0;
}
//used for all instruments but sine waves, no timing interrupts
void SD_DAC_SPI(char Data)
{
//writes 8-bit data to a DAC using SPI
char temph; //high byte
char templ = Data; //low byte
char tempd = Data;
tempd = tempd>>4;
tempd &= 0x0F; //mask top four bits
temph=(0xF0)|tempd; //high byte contains 4 bit op code and high nibble of data
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 20/28 Final Report: Keyboard Cam
templ=(templ<<4); //low byte contains bottom nibble of data, zeros for the
final two data points(we're using 8 instead of 10 bit resolution) and zeroes for the don't cares
templ &= 0xF0; //mask bottom 4 bits
PORTA &=~(1); //Drive CS low
USART_SPI_transmit(temph); //send high byte
USART_SPI_transmit(templ); //send low byte
_delay_us(2); //wait for transmission to complete
PORTA |= (1); //Drive CS high
}
//only used for sine waves, uses timing interrupts (interrupt every 1/32khz)
void DAC_SPI(char Data)
{
//writes 8-bit data to a DAC using SPI
y=1;
char temph; //high byte
char templ = Data; //low byte
char tempd = Data;
tempd = tempd>>4;
tempd &= 0x0F; //mask top four bits
temph=(0xF0)|tempd; //high byte contains 4 bit op code and high nibble of data
templ=(templ<<4); //low byte contains bottom nibble of data, zeros for the
final two data points(we're using 8 instead of 10 bit resolution) and zeroes for the don't cares
templ &= 0xF0; //mask bottom 4 bits
while(y)
{
PORTA |= (1<<1);
}
PORTA &=~(1); //Drive CS low
USART_SPI_transmit(temph); //send high byte
USART_SPI_transmit(templ); //send low byte
_delay_us(2); //wait for transmission to complete
PORTA |= (1); //Drive CS high
}
//send a byte with USART in SPI Mode
void USART_SPI_transmit(unsigned char data)
{
while ( !(UCSR1A & (1<<UDRE1)) ); // Wait for empty transmit buffer
UDR1 = data; //Start transmission
}
//receive a byte with USART in SPI Mode, not needed for this project
/*unsigned char USART_SPI_receive(void)
{
unsigned char data, status;
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 21/28 Final Report: Keyboard Cam
UDR1 = 0xFF;
while(!(UCSR1A & (1<<RXC1))); // Wait for incoming data
status = UCSR1A;
data = UDR1;
return(data);
}*/
unsigned char USART_Receive()
{
/* Wait for data to be received */
while ( !(UCSR0A & (1<<RXC0)) );
/* Get and return received data from buffer */
return UDR0;
}
void USART_Transmit( unsigned char data )
{
/* Wait for empty transmit buffer */
while ( !( UCSR0A & (1<<UDRE0)) );
/* Put data into buffer, sends the data */
UDR0 = data;
}
//sets SPI to 62.5kHz
void SPI_MUSIC_SPEED(void)
{
SPCR |= (1<<SPR0); //set clock to fck/16
SPCR &= ~(1<<SPR1);
SPSR &= ~1; //make sure SPI2x is off
}
SD Card Library/ Routines adapted from:
http://www.dharmanitech.com/2009/01/sd-card-interfacing-with-atmega8-fat32.html
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 22/28 Final Report: Keyboard Cam
Appendix C: Tracking Code
Tracking.c ---main image tracking program
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <cv.h>
#include <highgui.h>
#include "COM.cpp"
void color(IplImage* img,int thresh,IplImage* colors){
//This function takes an image and a threshold and replaces all non-red pixels with black. Red pixels are
mapped to a value of 255
int i,j,k;
int height,width,step,channels;
int stepr, channelsr;
int temp=0;
uchar *data,*datar;
i=j=k=0;
//cvNamedWindow("original",CV_WINDOW_AUTOSIZE);
//cvNamedWindow("Result",CV_WINDOW_AUTOSIZE);
height = img->height;
width = img->width;
step =img->widthStep;
channels = img->nChannels;
data = (uchar *)img->imageData;
stepr=colors->widthStep;
channelsr=colors->nChannels;
datar = (uchar *)colors->imageData;
for(i=0;i < (height);i++) for(j=0;j <(width);j++){
if(((data[i*step+j*channels+2]) > (thresh+data[i*step+j*channels]))&&
((data[i*step+j*channels+2]) > (thresh+data[i*step+j*channels+1])) &&
data[i*step+j*channels+2]>=150 )
datar[i*stepr+j*channelsr]=255;
else
datar[i*stepr+j*channelsr]=0;
}
cvDilate(colors,colors,0,3);
cvSmooth(colors,colors,2,3,0);
//cvShowImage("original",img);
//cvShowImage("Result",color);
//cvWaitKey();
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 23/28 Final Report: Keyboard Cam
//return color;
}
CvRect FindBlob(IplImage* img){
//This function takes an image and returns a box around the first blob it finds
int x=300;
int y=300;
int found=0;
int i,j;
int height,width,step;
uchar* data;
data = (uchar *)img->imageData;
height=img->height;
width=img->width;
step=img->widthStep;
cvShowImage("Image",img);
for(i=height-1; i>320;i--){
if(found==1) break;
for(j=0; j<width; j++){
if(data[i*step+j]==255){
x=j;
y=i;
found=1;
break; }
}
}
//printf("x: %d",x);
//printf("y: %d",y);
if(found==0)
return cvRect(0,0,40,40);
return cvRect(x-20,y-30,60,60);
}
CvRect FindBlob2(IplImage* img){
//this function takes an image and returns a rectangle containing the first blob it finds
int x=300;
int y=300;
int found=0;
int i,j;
int height,width,step;
uchar* data;
data = (uchar *)img->imageData;
height=img->height;
width=img->width;
step=img->widthStep;
cvShowImage("Image",img);
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 24/28 Final Report: Keyboard Cam
for(i=320; i<height;i++){
if(found==1) break;
for(j=width-1; j>0; j--){
if(data[i*step+j]==255){
x=j;
y=i;
found=1;
break; }
}
}
//printf("x: %d",x);
//printf("y: %d",y);
if(found==0)
return cvRect(0,0,40,40);
return cvRect(x-20,y-20,60,60);
}
void drawKeys(IplImage* img){
int height=img->height;
int width=img->width;
for(int i=0; i<12;i++){
cvRectangle(img,cvPoint(i*width/12,height*2/3),cvPoint((i+1)*width/12,height),cvScalar(255,2
55,255));
}
}
CvPoint COG(IplImage* img){
//this function takes an image and calculates it center of gravity. It also only affects the ROI but this is
relative
//to the origin of the original image so it must be shifted
CvMoments moments;
double M00;
double M10;
double M01;
CvPoint center;
cvMoments(img,&moments,1);
M00 = cvGetSpatialMoment(&moments,0,0);
M10 = cvGetSpatialMoment(&moments,1,0);
M01 = cvGetSpatialMoment(&moments,0,1);
center.x = (int)(M10/M00);
center.y = (int)(M01/M00);
return center;
}
char key(CvPoint led){
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 25/28 Final Report: Keyboard Cam
//this function takes a CvPoint and determines what key has been pressed
int x=led.x;
if(led.x==0 && led.y==0)
return 0;
if(x<=53)
return 12;
else if(x<=106)
return 11;
else if(x<=159)
return 10;
else if(x<=212)
return 9;
else if(x<=265)
return 8;
else if(x<=318)
return 7;
else if(x<=371)
return 6;
else if(x<=424)
return 5;
else if(x<=477)
return 4;
else if(x<=530)
return 3;
else if(x<=583)
return 2;
else if(x<=636)
return 1;
return 0 ;
}
int buttonp(IplImage* img){
//this function monitors the "button" for instrument toggle and determines whether it has been pushed or
not
int i,j;
int count=0;
int height,width,step;
uchar* data;
data = (uchar *)img->imageData;
height=img->height;
width=img->width;
step=img->widthStep;
cvShowImage("Image",img);
for(i=50; i<100;i++){
for(j=500; j<550; j++){
if(data[i*step+j]==255)
count++;
if(count==100)
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 26/28 Final Report: Keyboard Cam
return 1;
}
}
return 0;
}
int main(){
int thresh=60;
int height,width;
int target=0;
char key1=0;
char key2=0;
char key1d=0;
char key2d=0;
char buttontog=0;
int count=0;
//opening the COM port
//HANDLE comhandle=COM_open();
CvCapture* capture = cvCaptureFromCAM(0);
IplImage* img=cvQueryFrame(capture);
IplImage* colors=cvCreateImage(cvGetSize(img),8,1);
cvNamedWindow("Track",CV_WINDOW_AUTOSIZE);
cvNamedWindow("Image",CV_WINDOW_AUTOSIZE);
height=img->height;
width=img->width;
printf("height: %d width: %d ",height,width);
img=cvQueryFrame(capture);
color(img,thresh,colors);
CvRect rect=FindBlob(colors);
CvRect rect2=FindBlob(colors);
//cvRectangle(img,cvPoint(rect.x,rect.y),cvPoint(rect.x+25,rect.y+25),cvScalar(255,255,255));
CvPoint center;//=COG(colors);
CvPoint center2;//=COG(colors);
//cvCircle(colors,center,10,cvScalar(255,255,255));
cvShowImage("Image",img);
cvShowImage("Track",colors);
//cvWaitKey();
if(!img)
printf("Could not load image");
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 27/28 Final Report: Keyboard Cam
while(1){
img=cvQueryFrame(capture);
if(!img) break;
//removing all color but red
cvSmooth(img,img,CV_GAUSSIAN);
color(img,thresh,colors);
rect=FindBlob(colors);
rect2=FindBlob2(colors);
//cvRectangle(img,cvPoint(rect.x,rect.y),cvPoint(rect.x+40,rect.y+40),cvScalar(255,255,0));
//cvRectangle(img,cvPoint(rect2.x,rect2.y),cvPoint(rect2.x+40,rect2.y+40),cvScalar(255,255,255));
/*4 possible values for target
target==0----- no key is being pressed
target==1----- both feet are pressing the same key or only 1 foot playing a note
target==2----- both feet are on different notes*/
if((rect.x==0 && rect.y==0) && (rect2.x==0 && rect2.y==0))
target=0;
else if(rect.x==rect2.x && rect.y==rect2.y)
target=1;
else
target=2;
//drawing the keyboard on the screen
drawKeys(img);
cvRectangle(img,cvPoint(500,50),cvPoint(550,100),cvScalar(255,255,0),CV_FILLED);
int but=buttonp(colors);
if(but==1){
if(count<10){
buttontog=0x0;
count++; }
else {
buttontog=0x1;
count=0; }
}
else
buttontog=0x0;
cvSetImageROI(colors,rect);
center=COG(colors);
cvSetImageROI(colors,rect2);
center2=COG(colors);
cvResetImageROI(colors);
center.x=rect.x+center.x;
center.y=rect.y+center.y;
University of Florida EEL 4924—Spring 2012 25-Apr-122 Electrical & Computer Engineering
Page 28/28 Final Report: Keyboard Cam
center2.x=rect2.x+center2.x;
center2.y=rect2.y+center2.y;
if(target==0){
key1=0;
key2=0;
}
if(target==2){
cvCircle(img,center,10,cvScalar(255,255,255));
cvCircle(img,center2,10,cvScalar(255,255,255));
key1=key(center);
key2=key(center2);
if(key1!=key1d || key2!=key2d || buttontog==1){
//COM_write(comhandle,buttontog);
//COM_write(comhandle,((key2<<4)+key1));
printf("buttontog: %d \n",buttontog);
printf("count:%d \n",count);
}
key1d=key1;
key2d=key2;
}
cvShowImage("Image",img);
cvShowImage("Track",colors);
char c=cvWaitKey(33);
if(c==32) break;
}
cvReleaseImage(&img);
cvReleaseImage(&colors);
}
COM Port Wrapper/ Handler adapted from Examples at:
http://www.ftdichip.com/Support/SoftwareExamples/CodeExamples/VC.htm
top related