#ap 186

Okay, so we took a video of a ball falling and broke it up into frames. It looked something like this.

Look at it go. The ratio of pixels to meters at the full resolution I operated on was 387 pixels for every 0.5 meters, as measured using Paint. The camera operated at 59 frames per second.

Anyway, my plan was simple:

Perform colour segmentation on each of the frames like in Activity 7 to isolate the ball,

Apply Activity 8′s morphological operations to clean up the resulting blobs,

Determine the centroid of the blob like in Activity 4 and get the y-coordinates of these centroids,

Get the instantaneous speeds of the ball using the change in y-coordinates between two frames,

Use these speeds to calculate the average acceleration of the ball,

Convert this into SI units (m/s^2) with the ratio I got earlier, and

Compare this to the theoretical value of 9.8 m/s^2.

Here’s an example of the segmentation and cleaning done on one of the frames.

This particular frame had the centroid of the ball blob located at a y-coordinate of 91.5.

Here’s how the final segmented frames looked all together.

Things went rather smoothly this time, and before long I has a list of y-coordinates ready for operating on. I worked using WPS Spreadsheets and got an average acceleration of 10.72 m/s^2, which differed from 9.8 m/s^2 by 9.4%.

Unacceptable, I said to myself. I then turned to knowledge gained in our Physics 191 class on how to discard outliers in our data. Testing for varying thresholds of deviation, I managed to discard two acceleration values. Computing for the new average acceleration gave me 9.80 m/s^2. Acceptable.

I had to run through this one quickly, since the end of the semester (and te whole year of 2016) meant many more deadlines, both academic and non-academic, were catching up to me. Still, I did as the activity asked, and managed to apply a lot of the knowledge we gained in this course, and even from Physics 191. I’ll give myself an 11 out of 10 for this.

And my final acknowledgements for this course:

Joshua Abuel, Anthony Fox, Kit Guial, and Roland Romero for allowing me to join them in recording the video for this activity

Dr. Maricor Soriano, for guiding us through the course; it was a blast

Grayscale Enhancement

I know we’ve all experienced excessive shadows in pictures. They can come from the shot being in a poorly-lit area, from a side effect of using the flash without a reflector or diffuse, or from backlight. Here’s a photo illustrating the first example.

We may never know what lies in the void I was sticking my hand into; or so I thought at first, but it turns out those shadows still contain information. We just have to force it out, and that’s where histogram manipulation comes in.

First, we test it on the image in grayscale. Since we only have one channel here, the normalized histogram of the image corresponds to its probability distribution function or PDF.

From this, we use the cumulative sum function to allow us to obtain the image’s cumulative distribution function or CDF.

This is what we will use to modify the image’s histogram. We’ll want to pick a CDF to backproject onto the PDF. For this activity we’ll try both a linear CDF and a nonlinear sigmoid CDF, generated using an exponential Gaussian function.

And with no further ado, here are the results from the modified PDFs:

I’d say that while the linear CDF produces a brighter image, it seems a bit overwhelming in its brightness. The sigmoid CDF gives an image that doesn’t go as bright, but it reveals enough detail.

Colour Enhancement

Alright, let’s get to it. The modification we made to the image’s histogram on the grayscale level was basically modifying the “intensity” of each pixel, with high intensity being closer to white.

For coloured images, we use our knowledge of normalized colour coordinates (NCC) to obtain the intensity I, as well as the previously used r, g, and b values for each pixel in the image.

From here, it’s just a matter of applying histogram modification to the intensity matrix of the image, then using it to compute for the new RGB values of the matrix from NCCs r, g, and b. I used both the linear and sigmoid CDFs for this, and the results look pretty good.

I had to try it on something else, and so I found this picture of my niece’s furby I had for.... some reason.

Working on this was pretty straightforward. I had a problem when I tried to apply my grayscale code to the coloured image, but somehow I managed to fix it by starting the code again from scratch. I assume I just missed something while modifying it. I delivered what the activity required, and my images are all there so I say I get a 10 out of 10 for this one.

Here are my acknowledgements and references:

Roland Romero, for helping me troubleshoot my code

My niece, Keira Paulino, for having that furby

The sheet music I used for this activity is a piano adaptation of the song ECHO by Crusher-P. The piano piece can be found here, while the original song can be found here. For this activity, I only included the melody following the treble clef up until the 64th measure of the piece.

First of all, we should know that scilab can produce sounds from sinusoid functions.

The frequency f of the wave here determines the pitch of the sound, and the time t defines how long the sound plays.

Now the product of this is called pure tone, a continuous sound that doesn’t quite sound like a note played by a piano. To fix that, we applied an sound envelope or an ADSR (attack, decay, sustain, release) envelope to emulate the sound of an instrument.

Here’s the note function I used, based on the one provided by our professor, Dr. Soriano:

//note function function n = note(f, t)

   n = sin (2*%pi*f*t);    //defines the envelope    env1 = linspace(0, 1, round(0.025*length(t))); //attack    env2 = linspace(1, 1, round(0.05*length(t))); //sustain    env3 = linspace(1, 0.9, round(0.05*length(t))); //drop before decay    env4 = linspace(0.9, 0.45, round(0.85*length(t))); //decay    env5 = linspace(0.45, 0, length(t)-length([env1,env2,env3,env4])); //release    env = [env1, env2, env3, env4, env5];    //applies the envelope    n = n.* env; endfunction;

So with that done I had two main objectives: identifying the kinds of notes (half, quarter, eighth, sixteenth), and their pitch, in addition to recognizing rests and dotted notes.

I stitched together the many lines of music to form one long continuous line from the first to 64th measure. This was my input image. 

As I said before, I used Dr. Galda’s IPD kit to analyze the notes here. After reading the image, I realized I had a problem. If I applied morphological operations to isolate the quarter notes, I’d be removing the half notes. And even so, how would I differentiate between quarter, eight, and sixteenth notes?!

I ended up using three segmentation methods for the notes:

one to detect the “filled” note heads that quarter, eight, and sixteenth notes shared,

another to detect the “hollow” half note heads, and

the last to detect the bars and flags of eighth and sixteenth notes

Here’s an example of how the first segmentation process worked. 

The other two used different thresholds and morphological operations.

With the notes indexed, I’d get the x and y-coordinates of the centroids of the note heads and used the x-coordinates to control the sequence at which the notes would be played, and the y-coordinates to determine the pitch of the note. Additionally, if the x-coordinate of a note head coincided with those included in a bar or flag, the note was identified as an eighth or sixteenth note.

For the rests and the dotted notes, I had to use two more segmentation processes. Fortunately, all rests are located in the same spots on the musical staff, so if a blob was found to occupy that spot, I could mark it as a corresponding quarter or eighth rest. For the dots, I had a bit of trouble since most of my notes were staccato notes. To remove the staccato marks, I only considered dots that didn’t line up vertically with other notes. After finding the notes closest to these dots, I applied the corresponding modification to their time (dotted notes are played at a total time equal to 1.5 times the time of the usual note). I... ignored the staccato notes for now, as modifying the envelope either produced no noticeable effect, or too much of an effect on the final notes in the wave file.

With all of that done, I have my final output. You can listen to it here. HEADPHONES WARNING!! It can get kinda loud.

That was a really tedious activity, but I can’t deny it was a lot of fun too. I met the requirements of the activity, plus adding in the sound envelope, together with recognizing rests and dotted notes. For that, I think I deserve a 12 out of 10.

Now I get to sing praises to the following as my acknowledgements:

Dr. Maricor Soriano, our professor who provided the note function

Joshua Abuel, for helping me with the process of detecting notes

Physics of Music Notes, for the note frequencies

Luman Magnum, for the info on how to  apply a sound envelope

Crusher-P, for creating the beautiful and addictive song ECHO

Okay, to begin, we had to familiarize myself with normalized chromaticity coordinates (NCC), which were used to represent the colour values in our images. Basically, we can take the colour of a single pixel and represent it in values of Red, Green, and Blue. If we choose our coordinate system such that the sum of all three values representing red, green, and blue must be equal to 1, we can instead represent this colour using only two coordinates; chosen here to be r and g.

With that laid out, let me introduce you all to my test image: Happy Rabbit.

It’s so cute.

So we had two methods of segmentation to cover: parametric, and non-parametric. 

Parametric Segmentation

This method relies on a Probability Distribution Function (PDF) generated from a selected subregion within the region of interest (ROI) we want highlighted. Here’s what I got from a particularly orange part of Happy Rabbit.

So this should give an idea of whether or not a pixel can be considered as part of your ROI depending on its r and g values.

With this, we found the probability for each pixel in the Happy Rabbit image to be part of the ROI. The result is... frightening.

The demon is rising. So to help Demon Rabbit enter our world, we set a probability threshold for which points should be considered in our ROI. Naturally, decreasing the threshold allows more points to be considered, but these points might not be part of Happy Rabbit, but instead part of the soft fabric it rests its happy body on. It’s up to us to find a good threshold.

Non-Parametric Segmentation

This method relies on histogram backprojection. Basically, we take the NCC histogram for our subregion, and use this to pick out pixels that should be included in our ROI. It’s kind of like having a a phone book to look up numbers you wanna call.

And with this histogram, we got Demon Rabbit v2.

No thresholds here. The segmentation is entirely dependent on the subregion chosen, and because of that, it can include a wider variety of colours in the segmentation, shown here with most of the shadow and face now being included, in contrast to our previous Demon Rabbit.

Now wasn’t that fun, summoning forth the Demon Rabbit from the abyss? I think it was. And hey, I also accomplished the requirements for this activity. Not bad, if I do say so myself. I’ll give myself a 10 out of 10 here for that. I didn’t really explore far beyond what was asked, so I don’t think I deserve any bonus points in this one. All in all, I had a good time.

Long live Happy Rabbit.

Here are my Acknowledgements and References:

Wondercat, the creator of Happy Rabbit

Anamorphism in Fourier Space

Fourier space demonstrates anamorphism, which is a property that makes it so something wide in one axis in regular space becomes thin in Fourier space, and vice versa. To observe this easily, we used a pair of rectangular apertures.

They look squished, as expected. But what happens if I try it with two dots? In the last activity, we showed the FT of two dots on the x-axis is a corrugated roof, based on a sinusoid.

Looks like spreading the dots further apart increases the frequency of the sinusoid. This makes sense, since these dots would represent the frequencies of the sinusoid, and spreading them further apart would have them correspond to a greater displacement from the origin, hence a greater frequency.

Rotation Property

“You spin me right round, baby, right round. Like a record, baby, right round, round, round.”

-Dead or Alive, 1984

Using these convenient sinusoids, we can demonstrate the rotation property of the Fourier transform.

Look at them go. The transforms essentially follow the rotation of the original signals. While we’re rotating sinusoids, we can show how the FT handles superposition.

Now that’s interesting. Here it is again with one of the sinusoids having a different frequency.

When the signals are added, their transforms seem to be mirrored somehow. Let’s take this a step further and add some rotated sinusoids and see what happens.

Crazy, but still following some form of symmetry.

Convolution 2: Electric Boogaloo

So convolution was fun in the last activity. Time to do more.

First, we get that our slightly overused two-dot pattern produces a sinusoid, but what happens if we replace the dots with circles? How about Gaussians?

As you can see, the result shows the corrugated roof, but it’s constrained to the pattern you’d expect to see when you take the FT of a single circle or Gaussian, as shown in the previous activity.

This actually shows a part of what happens in convolution. The two-dot pattern can be considered as two dirac delta functions, having a value of 1 at two points, and zero everywhere else. When you convolve two functions, you multiply them in Fourier space. Look at the FT’s. They’re the result of a multiplication between a sinusoid and the FT of the circle/Gaussian! This means that the original images we used are actually the result of a convolution of the two-spot pattern and a corresponding single circle or Gaussian pattern.

Now let’s prove this using the conv2 function in scilab. I’ll be convolving two dirac delta patterns with arbitrary patterns.

Aha, so I understood it right. Nice.

Mask Charge!!

With the words “transform” and “mask” being brought up, there’s no way I couldn’t reference this.

Last activity, we used convolution to do edge detection. This time, we’ll use it to do filtering. Filtering is an important part of image processing, as it can be used to remove or enhance patterns. Here, we’re gonna clean up a picture of a fingerprint, just like how the Power Rangers cleaned Angel Grove of the evil of Rita Repulsa’s monsters.

Like the Power Rangers, we need masks to do the job. Our masks, however, don’t hide our faces while we perform martial arts moves, as much as I want them to. Instead, they hide unwanted frequencies in Fourier space.

In this case, I use a filtering mask in Fourier space to eliminate those nasty frequencies I don’t want showing up in my image so that I can get a clearer look at this fingerprint, generously contributed by my classmate, Roland.

After the filtering, it was much easier to binarize the image, so we can see all those lines and whorls in their curvaceous glory.

Mask Change I

Since we literally started small with fingerprints, let’s think big. Like really big. Like moon big.

Look at this picture of the moon. How are the Power Rangers supposed to find Rita Repulsa’s moon palace with all these horizontal lines messing up the view? Well, we can help them out with our filtering powers!

I knew which frequencies to filter out by applying that anamorphic property we learned about earlier. And with this, we can get back to our fight against evil finishing this activity.

Mask Change II

Here’s something that I’d find challenging to sneak a Power Rangers reference into.

We can use filtering masks to remove the canvas from this image of a painting. This can be used to study the brush strokes or the pigmentation used by the artist. Not only that, but by studying the canvas pattern itself, we can determine what kind of canvas the painting was made on, which can help when dealing with art fraud and the like. Then we can toss them into a space dumpster where they will be trapped for 10,000 years. Ha, I did it.

While we’re disposing of evil people that seek to profit from lies and deceit, we should take another look at that filtering mask I made using Photoshop. I’ll admit it, I couldn’t automate this one.

The resulting pattern looks kinda like the canvas, which shouldn’t be surprising, since that’s the very pattern we were filtering out.

I know this time I didn’t do a lot of exploring out of the scope of the activity, but I was pressed for time. Maybe if I were a Power Ranger...

Anyway, I hope I did a good job explaining what the activity was about. I didn’t put any codes in this time, as they were similar if not identical to those used in the last one. I’ll give myself a 9 out of 10 for this one. I satisfied the requirements, but I think I could’ve done more. Again, maybe if I were a Power Ranger...

I’d like to acknowledge these people as my Power Rangers, arriving just in time to save me from not finishing my codes:

Joshua Abuel, for helping me eliminate the distracting noise and enhance the power of the fingerprint pattern

Louie Rubio, for guiding me in figuring out which frequencies to block in order to scan the moon for Rita Repulsa’s moon palace

Roland Romero, for lending me a hand, or finger, in the fight against evil

Dr. Maricor Soriano, for allowing me to post in this format in which I can reference the Power Rangers

Circular Aperture

Alright, so our professor gave us an example that plotted a circular aperture. I used this as the basis for pretty much every other code I wrote here. The code and the plot produced is as follows:

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny);

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

r= sqrt(X.^2 + Y.^2); //note element-per-element squaring of X and Y A = zeros (nx,ny); A (find(r<0.7) ) = 1; f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Square Aperture

“Alright, cool. It’s easy to understand,” I told myself as I started on the square aperture. I immediately thought to use an absolute value function to define the shape. A good idea, but in my zeal, I didn’t notice I was using two absolute value functions instead of using an and operator, and long story short, I made a big cross shape. At least then I had an idea for the cross shape later on. Anyway, here’s my code and the plot it produced:

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny);

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

rx= abs(X); //absolute value function ry= abs(Y); A = zeros (nx,ny); A (find(rx<0.6 & ry<0.6) ) = 1; //finds the perimeter of the square f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Corrugated Roof

Progress. Next was the corrugated roof. It needed a sine function, but I wanted to keep the plot to be from -1 to 1. Now if you know the sine function, you know it has a period of 2π. If I plotted the sine function from -1 to 1, it’d be pretty boring, wouldn’t it? The answer is yes. Okay, so the solution I came up with was to map the sine function over something like -6π to +6π onto the plot we had now. So now the axes match, and my somewhat unnecessary problem is solved. Just a warning, it’s a little tiring on the eyes. Here’s what I did:

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny); x1 = linspace(-7*%pi,7*%pi,nx); //range for the sinusoid

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

rx= cos(x1); //sinusoid function A = zeros (nx,ny); for j = 1:nx //inserts the sinusoid function into the array    A (j,:) = rx(j); end f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Grating

Okay, for the grating, I wanted to take my sine result and apply an if condition that replaces all positive values with 1, and all negatives with 0. It worked and I was happy with my achievement. Especially after thinking very hard about the for loops.

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny); x1 = linspace(-7*%pi,7*%pi,nx); //range for the sinusoid

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

rx= cos(x1); //sinusoid function for i = 1:nx; //converts the sinusoid into a grate function    if rx(i)<0        rx(i)=0    else        if rx(i)>0            rx(i)=1        end    end end A = zeros (nx,ny); for j = 1:nx //inserts the sinusoid function into the array    A (j,:) = rx(j); end f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Annulus

Up next was the annulus, a scientific way of saying a ring shape. Not much explaining here, I just added another line of code defining a smaller circle. Ta-dah.

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny);

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

r= sqrt(X.^2 + Y.^2); //equation of a circle A = zeros (nx,ny); A (find(r<0.7) ) = 1; //forms the outer circumference A (find(r<0.4) ) = 0; //forms the inner circumference f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Circular Aperture with Graded Transparency

This next one, I skipped at first. But I eventually came back and did it. The graded transparency is added to the circular aperture using a Gaussian function. This function was defined first, then applied to that A array I’ve been using. I then cut the rest of it off with the circular aperture from the sample code. The result looks pretty cool.

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny);

function y = gauss(x,s) //defines the gaussian function to be used    y = exp(-x^2/(2*s^2)) endfunction

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

r= sqrt(X.^2 + Y.^2); //note element-per-element squaring of X and Y A = zeros (nx,ny); A = gauss(r,8); //applies the gaussian to the A array A (find(r>0.7) ) = 0; //adds the circular limit f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Ellipse

I’m not gonna lie, the ellipse was the hardest for me. Not because of anything with the equation, mind you. I know the equation of an ellipse. I could have sworn it was just refusing to work, no matter what I tweaked. I even asked my classmate and friend, Josh, about it and we were both stumped for a while... Then we saw it. I had put the periods inside the parentheses beside X and Y like in the example code. I forgot to move them out. Correcting that, it produced an ellipse and I could finally breathe a sigh of relief.

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny); x1 = linspace(-5,5,nx); //defines the range y1 = linspace(-5,5,ny);

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

r= sqrt((X/0.7).^2 + (Y/0.5).^2); //equation of an ellipse A = zeros (nx,ny); A (find(r<1) ) = 1; f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Cross

And last but not least is the cross. No problems here, I applied what I’d learned earlier from making the square to get the horizontal and vertical parts of the cross. Then I defined a perimeter using the or operator that clipped the cross off where I wanted it.

nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny);

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

rx= abs(X); //absolute value function ry= abs(Y); A = zeros (nx,ny); A (find(rx<0.3 ) ) = 1; //creates the horizontal part of the cross A (find(ry<0.3 ) ) = 1; //creates the vertical part A (find(rx>0.6 | ry>0.6) ) = 0; //limits the cross f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Exploration

Alright, then came the fun part. We got to combine a bunch of images to form something cool. I already knew what I wanted to form, so I took it step by step, masking the images on like they were layers in an image editing software. It was nice getting to combine the and and or operators to define boundaries, particularly in the cross sections within the iris/pupil. Haven’t played around with logic like that since the philosophy GE courses. That’s really all I gotta say on the method. So behold, an eye, definitely not representing how awake I am, writing this at 2am. Rather, it’s what came to mind when I thought of the word “exploration.” To explore is to see, or witness, in a sense.

stacksize(13000000); //changes the stacksize to allow for more memory nx = 1000; ny = 1000; //defines the number of elements along x and y x = linspace(-1,1,nx); //defines the range y = linspace(-1,1,ny);

[X,Y] = ndgrid(x,y); //creates two 2-D arrays of x and y coordinates

riris= sqrt(X.^2 + Y.^2); //origin-centered circle rtop = sqrt(X.^2 + (Y+0.4).^2); //circle centered at y=+0.4 rbot = sqrt(X.^2 + (Y-0.4).^2); //circle centered at y=-0.4 rx= abs(X); //absolute value function ry= abs(Y);

A = ones (nx,ny); A (find(riris<0.45) ) = 0.5; //draws the "iris" of the eye A (find(riris<0.35) ) = 0.3; A (find(riris<0.3) ) = 0.1; A (find(rx<0.075 & riris<0.3) ) = 0.15; //creates the x part of the cross A (find(ry<0.075 & riris<0.3) ) = 0.15; //creates the y part of the cross A (find(rx<0.05 & riris<0.3) ) = 0.5; //creates the x part of the inner cross A (find(ry<0.05 & riris<0.3) ) = 0.5; //creates the y part of the inner cross A (find((rx>0.2 | ry>0.2) & ((rx<0.075 & riris<0.3) | (ry<0.075 & riris<0.3))) ) = 0.15; //limits the inner cross A (find(rtop>1.) ) = 0; //cuts off bottom part of the eye A (find(rbot>1.) ) = 0; //cuts off top part of the eye f = scf();

grayplot(x,y,A);

f.color_map = graycolormap(32);

Overall, I’d say I really did have fun making all these. I learned quite a bit, and I liked looking at these plots, especially when I figured out how to set the axes to have the same scale. Look at them, perfect squares... Anyway, this was a good activity that I could do at a nice pace, especially with the SPP conference happening around the time. I’m thankful for being able to do it.

Now comes the part where I evaluate myself. Hmm, all the required outputs are there. And I took great pains to ensure they were of uniform size, but oh man, my captions could probably be better. Also, I was able to use past lessons on both programming and logic in this so hey. I’d give myself a solid 10 out of 10, with initiative making up for the overall lack of captioning.

As far as my acknowledgements and references go,

Scilab Online Help, for references on the syntax and such