Just a quick update people, I've been waiting for this to clear for the last week or so.
my blog has been moved to a new location, my own personal little bit of the web at :
Please update links, and tonight I'll be updating my current progress! thankyou.
Thursday, September 20, 2007
Monday, September 3, 2007
GUI's and OpenGL
First things first, thought I'd just mention the latest update to my blog, on the right hand side you can now see a list of what needs to be done in order to complete my prototype. Highlighted in green are the elements that are complete, highlighted in orange are the elements I'm currently working on. So you can keep track of my progress. And this leads in to my current problem.
In processing there are two options for GUI's ( Graphical User Interfaces ).
1) Use a GUI Library
2) Code it all yourself from scratch
Option 2 would take a *long* time, so this leaves option 1. However, even option 1 has problems. Processing has 4 GUI Libraries ( that I am aware of ).
- controlP5
- MyGUI
- Interfascia
- SpringGUI
They all work fine by themselves. However add P3D or OpenGL, and they tend to break. MyGUI and Interfascia seem to die all together. SpringGUI works with P3D but not OpenGL, and controlP5 works with openGL, however the 3D elements always draw on TOP of the GUI. I tried running the GUI elements in a separate window, however that caused issues, and it seems to only work part of the time.
I have heard rumors of a way of re-coding MyGUI to work in conjunction with OpenGL, but I am not 100% sure of its authenticity ( or on the re-compiling process, I'd have to brush up on that ).
So I've gone to the processing forums for support, I'm hoping someone there can offer a suitable solution - Until then, I'm sort of forced to put everything on hold - so hopefully I'll get a solution before then.
If I can't get a solution, I will have to work out a possible solution. Even if this means ensuring no 3D elements move closer than 0 on the Z axis, and drawing all the GUI elements on that point. This will mess up perspective, but it may be the only compromise open to me.
- Anthony
In processing there are two options for GUI's ( Graphical User Interfaces ).
1) Use a GUI Library
2) Code it all yourself from scratch
Option 2 would take a *long* time, so this leaves option 1. However, even option 1 has problems. Processing has 4 GUI Libraries ( that I am aware of ).
- controlP5
- MyGUI
- Interfascia
- SpringGUI
They all work fine by themselves. However add P3D or OpenGL, and they tend to break. MyGUI and Interfascia seem to die all together. SpringGUI works with P3D but not OpenGL, and controlP5 works with openGL, however the 3D elements always draw on TOP of the GUI. I tried running the GUI elements in a separate window, however that caused issues, and it seems to only work part of the time.
I have heard rumors of a way of re-coding MyGUI to work in conjunction with OpenGL, but I am not 100% sure of its authenticity ( or on the re-compiling process, I'd have to brush up on that ).
So I've gone to the processing forums for support, I'm hoping someone there can offer a suitable solution - Until then, I'm sort of forced to put everything on hold - so hopefully I'll get a solution before then.
If I can't get a solution, I will have to work out a possible solution. Even if this means ensuring no 3D elements move closer than 0 on the Z axis, and drawing all the GUI elements on that point. This will mess up perspective, but it may be the only compromise open to me.
- Anthony
Saturday, September 1, 2007
Audio Analysis and Beat Detection version 2
I was bored yesterday afternoon, so I sat down and re-coded from scratch my beat detector. I received an email from Robert Hodgin ( aka Flight404 ) on some tips to getting beat detection working effectively.
His short answer was it's very tricky. He's still trying to figure out the best way to do it himself, and throughout the googleverse it's about the same story wherever you go. The best way to do it I've decided is a version very similar to his own :
Basically you divide the FFT stream up into X number of segments ( I've chosen 6 ), the sizes get larger as they move to the right of the stream ( bass to treble ). For each of these segments ( or "Zones" ), you then calculate an average value, and a "threshold" value. If the average moves higher than the threshold, then you have a dramatic increase in that zone, and possibly ( I say possibly ) a beat.
Talking is boring, how about I show you.
Windows Users : Click here ( .rar 1.4mb )
Mac Users : Click here ( .rar 1.3mb )
Unrar the files. The instructions are in the mac .rar file ( there's a readme.txt ). The program listens to your microphone. Windows users, you can go to your recording preferences and set your recording input to equal your audio output and get direct sound from winamp or wmp or whatever you use. Mac users ? no idea ( if someone figures out how to do it, tell me please !).
Let me explain what you're seeing :
The Red lines are the zone "averages". The yellow lines are the "threshold" values. When 2 or more areas trigger at the same time, I'm saying that's a "beat", the screen will flash white ( very quickly ). You'll notice it's still very jumpy, and requires a LOT of tweaking ,but I think I'm getting there. It's kinda cool though.
Watching it, I've noticed certain trends. In the next version, I'm going to experiement with the timing of the thresholds. How much higher than the current value it has to be to trigger a threshold change, and maybe even "weighting" certain zones to be worth more than others ( as they seem to contain "beat" related frequencies more often than others ... ) Suggestions ?
That's it for now. Enjoy.
- Anthony
His short answer was it's very tricky. He's still trying to figure out the best way to do it himself, and throughout the googleverse it's about the same story wherever you go. The best way to do it I've decided is a version very similar to his own :
Basically you divide the FFT stream up into X number of segments ( I've chosen 6 ), the sizes get larger as they move to the right of the stream ( bass to treble ). For each of these segments ( or "Zones" ), you then calculate an average value, and a "threshold" value. If the average moves higher than the threshold, then you have a dramatic increase in that zone, and possibly ( I say possibly ) a beat.
Talking is boring, how about I show you.
Windows Users : Click here ( .rar 1.4mb )
Mac Users : Click here ( .rar 1.3mb )
Unrar the files. The instructions are in the mac .rar file ( there's a readme.txt ). The program listens to your microphone. Windows users, you can go to your recording preferences and set your recording input to equal your audio output and get direct sound from winamp or wmp or whatever you use. Mac users ? no idea ( if someone figures out how to do it, tell me please !).
Let me explain what you're seeing :
The Red lines are the zone "averages". The yellow lines are the "threshold" values. When 2 or more areas trigger at the same time, I'm saying that's a "beat", the screen will flash white ( very quickly ). You'll notice it's still very jumpy, and requires a LOT of tweaking ,but I think I'm getting there. It's kinda cool though.
Watching it, I've noticed certain trends. In the next version, I'm going to experiement with the timing of the thresholds. How much higher than the current value it has to be to trigger a threshold change, and maybe even "weighting" certain zones to be worth more than others ( as they seem to contain "beat" related frequencies more often than others ... ) Suggestions ?
That's it for now. Enjoy.
- Anthony
Thursday, August 30, 2007
Presentation Feedback, and Change of Focus
Ok, so we had our project presentations on monday. Boy did mine not go well. I was actually feeling a little under the weather, so I wasn't really thinking straight, not a good start. Eventually ( after 2 hours of waiting ) I got to present. Ralf I don't think was amazingly impressed. Here is the feedback I got from my tutors, Petra and Lorna :
Taking all this, and Ralf's comments into consideration - I am actually going to be more ambitious, and I'm going to move back to my original idea. Which is a Visualisation *CONSTRUCTING* tool. Based in processing still, and still using 3d. But so the user has the ability to construt their own 3D visuals, and play with them as they see fit. Much more fun, right ?
So, I've sat down and gone over the preliminary details, and boy, will it be a BIG assignment. I've got a lot to do. I've roughly broken it up into four parts :
1. Audio - Analysing of Audio Stream, Returning Useable Data.
2. Creation - Creation of Shapes/Particles/Effects to be displayed
3. Display - HOW the objects are displayed. 3d, blending, blurring...etc
4. Presentation - Live mixing of saved created presets.
I've got to delve into a little more detail, but I think this will be the direction I will take. How complex it will all be when it's done ? I'm not sure. But it should be an interesting journey none the less! I'm tired and a little sick, so that's all I'm writing for now - but I'll write again tomorrow after I get the feedback from my draft report.
- Anthony
"- Sounds like an interesting product
- Ralf suggested getting it up to prototype
- Looking forward to seeing what you produce, played with processing a little, and is lots of fun.
- Well presented, and understandable
- Scope needs to be considered to make sure that it is achievable"
"Interesting project - though you need to sell it more! Aiming for the casual desktop user is kinda selling it short, considering the potential it has for inclusion into club setting. How are you going to differentiate it from the others out there - why would ppl use yours over theirs? Allowing for customisation would help to push up a level - and aiming for a particular group of users (like VJ's) would also help."
Taking all this, and Ralf's comments into consideration - I am actually going to be more ambitious, and I'm going to move back to my original idea. Which is a Visualisation *CONSTRUCTING* tool. Based in processing still, and still using 3d. But so the user has the ability to construt their own 3D visuals, and play with them as they see fit. Much more fun, right ?
So, I've sat down and gone over the preliminary details, and boy, will it be a BIG assignment. I've got a lot to do. I've roughly broken it up into four parts :
1. Audio - Analysing of Audio Stream, Returning Useable Data.
2. Creation - Creation of Shapes/Particles/Effects to be displayed
3. Display - HOW the objects are displayed. 3d, blending, blurring...etc
4. Presentation - Live mixing of saved created presets.
I've got to delve into a little more detail, but I think this will be the direction I will take. How complex it will all be when it's done ? I'm not sure. But it should be an interesting journey none the less! I'm tired and a little sick, so that's all I'm writing for now - but I'll write again tomorrow after I get the feedback from my draft report.
- Anthony
Friday, August 17, 2007
FFT Research, Project Choice and Processing Issues
Project Choice
I've decided to go with the audio-responsive visuals. I figure it's the most challenging option available for me and so it's therefore the best way to show off my skills for the end of year expo and beyond.
In Saying this, I wanted to get a decent amount of research done into FFT and Frequency analysis.
Processing Problems
Before I start anything, I've got problems. I wanted to start uploading my processing "sketches". However I've encountered a problem. I spent a large portion of time yesterday afternoon attempting to get the sketches to work as a java applet from within a browser - no joy there, and now I've found that something has broken, and I can't even export as a standalone application - BIG PROBLEM.
The error I get is : "File sonia_v2_9.jar,JSynClasses.jar,JSynV142.dll,libJSynV142.jnilib does not exist"
If anyone knows anything about processing and can help me, I'd appreciate it. I've also posted for support on the processing website here. Until I get this fixed, I can't show you where I'm up to in my experiments - as a last resort I'm sure I can export them from my laptop, so I will have to try that later.
FFT Research
Now to the intersting part ...
I've been doing some research into FFT and Spectrum Analysis. Just so I know what I'm dealing with here. I suppose I should start from the beginning.
In order to work with sound, I need to understand the format in which I can access it. Using processing, and one of the various sound libraries ( In my case, I am using 'Sonia' I can access the live stream of music via my soundcard. This data is intepreted via FFT ( More on this later ), and then outputted into a spectrum array. This array can then be accessed and the music or sound can be intepreted.
So firstly, What is FFT ?
FFT ( or Fast Fourier Transform ) is an "Algorithm to compute the "Discrete Fourier Transform" and it's inverse." Ok, so what is a Fourier Transform ?
The Fourier Transform Produces a Spectrum. It saves the Magnitidue of each frequency component. This data can then be represented as a graph or plot.
So how does this tie in with processing ?
The FFT Data is collated and stored in the array. Each array element is an "FFT Bin". These "bins" are basically storage elements for a *collection* of frequency ranges - not a single frequency. So how are you meant to know what frequencies are contained in which bins ?
Say you have a file with a Sample Rate of 44100Hz. A frequency range of 22050Hz is equally split into 512 bins. Each bin is : 22050 / 512 : 43.066Hz Wide.
The Left most bin ( Lowest Frequency ) has *half* the width of all other bins. ( contains frequencies 0.000Hz to 21.5Hz ).
So what does all this mean ?
This means that I can get data from a sound source, live streaming sound data - examine the frequencies, and calculate which elements represent which aspects of music ( kicks, snares, synths etc ).
And where will this lead ?
Eventually with a sufficiant understanding of the backbone behind the software, and an understanding of the data input and how it is aquired, I can isolate parts of my motion system to react to different parts / types of music in different ways. A greater understanding will help me to create a project of greater impressivenessnessness.
Any Questions ? Any Comments ? Any Suggestions ?
Where now ?
So what I have to do now, is try and get processing working. From there, the experiments continue.
- Anthony
I've decided to go with the audio-responsive visuals. I figure it's the most challenging option available for me and so it's therefore the best way to show off my skills for the end of year expo and beyond.
In Saying this, I wanted to get a decent amount of research done into FFT and Frequency analysis.
Processing Problems
Before I start anything, I've got problems. I wanted to start uploading my processing "sketches". However I've encountered a problem. I spent a large portion of time yesterday afternoon attempting to get the sketches to work as a java applet from within a browser - no joy there, and now I've found that something has broken, and I can't even export as a standalone application - BIG PROBLEM.
The error I get is : "File sonia_v2_9.jar,JSynClasses.jar,JSynV142.dll,libJSynV142.jnilib does not exist"
If anyone knows anything about processing and can help me, I'd appreciate it. I've also posted for support on the processing website here. Until I get this fixed, I can't show you where I'm up to in my experiments - as a last resort I'm sure I can export them from my laptop, so I will have to try that later.
FFT Research
Now to the intersting part ...
I've been doing some research into FFT and Spectrum Analysis. Just so I know what I'm dealing with here. I suppose I should start from the beginning.
In order to work with sound, I need to understand the format in which I can access it. Using processing, and one of the various sound libraries ( In my case, I am using 'Sonia' I can access the live stream of music via my soundcard. This data is intepreted via FFT ( More on this later ), and then outputted into a spectrum array. This array can then be accessed and the music or sound can be intepreted.
So firstly, What is FFT ?
FFT ( or Fast Fourier Transform ) is an "Algorithm to compute the "Discrete Fourier Transform" and it's inverse." Ok, so what is a Fourier Transform ?
The Fourier Transform Produces a Spectrum. It saves the Magnitidue of each frequency component. This data can then be represented as a graph or plot.
So how does this tie in with processing ?
The FFT Data is collated and stored in the array. Each array element is an "FFT Bin". These "bins" are basically storage elements for a *collection* of frequency ranges - not a single frequency. So how are you meant to know what frequencies are contained in which bins ?
Say you have a file with a Sample Rate of 44100Hz. A frequency range of 22050Hz is equally split into 512 bins. Each bin is : 22050 / 512 : 43.066Hz Wide.
The Left most bin ( Lowest Frequency ) has *half* the width of all other bins. ( contains frequencies 0.000Hz to 21.5Hz ).
So what does all this mean ?
This means that I can get data from a sound source, live streaming sound data - examine the frequencies, and calculate which elements represent which aspects of music ( kicks, snares, synths etc ).
And where will this lead ?
Eventually with a sufficiant understanding of the backbone behind the software, and an understanding of the data input and how it is aquired, I can isolate parts of my motion system to react to different parts / types of music in different ways. A greater understanding will help me to create a project of greater impressivenessnessness.
Any Questions ? Any Comments ? Any Suggestions ?
Where now ?
So what I have to do now, is try and get processing working. From there, the experiments continue.
- Anthony
Project Two Blog Links
I thought I'd keep a record of all the other studio students blogs - that way it's easy to keep in contact and see where other students are at. I'll update this post as I find out more blog URLs. I will also post brief project details here as well, so people can browse via interest.
- Anthony Massingham ( http://amassingham.blogspot.com/ )
Sound-Reactive Visuals
- Tim Cooper / Brendan Lidster ( http://timandbrendansstudio6.blogspot.com/ )
Business Ranking Social Network
- Dave Whipps ( http://www.davidwhipps.com/studio6 )
Local Designer Social Network
- Ben Bradford ( http://benrocksmmds3802.blogspot.com/ )
Portfolio
- Joy Christensen ( http://jchri63.wordpress.com/ )
Undecided
- Damian Carroll ( http://stoodio6.blogspot.com/ )
Magazine
- Grand Van Zutphen ( http://grantvzmajorproject.blogspot.com/ )
Music Video
- Jessica van Wegberg ( http://jessicavanwegberg.blogspot.com )
Title Sequence for a Film
No Details as of Yet
- Tom Ireland ( http://tomirelandsstudio6blog.blogspot.com/ )
- Robert Ninness ( http://rninne-mmds3802.blogspot.com/ )
- Dead Dredge ( http://ddredge22.blogspot.com/ )
- Anthony Massingham ( http://amassingham.blogspot.com/ )
Sound-Reactive Visuals
- Tim Cooper / Brendan Lidster ( http://timandbrendansstudio6.blogspot.com/ )
Business Ranking Social Network
- Dave Whipps ( http://www.davidwhipps.com/studio6 )
Local Designer Social Network
- Ben Bradford ( http://benrocksmmds3802.blogspot.com/ )
Portfolio
- Joy Christensen ( http://jchri63.wordpress.com/ )
Undecided
- Damian Carroll ( http://stoodio6.blogspot.com/ )
Magazine
- Grand Van Zutphen ( http://grantvzmajorproject.blogspot.com/ )
Music Video
- Jessica van Wegberg ( http://jessicavanwegberg.blogspot.com )
Title Sequence for a Film
No Details as of Yet
- Tom Ireland ( http://tomirelandsstudio6blog.blogspot.com/ )
- Robert Ninness ( http://rninne-mmds3802.blogspot.com/ )
- Dead Dredge ( http://ddredge22.blogspot.com/ )
Monday, August 13, 2007
Project Choices
So I've thought of something. It came to me during todays lecture. It requires a bit of history though. It's all about some Not So Impressive Entertainers.Back in early July I needed something to help keep me sane over the holidays. So, inspired by one of my favourite artists ( Scott C ) I decided to create a "series" of images. And so I came up with the "Not So Impressive Entertainers". A collection of the worst, most pathetic, and unusual entertainers that one could think of. Ideally I wanted to come up with about 100 of them, get them all printed out and displayed somewhere.
So, During todays lecture, it came to me. I could do these as my project!
To give you an example, here are the first 6 NSIEs ( and the blogposts that accompany them, ensure you read the comment reply from "wonko" as he's been providing little biographies as they come :
I. Pretty Ordinary Hat Man
II. Squid Man
III. The Pied Piper of Greater South-East Queensland
IV. Charles and his Amazing Performing Pancreas
V. Lionel the Brussel Sprout Tamer
VI. Sicko the Clown
So I'm mainly posting this here so you can see my thought process and the options I'm going through. Both projects have pros and cons. However I think the way I'm leaning at the moment is to keep NSIE as a side-project, a more personal thing - and focus all my efforts on my sound-based project.
Any thoughts ?
To quote a friend , "Tap tap, is this thing on ?"
- Anthony
Subscribe to:
Posts (Atom)
