Final Post

Artist’s Statement:

Differing slightly from our original plan, our program allows the user to see a visual representation of a note on a display (as well a hear the actual note) when you play it on a midi keyboard. Our program uses the qualities of the note to determine its color, and position on the display. Additionally, our program includes a GUI control panel so that the user can restart their display with ease.

Our program acts as a way for users to better understand and interpret the music that they are creating. By both hearing and seeing musical creations, we can better interpret them.

Users Guide:

Open the program in Jython Environment for Music and press play.
Press Play on the user control panel
Select your current midi controller from the drop-down list
Enjoy the program!
When you are ready for a blank canvas, press restart on the control panel

Code Repository

Errata:

While no features of our final program are broken, we are not entirely satisfied with our latency issue. There is a slight delay between when the button on the midi keyboard is pressed and when the note sounds/shape appears. This latency gets progressively worse as more shapes are added to the display. This is likely because the information of each note is run through several different functions before its final return is made.

With little to no knowledge on programming efficiency, we did not know how to fix this.

Another issue we are experiencing is the playback of notes. If the user presses 3 keys at once on the midi controller, the program will play back the notes separately and they sound at different times. This is likely due to the lack of threading in our program. Since it is only able to process one note at a time, the notes play one at a time.

Occasionally, an individual graphic will have a glitch and disappear from the display.

Demonstration Video

Getting the final code ready

This week we have been mainly working on removing any bugs and getting our final submission ready. One thing that was clearly an issue was the latency in our program. Ben recommended revising the gradient function so that it only used timers rather than time.sleep. This was somewhat difficult as the structure of the function did not work with timers. I re-wrote the function like this:

	from gui import *
	from timer import *

	d = Display("Gradient", 300, 300)
	rect = Rectangle(0, 0, 300, 300, Color.BLACK, True, 1)
	d.add(rect,0,0)

	r = 0
	color =None


	# This function either updates the color from red to blue or blue to red using if statements
	# The arguments are the r variable which is used to change the color by subtracting it or
	# adding it to the
	def updateColor():
	global rect, color, r
	#From blue (0,0,255) to red (255,0,0)
	if color == Color(1,0,254):
	red = r
	green = 0
	blue = 255 – r
	color = Color(red,green,blue)
	rect.setColor(color)
	r = r + 1
	if color == Color(254,0,1):
	r = 0
	color = 0
	pass
	else:
	color = Color(1,0,254)

	#for count in range(0,3):
	#From red (255,0,0) to blue (0,0,255)
	else:
	red = 255 – r
	green = 0
	blue = r
	color = Color(red,green,blue)
	rect.setColor(color)
	r = r + 1
	if color == Color(1,0,254):
	r = 0

	# Calls the updateColor function continuosly after a time interval
	t = Timer(10, updateColor)
	t.start()

view raw

Faster Gradient

hosted with ❤ by GitHub

I think this will run somewhat fast but our program still has considerable latency when a note is played. I do not think we have the time or ability to make this program run faster.

One of our stretch goals was to run an animated gradient on each shape on the display, however, based on how the program is currently running, I think it will have even more delay if we were to add this feature.

Regardless, we are nearing completion and our program is doing what we expected it to.

New developments

This week, we have made some changes to what we expect our final program to do. Our biggest issue was playing back a midi file and simultaneously calling a drawShape() function. The MIDI library has a function: ( *midi device*).onNoteOn( *function*) which calls a function whenever a note is played on a midi controller. The issue is that we intended our program to do this but with a midi file. The main issue surrounding this is dealing with rests in the midi file (we didn’t want our program to ignore rests). The one possibly feasible solution to this is to find a way to have a Digital Audio Workstation set up to output to our program as if it was a midi controller. I’m not entirely sure this is possible. I have reached out to the Apple Logic Pro X to determine if this is doable. In the meantime, we are changing the goal of this program to be more of an aid to live performance. It will create the displays in real time as someone plays on a midi keyboard.

One of the main functions I worked on this week is an animated color gradient. This will be the background of our display.

	def updateColor(shape):
	while b:
	for count in range(0,3):
	#From red (255,0,0) to blue (0,0,255)
	for i in range(0,255):
	red = 255 – i
	green = 0
	blue = i
	color = Color(red,green,blue)
	shape.setColor(color)
	time.sleep(0.01)
	if color == Color(1,0,254):
	#From blue (0,0,255) to red (255,0,0)

	for i in range(0,255):
	red = i
	green = 0
	blue = 255 – i
	color = Color(red,green,blue)
	shape.setColor(color)
	time.sleep(0.01)

view raw

Animated Color Gradient

hosted with ❤ by GitHub

A stretch goal for this function would be to have it update from side to side so that it has a more dynamic look.

Additionally, I have been working on putting together our setUp() function.

At this point, I feel like the completion of this program is realistic and within our ability. From here on we need to devise the main function that will modify and add our graphics to the display.

Blockers

Currently, our group has a few blocks that are preventing us from continuing on the JEM portion of our project. Mainly, the midi library has a function onNoteOn( *function* ) that is calls a function whenever a midi note is played. The problem is that this does not accept a midi file as input. I suspect that there is a simple solution for this (and will be working with Tanner on this later today). If a function that calls a function for each note in a midi file does not exist, there is a potential work around that would involve looping through each note of the note list, calling a function to make a graphic, the pausing the loop for the duration of the note. If a simpler option exists, it will likely run quicker than this loop.

In the mean time, I have written two pieces code pertaining to the graphics. The first determines what note is being read, and from that choses what shape the note will represent.

	def shapeChoice(shapeVar):
	shape = getNote(note)


	def getNote(note):
	shape = none
	noteValue = none
	pitch = note.getPitch()
	if pitch in (0,12,24,36,48):
	noteValue = c
	if pitch in (1,13,25,37,49):
	noteValue = db
	if pitch in (2,14,26,38,50):
	noteValue = d
	if pitch in (3,15,27,39,51,63,75,87,99,111,123):
	noteValue = eb
	if pitch in (4,16,28,40,52,64,76,88,100,112,124):
	noteValue = e
	if pitch in (5,17,29,41,53,65,77,89,101,113,125):
	noteValue = f
	if pitch in (6,18,30,42,54,66,78,90,102,114,126):
	noteValue = gb
	if pitch in (7,19,31,43,55,67,79,91,103,115,127):
	noteValue = g
	if pitch in (8,20,32,44,56,68,80,92,104,116):
	noteValue = ab
	if pitch in (9,21,33,45,57,69,81,93,105,117):
	noteValue = a
	if pitch in (10,22,34,46,58,70,82,94,106,118):
	noteValue = bb
	if pitch in (11,23,35,47,59,71,83,95,107,119):
	noteValue = b


	if noteValue in (c,eb,gb,a):
	shape = square
	if noteValue in (db,e,g,bb):
	shape = square
	if noteValue in (d,f,ab,b):
	shape = pentagon

	return shape

view raw

Shape Choice

hosted with ❤ by GitHub

Additionally I found a method of making a gradient of colors and choosing a color based on the pitch of a note. The function works with any color combination but for this example we will say it goes from red to blue. This function will align the lowest note to red and the highest note to blue. Any notes in between will be assigned a color proportional to their distance from either end of the gradient.

	colors = []

	def getColor(note):
	import colorsys
	for i in range(101):
	rgb = colorsys.hsv_to_rgb(i / 300., 1.0, 1.0)
	colors.append(i, [round(255*x) for x in rgb])

view raw

Color Gradient

hosted with ❤ by GitHub

One goal I would like to achieve would be making the background an animated gradient. This background would smoothly shift between two colors when a note of certain parameters is played.

Preparation

Today and yesterday, I have been working on familiarizing myself with JEM. JEM has many useful features, however, several questions have surfaced in this process. Mainly, JEM does not have the same standard library that JES does so I was having difficulty using simple functions that we have used all year. I suspect that the solution may be as simple as importing a library. I will solve this tomorrow in lab.

Because I could not figure out how to import files, I was not successful in playing a midi file in JEM, however, I have researched it on the Jython Music website and feel confident on how it can be done.

I suspect the next big hurdle will be figuring out how to create a dictionary of the notes. The plan of our program is to assign values for the qualities of each midi note, which will require a dictionary of notes.

Project Introduction

Artist’s statement:

As an avid concert go-er and music listener, I speak from my personal preference to say that music is more enjoyable when there are synchronous motion graphics. The problem is that most music listeners do not have access to professional software that does this. Our program will be simple, user friendly, and allow music listeners to have a concert like experience on demand. As waveform analysis seems to be a quite daunting task, our program will run off of midi files. While the user will not be able to chose any mp3 file, the use of midi files will allow our program to be precise with note and beat alignment.

Our program will consist of 2 displays. One being the main display which will show the motion graphics, and the other being the control panel. The control panel will be a simple display with several options that will modify the graphics as well as a play and pause button.

Work schedule:

We plan to meet up at 1 p.m. on Thursdays, for 2 hours, and again on the weekends if necessary.

What we need to learn:

While some understanding of how midi files work will be necessary, a bulk of the work of this project will depend on learning how to use Jython Environment for Music. This is necessary for the use of midi files and playback. Additionally, we will need to better our understanding of graphics and how to make them change size, color, etc in a smooth and pleasing manner.

Our Team Members:

Calvin Ritger
Owen Grace
Sidd Nair
Tenny McFarlane

Tenny and I have some experience in with music production and processing; we will be able to teach Sidd and Calvin on this aspect of the project. We will all have learning to do on the rest of the code.

Stay Tuned!