Projects

Every student is required to complete a project in the second half of the semester. The project combines the development of some software with the creation of music using that software. Students may form teams, but it is essential that team members each take responsibility and credit for some aspect of the project. Team members may chose to create separate compositions and only collaborate on software. Alternatively, a "composition team" may form from different technical projects to combine multiple techniques in a single composition.

An Example Project

An example is the best way I can describe the features of a good project, so here it is: The latest Nyquist IDE (based on a project from a previous class) is great for editing text and displaying sounds (at least short ones), but there is no support for entering functions of time. This project adds a new view to the IDE allowing the user to edit piece-wise linear functions of time. Each function has a name, and multiple functions can be stored in one file. The functions are implemented by defining a Lisp function containing a call to PWL. If the user creates a time function named foo in the IDE, then the user can write (foo) in Nyquist code to obtain the function. When the user opens one of these files in the IDE, the IDE parses the code and can then display functions graphically. The user can add, move, and delete breakpoints, change function names, copy functions to a new name, etc. It would be great to be able to display multiple functions in different colors so they can be edited in concert. Also, it should be possible to snap to user-controlled grid lines, zoom in or out, change the vertical axis, etc.

Having convenient direct-manipulation access to functions of time opens up a new way to control synthesis. Complex sounds with several parameters can be made to evolve according to time-varying controls that are edited graphically. These parameters in the editor become a sort of musical score notation. The composition associated with this project uses this approach to composition, resulting in music with slowly-varying, but very carefully controlled and refined textures.

Why is this good? It has the following features that I am looking for:

	It's Computer Music: there's computation, and there's music.
	It requires some thought about music representation and music processing.
	It extends Nyquist (or Audacity). Notice that the project is not even a complete stand-alone system, only the extension of one, so the student does less work and leverages what's already there.
	The project isn't too easy, but it's not impossibly hard.
	Students will be able to use this in the future.
	The music is integral to the project. It's almost obvious what the music has to be and how the implementation is essential to the music creation. The music and the technology guide each other.
	The music is not conventional, popular music that might be better played on a piano. Instead, the music could only be created by computer.

Your project will probably not have to have all the features, but I hope this give you a good idea of what I am looking for. Review the list above with respect to your project.

The Project and its Parts

The project has several parts:

The Project Proposal (due March 7)

A proposal is required. Your proposal should state clearly:

	the general nature of what you will implement,
	what code, journal articles, etc. you will rely on, port, implement, modify, etc.,
	a significant milestone that you will report/deliver as the interim report
	a specific list of deliverables for your final project submission
	how will you measure your success? E.g. what functionality will you demonstrate?

All projects must be approved by the instructor, so this really is a proposal. I will usually suggest some changes, and if the project really seems out of line, I'll let you know early and without penalty. (A clear proposal for a bad project gets a good grade and a request for another submission, a vague proposal for a good project gets a bad grade and an OK to proceed).

Submit your proposal (as a PDF) to the server in folder hw8a using the filename andrewid_hw8a.pdf, as usual. The deadline is 10:00pm, Wednesday, Mar 7th.

The Interim Report (due Wednesday, April 4)

An interim report is required. Ideally, you will have completed the milestone described in your project proposal. If you fail, you should describe clearly what you have done, what problems you encountered, whether this will affect your ability to finish the project as planned, and any modifications (subject to instructor approval) you feel you should make to the project. You should submit interim code, sound results, and other data to substantiate your report. Put everything in a zip file and sftp/scp to hw10 on the server.

The Composition (due Wednesday, April 25)

Ideally, your composition project will demonstrate what you have learned in the class. Aim for a length of about 5 minutes. Longer works are acceptable, but length will be considered when we program the final concert. You should have some interesting ideas from the in-class listening sessions, and your project implementation will also guide you. It is probably better to make a piece in the tradition of contemporary art music than to create a rap tune, but the instructor will do his best to stay open-minded. Regardless of the musical genre, the music should reflect careful thought and imagination. On the technical side, the music should show some mastery of the techniques taught in class. E.g. a piece that is acoustically dry, monophonic, exhibits clipping, shows no variation in timbre or articulation, and which uses a completely flat tempo will not be acceptable. A piece should include appropriate reverberation, stereo placement, effective changes in dynamics (without clipping), timbres that evolve in interesting ways. and sounds shaped into flowing phrases. These are not easy requirements -- it takes time and effort to produce musical results from computers. Grading will be based on how well the piece communicates a musical idea, i.e. the intentions of the composer, and the extent to which the piece was carefully constructed using techniques learned in class and developed in the project. To a large extent, the grade will be based on effort. It is usually clear whether a piece was thrown together quickly or whether many details were carefully adjusted. If you feel you have worked very hard and still are not satisfied with the result or think that your effort is not apparent in the composition, you should submit early drafts and explain what you have done to refine the work. Copy the final piece to the hw11 folder on the server.

The Class Presentation (slides due Sunday, April 29, talks are Tuesday, May 1 and Thursday, May 3)

You will present your project to the class. Plan for 3 minutes. This is a very short presentation. You should present your work in three parts. Use slides (preferably PowerPoint) to supplement your talk.

Motivate your work. What purpose (outside of your education) is served by your project? What existing problem does your work solve? (1 slide)
State your approach and goals. How does your project solve the problem just stated? Be sure to state clearly and with some detail exactly what you did. (1 slide)
What is the state of your project? Tell what is complete and working as well as what would be the next step if you (or someone else) were to continue working. (1 slide)

Team projects will get 3 minutes per person, and each person in the team should talk about their individual contributions or the aspect of the project they know the best.

You must submit PowerPoint or PDF slides in advance. These will be organized into one monster file so that we have zero setup time between class presentations.

If you have sound file examples (recommended):

	collect them all in a directory named your-andrew-id
	at the appropriate points on your slides, put something like "play: filename"
	submit a zip file of the whole directory to hw12/andrewid_hw12.zip in the usual way
	we will unzip the file and it will be available for your class presentation
	do not assume that links from PowerPoint to soundfiles will be maintained when your slides are merged into one big file.

Assuming a 3-minute talk, you can rehearse it 5 times in 15 minutes. Do it.

The Concert (May 4, 6PM (sound check), 7:30PM (concert), Alumni Concert Hall, College of Fine Arts)

Your friends are invited. Attendance is required. This is a large class, so it will not be possible to perform every piece. The instructor and teaching assistants will select pieces if necessary to keep the program length reasonable. Or maybe we should have a mini-festival at the Quiet Storm where people can drop in? Ideas and organizers are welcome.

The Final Project Submission (due Friday, May 4)

Your goal is to make a complete package of software, documentation, example code, and sound examples. For example, if your project is to create a phase vocoder, you should submit the software implementation, document how to apply the vocoder and use all the parameters, provide code examples that apply the vocoder to a test sound, and one or more sounds that illustrate the effect of the vocoder. Another student should be able to use the vocoder given your final submission.

A List of Project Ideas

Feel free to ask the instructor for more details on any of these.

Nyquist-related

Extending Nyquist by writing functions in Nyquist:

	Port instruments from articles or other systems
	Develop library of effects with examples and documentation
	Improved reverberation functions
	Spectral features as control parameters for synthesis. What does this mean? Many interesting forms of synthesis are based on the analysis of one sound to obtain data to control another. In this case, spectral features like brightness (the average frequency weighted by amplitude), flux (derivative of spectral amplitude), and frequencies of peaks (also known as peak-tracking) can be used to obtain control information. A project would consist of some analysis software to extract spectral features and some compelling examples of sound synthesis using those features.
	Implementation or ports of spectral manipulation software (e.g. from Eric Lyon and Chris Penrose)
	Rythm guitar synthesizer that takes a strumming pattern, a chord progression, and a guitar string synthesizer function and computes a sound. This should make it possible to create a range of guitar performances without having to notate the time and pitch of each note.
	One thing that distinguishes live acoustic music is that every instrument radiates sound from a different location. In principle, synthesized instruments should sound more "live" if they are placed at different locations in a simulated room. To test this prediction, get room impulse responses (I have some recordings from Prof. Driessen at U. Victoria - RBD), and use convolution to "place" instruments in a room. Form a simulated ensemble (1) with each instrument in a different simulated location and (2) with all instruments at the same simulated position. Run some listening tests to see if you hear a difference. (This is an easy conference paper that I'd be happy to coauthor. I hope we do it first!)
	Tom Neuendorffer and Roger Dannenberg recently recorded Tom's hammered dulcimer using both wood and felt hammers and using single and multiple bounce playing. An interesting project would be to trim the recordings to keep only the good stuff, identify the note onsets and durations, build a hammered dulcimer instrument in Nyquist that pulls in and mixes samples from disk, possibly add some "extended dulcimer" functions like vibrato, pitch bend, an extended range, and string damping using envelopes, and create a demo piece with some (synthetic) virtuoso hammered dulcimer playing.

Extending Nyquist by writing functions in C

	Phase Vocoder support
	Physical models from Perry Cook (partially implemented)
	MQ analysis/synthesis (or the more advanced Loris system or Juan Pampin's system)
	Support VST plugins in Nyquist.
	Allow Nyquist to import csound unit generators
	Support SDIF I/O (see SDIF library from UCB)
	Add support for Allegro score representation
	I have some code from a previous project to read Gravis Ultrasound Sound Font files. The goal is to be able to import sounds, envelope data, etc. from these files to construct a Nyquist function that can play instruments described by these files. This would instantly expand Nyquist's library of conventional instrument sounds.
	Nyquist uses PortAudio for audio output, but it is based on an old version of PortAudio, modified to handle the synchronous, or blocking, API style. This is now supported in PortAudio, so it would be nice to update Nyquist to use the latest, and more complete, version of PortAudio.

Extending the Nyquist IDE by writing functions in Java

	Add GUI controls to Nyquist. Nyquist already has an array of floats that can be accessed at run time to get scalar (float) values or time-varying signals. These "sliders" can be set by sending special strings from the IDE or by sending OSC messaages over the network. What is lacking is some Java IDE code to create sliders, save configurations in the workspace.lsp file, etc. Probably slider creation should be done by graphical interaction, but the configuration should be saved in the workspace (see the envelope editor and EQ editors for examples).
	The Nyquist IDE has plenty of rough edges. I don't like to encourage "cosmetic coding" as a class project, but if you are an expert Java GUI programmer, there are a number of problems that could use your skills: The window layout is different on every system, but there must be a way to do a good job of window sizing and placement. Fonts are terrible on Linux. The sliders used in the Browser and other places are not compact -- it would be nice to have sliders that look like those in professional audio applications. Parenthesis balancing has come a long way from the original implementation, but it could still use some improvement. In particular, when the cursor is positioned after a close parenthesis, the corresponding open parenthesis should be highlighted. Graphical object event handling in the code is inconsistent. A "design pattern" for creating and handling graphical objects to guide future implementation and rewrites would be nice.
	The EQ editor is preliminary -- it should allow the user to specify the range of frequencies, the number of frequency channels, and support multiple audio channels, either locked together or independent.
	The envelope editor is awkward, lacks labels, and editing features, but it's a good start. A good project would propose an improved design and implement a replacement.
	The IDE's "piano roll" score editor is a rough start, and the user interface needs a lot of work, including scroll bars, labels, zoom, selection, access to attributes other than pitch and time, the ability to quantize or snap to grids, etc. A project in this area needs to make a specific interface design and implementation plan.
	Spatial positioning of sound is difficult to do in Lisp. A graphical editor might help to position sound sources in stereo or 5.1 or whatever. Especiallly interesting would be the ability to specify moving sound sources and to edit their paths.
	There have been several attempts to create a graphical score entry system. The idea is that you represent different Nyquist behaviors with different shapes and/or colors on a pitch vs. time graph. This is similar to a piano-roll editor except the shapes or colors indicate different behaviors (Nyquist functions). There may be other parameters encoded in the height, color, texture, or shape of each graphical object in the score. Each object can be edited using a property sheet, and all the properties are passed to Nyquist as parameters to the behavior, and behaviors are organized in terms of time and duration according to their (editable) graphical layout. The main idea is to provide some graphical score editing but to avoid strong ties to traditional music notation or assumptions that music is best represented in terms of pitch and time (only). Maybe this should be a superclass of the piano roll editor or the two should share an abstract superclass.
	The Nyquist IDE has a preferences dialog with a couple of things you can change. It would be great to have a graphical way to set things like the normalization style, default audio and control sample rates, whether to play sounds or save audio to disk or both when calling the play function, whether to apply reverb and/or EQ to the output signal when using PLAY, a default sound file directory, whether to print a stack trace when an error is encountered, etc. (All of these things can be set in Nyquist, but most users do not know how.)

Audacity-related -- In general, students have had a difficult time getting up to speed and contributing to Audacity, but at the same time, there is much room for improvement

Scales to display time, beats, samples, etc.

Add breakpoint envelope editor to Audacity

Ability to edit a tempo track and display beat locations

Add MIDI editing functions (requires better event representation)

Audacity editing interface for editing live recordings (consult with Riccardo Schultz)

	cross-fade editor dialog
	play list interface
	play list engine to compute and play segments and crossfades

Tutorials/Presentation

Develop animations, diagrams, and sounds to illustrate course concepts, such as:

Sampling theory

Synthesis methods

FFT

If you do something like this, you should be sure to get the content and presentation method approved.

Other

	Develop open-source beat-tracking software.
	Module to send digital audio over networks
	Add support for SDIF I/O to Audio File Library
	Build system to segment a performance into individual notes
	Play synchronized midi and audio using PortMidi and PortAudio