Transcribing is vital for sharing music with the world. My project consisted of creating a sample melody and generating the corresponding sheet music for a provided melody. The process involves recognizing the notes being played in the melody and then creating a graphic with the notes on a staff. This idea was sparked by my desire to be able to transcribe music into sheet music. By creating this project I have been able to take the first step of this process.
Introduction
Introduction
Motivation
Motivation
My interest for the project was sparked by my love of Kyle Landry’s original piano compositions. Most of these compositions do not have available sheet music which required me to start learning to transcribe music by ear. Transcribing music can be hard; it requires a well trained ear, a skill that takes a long time to develop. Even with proficiency, it is hard to identify exactly what is being played on both hands. I decided to make a project that would take the first step in automating the transcribing process by being able to transcribe melodies.
General Music Background
General Music Background
Semitones
Semitones
In music there is the concepts of semitones: A semitone can be represented by moving up or down a half step. This can be seen in the visual below:
Sharps
Sharps
Notes that are raised a half step are called sharps. The way to notate this on the staff is using the # symbol.
Key
Key
In music a key represents the music revolves around. It also represents also represents all the accidentals in the key. Since we are going to assume we are in C major, we will have no sharps and flats.
Staffs
Staffs
To properly notate a staff in music you need to define a clef which states the range that the notes will be played in. The two clefs that will be used in this project will be the treble and bass clef.
Stems
Stems
To properly notate a stem on a note you need to know its position. If the note is below line 3 on the staff, the stem points up. If the note is on the third line or above it, the stem points down.
Creating Sample Audio
Creating Sample Audio
In[]:=
melody = Audio[Sound[{SoundNote["A#4",0.5],SoundNote["A4",0.5],SoundNote["F4",0.5],SoundNote["A4",0.5],SoundNote["E3",0.5],SoundNote["C4",0.5], SoundNote["C#4",0.5],SoundNote["G#3",0.5]}]]
Out[]=
This piece of code generates a melody with these specific notes using built in audio functions :
Pitch Detection
Pitch Detection
This function takes in an audio input and returns a list of semitones for the audio. This is accomplished through PitchRecognize, which returns a time series of frequency, where the pitches are then extracted through data processing.
In[]:=
identifyNote[audio_]:=Module[{ts,anom,filter,timeNote},ts=PitchRecognize[audio,"SoundNotePitch"]["Path"];anom=DeleteAnomalies[DeleteCases[ts,{_,Missing[]}]];filter=SplitBy[anom,Floor[#[[1]],0.5]&];timeNote=Table[{Ceiling[filter[[i,-1,1]]-filter[[i,1,1]],0.25],filter[[i,1,2]]},{i,Length@filter}];Select[timeNote,#[[1]]>=0.5&][[All,2]]]
The output is a list of semitones:
In[]:=
pitches=identifyNote[melody]
Out[]=
{10,9,5,9,-8,0,1,-4}
Sheet Music Generation
Sheet Music Generation
Creating the staff
Creating the staff
This function takes in a clef and outputs a staff with the clef on it . The function creates 5 lines and then overlays the clef graphic on top of these 5 lines .
In[]:=
staff[clef_]:=ImageCompose[Graphics[Text[Style[clef,20],{1.4,.5}],PlotRange->{{1,20},{-2,2}}],Graphics[Table[Line[{{1,i},{20,i}}],{i,0,1,0.25}],ImageSize->1000,PlotRange->{{1,20},{-2,2}}]]
Generating a treble clef;
In[]:=
staff[𝄞]
Out[]=
Placing notes on the staff
Placing notes on the staff
Finding the coordinates for the notes
Finding the coordinates for the notes
To find the coordinates for the notes we need to determine the x position, the y position and the clef that the note belongs in. The y position is determined by the pitch value. The x position increments based on its index relative to the total amount of notes we have. To determine the clef that the note belongs to we assign it a binary value (0 or 1); if it is 1 we draw the note, and if it is 0 we do not.
In[]:=
coordinates[pitches_]:=Module[{x=0.6,y=0.125,octaves,cords,treble=Association[0->-0.25,1->-0.25,2->-0.125,3->-0.125,4->0,5->0.125,6->0.125,7->0.25,8->0.25,9->0.375,10->0.375,11->0.5], bass=Association[-1 -> 1.125, -2 -> 1.0, -3 -> 1.0, -4 -> 0.875, -5 -> 0.875, -6 -> 0.75, -7 -> 0.75, -8 -> 0.625, -9 -> 0.5, -10 -> 0.5, -11 -> 0.375, -12 -> 0.375, -13 ->0.25, -14 -> 0.125, -15 -> -0.125,-16->0,-17->0]},octaves=Quotient[#,12]&/@pitches;cords=Table[Which[pitches[[n]]==Infinity,{{1,1}, 0},pitches[[n]]>=0,{{2.5+(x*(n-1)),treble[Mod[pitches[[n]],12]]+((y*7)*octaves[[n]])}, 1}, pitches[[n]]<0, {{2.5+(x*(n-1)),bass[Mod[pitches[[n]],-12]]}, 1}],{n,Length[pitches]}]]
The output of the function is a list of coordinates with the corresponding binary values;
In[]:=
cords=coordinates[pitches]
Out[]=
{{{2.5,0.375},1},{{3.1,0.375},1},{{3.7,0.125},1},{{4.3,0.375},1},{{4.9,0.625},1},{{5.5,-0.25},1},{{6.1,-0.25},1},{{6.7,0.875},1}}
Creating the note-heads
Creating the note-heads
To create note-heads we need to determine if the note-head should be drawn in the clef, and if the note-head requires a sharp sign next to it. The sharp for the note is determined by the semitone value of the note. The note is drawn if the the number corresponding to the coordinates is 1. If it is 0, I create a blank note to act as a placeholder.
In[]:=
noteHeads[cords_,pitches_]:=Module [{sharps=Association[0->False, 1->True, 2->False, 3->True, 4->False, 5->False, 6->True, 7-> False, 8->True, 9-> False, 10->True, 11 -> False]}, ImageCompose[Graphics[Table[If[cords[[n,2]] == 1,{Black,Disk[cords[[n,1]],0.125]}, Nothing],{n,Length[cords]}],PlotRange->{{1,20},{-2,2}},ImageSize->1000],Graphics[Table[If[cords[[n,2]] == 1,If[sharps[Mod[pitches[[n]],12]], Text[Style[♯,23], cords[[n,1]]+{-0.25,0}],Nothing], Nothing],{n,Length[cords]}],PlotRange->{{1,20},{-2,2}},ImageSize->1000]]]
The output is a graphic of note - heads with sharps;
Creating the stems
Creating the stems
To create the stems we need to determine if the stem should be drawn and if the stem is an up stem or a down stem. We determine the direction of the stem by checking the y value of the corresponding note and seeing if it lies above the y value of the middle line of the staff. If the y value is greater than or equal to the y value of the stem, it points down. However, if it is less, the stem faces up.
This creates a graphic of stems:
Creating a list of treble or bass clef
Creating a list of treble or bass clef
To group the note-heads, the stems, and the staffs we need to determine how many note-heads and stems there should be per staff, and then create the proper amount of staffs. This amount is determined by the noteLimit. Then we group these separate elements together.
This creates a list of treble clefs that can hold up to 2 notes:
Combining the list of treble and bass clef into the final staff (grand staff)
Combining the list of treble and bass clef into the final staff (grand staff)
To create the grand staff I need to create a list of staffs for all the notes in the treble clef and all the notes in the bass clef. I divide the pitches into treble and bass by checking if the value is negative or non-negative. If the value is negative then it should be in the bass, and if it is non-negative it is treble. I then use the grouper function to generate a list of staff graphics with the notes. Finally, I group the staffs in these two lists to create the grand staff.
This is the original melody;
This is the sheet music:
Conclusion
Conclusion
Overall, I am very pleased with the outcome of this project. Although I was not able to create my original project goal which was to transcribe polyphonic piano music, I successfully transcribed monophonic music which would be the first step for transcribing polyphonic. Throughout this process, I gained a deeper understanding about the process of transcription like the importance of a spectrogram for identifying pitches. Additionally, the project provided me the opportunity to deeply improve my Wolfram skills.
Future Extensions:
Future Extensions:
◼
Rhythm
◼
Key signatures
◼
Dynamics
◼
Transcribing polyphonic piano music
Acknowledgements
Acknowledgements
I would like to thank my mentor William for all the help and my advocate Dugan. I would like to thank all the TAs for their help especially: Logan and Tyler for their consistent help with solving problems I encountered. Finally I would like to thank all the directors for organizing the program.
References
References
◼
Hamm, C. (2021). Half steps, whole steps, and accidentals. https://viva.pressbooks.pub/openmusictheory/chapter/half-and-whole-steps/
◼
Music 101: What is a sharp note? learn about sharp notes in music with examples - 2024. MasterClass. (n.d.). https://www.masterclass.com/articles/music-101-what-is-a-sharp-note
◼
Note stem direction– Mammoth Music. (n.d.). Retrieved July 11, 2024, from https://mammothmemory.net/music/sheet-music/reading-music/music-note-stem-direction.html
◼
Pouska, A. (n.d.). Keys in music. StudyBass. Retrieved July 11, 2024, from https://www.studybass.com/lessons/harmony/keys-in-music/#:~:text=In%20music%20a%20keyhttps://www.hoffmanacademy.com/blog/c-major-scale-on-piano/
◼
Pouska, A. (n.d.). Keys in music. StudyBass. Retrieved July 11, 2024, from https://www.studybass.com/lessons/harmony/keys-in-music/#:~:text=In%20music%20a%20key%20is,G%2C%20A%2C%20and%20B
◼
What are the clefs in music and how many actually are there? (n.d.). Classic FM. Retrieved July 11, 2024, from https://www.classicfm.com/discover-music/music-theory/clefs-treble-bass-alto-explained/
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
Generating sheet music for a provided melody
by Benjamin Sandler
Wolfram Community, STAFF PICKS, July 11, 2024
https://community.wolfram.com/groups/-/m/t/3217084
by Benjamin Sandler
Wolfram Community, STAFF PICKS, July 11, 2024
https://community.wolfram.com/groups/-/m/t/3217084