IMG_2035.jpg

Blog

Generative Music - Melody RNN

Generating Long-Term Structure in Songs and Stories, 2016

(referenced Composing Music With Recurrent Neural Networks,2105 and Neural Machine Translation by Jointly Learning to Align and Translate,2014)

Basic RNN

input :

a one-hot vector of the previous event , the label was the target next event.

event :

-note-off (turn off any currently playing note)

-no event (if a note is playing, continue sustaining it, otherwise continue silence)

-note-on event for each pitch (which also turns off any other note that might be playing)

Lockback RNN

new inputs:

-events from 1 and 2 bars ago β€”>(mirrored or contrasting melodies)

-whether the last event was repeating the event from 1 or 2 bars before it β€”>(repetitive or non-repetitive state.)

-the current position within the measure
Step 1: [0, 0, 0, 0, 1]
Step 2: [0, 0, 0, 1, 0]
Step 3: [0, 0, 0, 1, 1]
Step 4: [0, 0, 1, 0, 0]

new labels:

-The label to repeat the event from 1 bar ago.

-The label to repeat the event from 2 bars ago. 

β€”>his allows the model to more easily repeat 1 or 2 bar phrases without having to store those sequences in its memory cell.

Attention RNN

Neural Machine Translation by Jointly Learning to Align and Translate

[an encoder-decoder RNN

the model uses attention to look at all the encoder outputs during each decoder step.]

outputs from the last π‘›n steps when generating the output for the current step.

Screen Shot 2018-11-15 at 6.24.10 PM.png

Step 1: [1.0, 0.0, 0.0, 1.0]
Step 2: [0.0, 1.0, 0.0, 1.0]
Step 3: [0.0, 0.0, 0.5, 0.0]

π‘Žπ‘‘π‘– =[0.7, 0.1, 0.2]

Step 1 (70%): [0.7, 0.0, 0.0, 0.7]
Step 2 (10%): [0.0, 0.1, 0.0, 0.1]
Step 3 (20%): [0.0, 0.0, 0.1, 0.0]

β„Žβ€²π‘‘ =[0.7, 0.1, 0.1, 0.8]

The β„Žβ€²π‘‘ vector is concatenated with the next step’s input vector and a linear layer is applied to that concatenated vector to create the new input to the RNN cell.

This helps attention not only affect the data coming out of the RNN cell, but also the data being fed into the RNN cell.

Encoding

melodies_lib.Melody format (-2 = no event, -1 = note-off event, values 0 through 127 = note-on event for that MIDI pitch)

Twinkle Twinkle Little Star

--primer_melody="[60, -2, 60, -2, 67, -2, 67, -2]"

Thoughts

Total new transition

Text + Accent β€”> Musical RNNs = ?

YOU JIN CHUNG