Transfer Learning
Transfer Learning
Obtain the “Audio Cats and Dogs” dataset and divide it into training and test sets of approximately equal cumulative duration.
Obtain the “Audio Cats and Dogs” dataset and divide it into training and test sets of approximately equal cumulative duration.
In[]:=
ResourceObject["Audio Cats and Dogs"];{dogDuration, catDuration} = Values@Normal[ GroupBy[ResourceData["Audio Cats and Dogs"], "Label"][All, Mean@*Duration, "Audio"]];SeedRandom[42];classLength=93;{catTrain,catTest}=TakeDrop[RandomSample[ResourceData["Audio Cats and Dogs"][Select[#Label==="cat"&]]],Round[classLength*dogDuration/catDuration]];{dogTrain,dogTest}=TakeDrop[RandomSample[ResourceData["Audio Cats and Dogs"][Select[#Label==="dog"&]]],classLength];balancedTrain = RandomSample[Join[catTrain, dogTrain]];SeedRandom[42];catTestex=RandomSample[catTest,10];balancedTest = RandomSample[Join[dogTest, catTestex]];
Now for the transfer learning.... Create the AudioIdentify feature extractor net which will be used as the starting point for training.
Hint:Extractthecorenet,chopofftheclassificationlayersandreconstructthefullnetusingtocomputefeaturesoneachchunkandtoaveragethesesequencesoverthetimedimension.Theoutputshouldbeasingleflatvectoroflength1664.
In[]:=
net=NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data","Size""Large"];mainNet=NetExtract[net,{1,"Net"}];featureExtractor= NetChain[NetMapOperator[NetDrop[mainNet,-3]], AggregationLayer[Mean, 1], FlattenLayer[],"Input"net[["Input"]]]
Out[]=
NetChain
Create the net that will be in charge of classifying cats and dogs.
Try to implement the following structure:• Start with a form of regularization (such as Dropout) applied to the incoming features (to avoid overfitting the small dataset and to make the net’s prediction more stable by averaging the predictions of an ensemble of smaller nets).• Project the learned features onto a smaller space (try a magnitude similar to the number of classes) to perform classification..
• Apply
to the final output vector so that it can be interpreted as a probability distribution (use a layer that implements the
equivalent of
Normalize[Exp[..], Total]
).
• Decode the probability distribution into class labels using a In[]:=
leakyRelu = ParametricRampLayer[{}, "Slope" -> 0.1, LearningRateMultipliers -> 0, "Input" -> 2]
Out[]=
ParametricRampLayer
In[]:=
classifier = NetChain[{DropoutLayer[0.2],leakyRelu,BatchNormalizationLayer["Input" -> 2],SoftmaxLayer[]}, "Output" -> NetDecoder[{"Class", {"dog", "cat"}}]]
Out[]=
NetChain
Train the classification layers for the new task.
Train the classification layers for the new task.
Hint: There are a couple different ways to accomplish this: format.)
1. Precompute the features using the feature extractor and train only the classifier net.
( An advantage of this approach is that you can save the features in 2. Train the full joint feature extractor + classifier net with
specified to avoid relearning weights in layers except for the final
classification layers.
In[]:=
trainedNet = NetTrain[classifier, balancedTrain,LossFunction -> "Loss2"]
Out[]=
$Failed