Blog Post 7

Data Issues:

This week came with a couple of setbacks and some pretty cool break throughs.

At the end of last week I had a pretty robust model that could acuratley classify a sound file on weather it had been recorded in one of two rooms.

After this I decided to up the ante by making a new model that would have not two, but 8 distinct rooms to classify between.
I got the model working on a few training samples but problem arose when I tried to train the model with more data, as the bigger database was actually too big to be loaded into RAM.
I fixed this by decreasing the number of files in the training database.
After a few test runs however it turned out that the model was far less accurate than the previous binarly classifier, and took a lot longer to train.

Cepstral Analysis:

After reading a few papers on music information retrival techniques I found that many projects utilise cepstral analysis to help corerelate audio simmilarity.

In DSP a cepstrum is ‘the result of taking the inverse Fourier transform of the logarithm of the estimated spectrum of a signal’ which sounds (and is) pretty complicated but it can kind of be thought of as a way to look at the rate of change of diffent spectrum bands.

By adding a prepreccessing step where I take the cepstrum of each sample in the traing set and generate a new data set using this information, I was able to increases the accuracy of the models classifcations and decreases the size of the database dramatically, saving huge ammounts of memory and training time.

A further step I might take is testing whether or not the auto-cepstrum (the cepstrum of the autocorrelation signal)
which is sometimes used in the analysis of singal data with echoes, produces more accurate results.

Next week I plan to tweak the model using tensorboard to analyse the sucess of slightly different models, and to begin working more with GNN to implement the model with ECL.

Blog Post 6

More Data

This week was all about generating more data.

- In order to train a more complex neural net with multipule outputs I needed a database of sound files with simulated recordings for multiplue room. However, going back and looking through my data base of convolved wav files, a lot of the files for certain IRs were corruppted and couldnt be used at all. On top of this I also wanted more data points to train the NNs on, as cutrrently I had about only 500 samples for each IR. 

- I spent the better part of the last week looking for and downloading more samples with wget to rebuild and update my database. 

After convolving each of the clean samples I currently I have around an hours woth of training data, for each of the 8 selected IRs.

Data Management

- Currently, I am running some tests on my data using tensorflow and python in jupyter notebook, however due to the amount of data I am analysing, the size of the data base is currently too big to be stored directly in RAM and throwing an error. Im going to try a few fixes next week and hope to have a multi output NN that can be translated to ECL and the HPCC clusters.

Blog Post 5

Week 7

Dataset management

This week I began using the tensorflow library with my data to generate some preliminary neural nets and test out which models performed best.

After looking at a plotted graph of my wav file data set I realised that the majority of useful information was contained in the first 4 seconds (or 4*44100 samples) of my Wav Files, so I wrote a quick bit of python to truncate all files to 4 seconds long.

Then, using the IR used to create the file as its classifier, I made a data set containing the 583 wav files from each of the IR_2 and IR_3 data sets to test my Neural Nets with.
Before adding the other 7 or so IRs into the mix I wanted to know whether or not a neural net could be trained to differentiate between two rooms.

Tensor Tests

After reading a few papers on the subject I found that a sound file can be thought of as a single dimensional temporal data set, as the data essentially represents the amplitude of a signal that evolves over time. Recently a lot of projects such as WaveNet and this Pydata project (https://www.youtube.com/watch?v=nMkqWxMjWzg) have found a lot of success working with this type of temporal data by utilising 1 dimensional convolutional neural networks.

I designed a simple model of a 1D Convolutional NN, based the ‘Best Network’ CNN developed by Nathan Janos and Jeff Roach in the previously linked video, 6 filters wide, 3 deep and a window of 7350 samples (since they were using a window of a 24 hour period with 8 weeks of data – I scaled it to be the same percentage of the 4*44100 samples I was working with) and trained it on my dataset for about 100 epochs.

After About 50 epochs the validation accuracy began to plateau at around 90% and the classification accuracy was at about 85%.

I think I will be able to increase the accuracy of this NN by feeding a larger data set as training data and tweaking the model. After testing this hypothesis my plan is to further alter the model so that it can facilitate more than 2 outputs, so that I will be able to identify even more types of rooms.

Furthermore, I’ve noticed that the majority of projects involving sound and ANNs convert the raw audio files into Mel frequency cepstrums before analysing them. I plan to see if this has any effect on the accuracy of the model compared to just training it on the raw Wav files.

Due to a combination of things, I’ve recently had to put the project about a half ways on hold. As such I haven’t had a chance to update the blog recently, so this post will cover my progress over the last few weeks from 24/7/19 to the end of this week 9/8/19.

Permissions Granted / Setting Up

The most pressing matter of gaining admin rights for my work machine was solved fairly early on, after a few calls to the help desk and I was able to download and install the ECL IDE, HPCC Systems and the other software I required for this project. This Included the GNN bundle and the ML_Core Bundle for ECL IDE which provide the necesarry tools for Generalised neural nets and Machine learning tools in ECL.

My next goal was to find a cluster to work on, and after an email or two I was sent the IP for the ‘oxford cluster’ where I would be able to Spray my data and begin messing with it in the ECL IDE. But first I had to figure out the best wav to spray my wav files.

ECL Watch / BLOBSPRAY

ECL works by accessing data from a cluster, however, firstly you have to actually put that data on the cluster. The most common way this is done is by uploading your data to the landing zone of an ECL Watch page and from there using the built in Spray function to move it to the cluser. When spraying data from the ECL Watch and there are many options to determine how its done, depending on whats best for the type of data you want to spray. As I am currently using WAV files, I went with a BLOB Spray which allows you to spray any kind of data as a BLOB (Binary Large OBject). Using this method I sprayed a traing data set ‘IR8’ of around 500 IR convolved Wavs to my cluster.

The Problem with BLOB Spraying is that BLOBs dont have a specific structure so you have to define the structre yourself in ECL to create a meaningful database from your data. I made a quick record structure for the IR8 database with 3 collunms for the file name, actual wav data and the virtual file position like so:

Layout_IR8 := RECORD
STRING filename;
DATA wavData;
UNSIGNED8 recordPos{virtual (fileposition)};
END;

IR8 := DATASET(~mbo::db::training::ir8′, LAYOUT_IR8, THOR, COMPESSED);

At this point I ran a few tests to make sure it was all working, amd began to get to grips with the documentation for the GNN bundle, which I plan to start using with my data from next week.

Blog Post 3

Muiredach O’Riain

week: 3
Starting: 15th July 2019 – 15/7/19

GNN –

An exciting development for this project came early in the week when I was put in touch with Roger Dev, the leader of HPCC Systems Machine Learning Library. Roger and his team are currently developping a Generalized Neural Network (GNN) Bundle for use with HPCC systems, which will eventually be a distributed ECL interface to Tensorflow

Over the coming weeks I plan to utilise an Alpha version of this tool set in order to help construct and develop Neural Nets with my data using ECL rather than with embedded python code as was originally my plan. This could be a huge boost for this project in terms of productivity as it would allow me more flexibility when coding with ECL rather than swapping back and forth between ECL and Python. Using the Alpha also offers a unique opportunity to be one of the first people outside of the Dev team to test the GNN bundle and help contribute to its development.

Permissions & Setbacks-

This week has been a bit slower in terms of actaul development compared to the previous two. Due to an issue with admin rights on my work machine I have been unable to download the necessary software to begin working with ECL and HPCC systems.

Despite this setback I am still ahead of schedule having previously surpassed my predicted goals from my proposal. Furthermore the downtime I have had whilst working to resolve this issue has allowed me to brush up on the ECL language and surrounding documentation, which I feel will help greatly improve my programming in the coming weeks.

This week constitued a break from programming and allowed me some time to reflect on the direction of the project and my approaches to the problems it presents.

Next week my goals are :

Get ECL-IDE & HPCC Systems Running on work machine

Figure out the best data format to use and spray my data to a cluster

Get to grips with ECL and begin some tests with ECL on my data

Begin to look at the GNN Alpha starting with the provided Documentation

week: 2

Starting: 8th July 2019 – 8/7/19

Goal For this week was to finish extracting the .wav files from the .mat files and batch convolving code.

Distorted files

Emailed the people at Aachen University for the .wav file version of the IRs, not in the .mat files, however they were 24bit files which wavefile.read() doesn’t work with. (ref:https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.wavfile.read.html)

I realised that the audio distortion problem from last week was only happening when the data was being exported as a .wav. This was fixed by changing the export function I was using to the scipy.io wavfile.write() function.

As such I managed to extract the IRs from the MAT Files as 2channel, 64bit, 48000hz .wav files. This also meant I was able to export the convolved files.

Batch Convolving and Exporting

Wrote a program that could batch convolve a folder of wav files with an IR.

Realised some files didn’t work with the convolution because they were stereo and the IRs were Mono, to solve I wrote a script to check the number of channels on each file and Convert them to mono if they were stereo.

Also Realised that when convolving some files could get too loud or quiet so I wrote a function to normalise all the output wav files between -0.8 & 0.8 so they don’t clip and distort.

Finished batch convolver program and gave it a naming function that would combine the IR and Name of the Dry Wav file.

Ran it over night and generated about 10GB of convolved and labeled files to begin training the first round of Neural Nets.

Plan for next week is to spray data to a cluster and begin processing the data with ECL

The goal of this project is to use HPCC Systems technology to help build a reliable classification model that is able to accurately classify an input sound file to a description of its location. The project intends to not only demonstrate a proof of concept for this technology but also to lay the groundwork for what could be the next step in forensic audio analysis and a new means of gathering information through sound.

Impulse Responses

In digital signal processing there are things called Impulse Responses (IRs) that are often used to capture and recreate the audio characteristics of a certain space or piece of equipment. Simply put the audio characteristics of any room can be described as an equation or a system that takes an input sound and then transforms it. An IR is essentially the result of putting a very short burst of sound, or an impulse, into this system and recording the response. We can then take this IR and use it to transform other sounds, making it seem as if they were recorded in the same room.

Using this principle it is possible to create a huge amount of audio data from a few impulse responses and some dry samples which we can then use to train a neural net as a classifier.