Weeks 4/5/6

Due to a combination of things, I’ve recently had to put the project about a half ways on hold. As such I haven’t had a chance to update the blog recently, so this post will cover my progress over the last few weeks from 24/7/19 to the end of this week 9/8/19.

Permissions Granted / Setting Up

The most pressing matter of gaining admin rights for my work machine was solved fairly early on, after a few calls to the help desk and I was able to download and install the ECL IDE, HPCC Systems and the other software I required for this project. This Included the GNN bundle and the ML_Core Bundle for ECL IDE which provide the necesarry tools for Generalised neural nets and Machine learning tools in ECL.

My next goal was to find a cluster to work on, and after an email or two I was sent the IP for the ‘oxford cluster’ where I would be able to Spray my data and begin messing with it in the ECL IDE. But first I had to figure out the best wav to spray my wav files.

ECL Watch / BLOBSPRAY

ECL works by accessing data from a cluster, however, firstly you have to actually put that data on the cluster. The most common way this is done is by uploading your data to the landing zone of an ECL Watch page and from there using the built in Spray function to move it to the cluser. When spraying data from the ECL Watch and there are many options to determine how its done, depending on whats best for the type of data you want to spray. As I am currently using WAV files, I went with a BLOB Spray which allows you to spray any kind of data as a BLOB (Binary Large OBject). Using this method I sprayed a traing data set ‘IR8’ of around 500 IR convolved Wavs to my cluster.

The Problem with BLOB Spraying is that BLOBs dont have a specific structure so you have to define the structre yourself in ECL to create a meaningful database from your data. I made a quick record structure for the IR8 database with 3 collunms for the file name, actual wav data and the virtual file position like so:

Layout_IR8 := RECORD
STRING filename;
DATA wavData;
UNSIGNED8 recordPos{virtual (fileposition)};
END;

IR8 := DATASET(~mbo::db::training::ir8′, LAYOUT_IR8, THOR, COMPESSED);

At this point I ran a few tests to make sure it was all working, amd began to get to grips with the documentation for the GNN bundle, which I plan to start using with my data from next week.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create your website at WordPress.com
Get started
%d bloggers like this: