IoD Day 1 – Big Data Machine Learning: Applications, Tools and Best Practices

About Enterprise Machine Learning

It is the Magic creating the insights???????!!!!!!!

A complex field including a wide range of techniques, statistics, probability, speech processing, CS theory, signal processing and numerical analysis.

Very complex but this needs to be hidden from the user.

often will need to increase the feature space and also explore the hypothesis space of possible and feasible options.

One example is in the car business to be able to identify the 2% of cars that are lemons. This is very difficult to teach a machine as it would always predict all cars are ok and be wrong only 2% of the time using 100 columns, the solution was to grow the feature space to 10000 attributes. This improved the accuracy by 25 times

Algorithms must be “correct”, otherwise the algorithms will give different answers on different days with the same set of data. They must be written once and optimised for “everywhere”

Many different issues with scalability

Sampling real world sparse data is a significant problem as it is probable that the samples will not represent the “world” and led to incorrect insights.

The objective is to turn Data Scientists into Data Artists, who do not need to worry about the number crunching or technology.

 

This entry was posted in Big Data, Data Analytics, Machine Learning, Richard Self. Bookmark the permalink.