The Deep Dojo Machine Learning Blog

A Different Kind of Software

2019-04-08T00:00:00+00:00

What makes machine learning different? What implications does it have for how software is developed?

We dive into these questions in Episode 2 of “Demystifying Machine Learning”. See an example in action, from training on a Mac to deployment on a mobile phone.

An Unlikely Robot Apocalypse

2019-01-18T00:00:00+00:00

Machine learning has a lot of hype these days. Some positive. Some negative. We don’t often see examples of where it fails. The mystery surrounding the technology can make it hard to understand.

This is the first video in the “Demystifying Machine Learning” series.

Apple Introduces Create ML

2018-06-27T00:00:00+00:00

Training a machine learning model on a Mac hasn’t been easy. There are open source frameworks that run on a Mac CPU. But trying to accelerate training with a GPU meant using Nvidia hardware most likely on a Linux box.

With Create ML, Apple isn’t aiming for parity with these data science packages. Create ML takes the tangle of algorithmic detail that comes with machine learning and encapsulates it away. The result is an astonishingly simple training environment that is hardware-accelerated for the Mac.

Drag your training data in then drag your trained model out using three lines of Swift.

import CreateMLUI
let builder = MLImageClassifierBuilder()
builder.showInLiveView()

This reminds me of the iPod. Late to the party when it was introduced to the market in 2001, that didn’t stop it from becoming a success. Apple has a knack for targeting technology that is locked behind a “chain of pain”. Remove that pain and the technology becomes accessible to a wider group of people. It’s a strategy that runs deep in the Apple play book.

David Sobotta back in 2009:

“Any time you have a device that is hard to use, but has a high potential for usage, you raise just huge flags for Steve Jobs. It’s like waving a red flag in front of a bull. All of a sudden he sees something and says, wait a minute, if we do this really right, we can own it.”

What do we loose when algorithmic detail disappears from a machine learning environment? For a lot of data scientists those details are where the opportunity for innovation is. In a community that lives on the bleeding edge, a few percentage points of accuracy is a big deal.

To be clear, Metal Performance Shaders also offer training capabilities. There is customization and gradients galore at that level of detail if that’s your cup of tea. Core ML is Apple’s deployment engine for machine learning and also offers customization.

For many developers though, bleeding-edge accuracy is not the goal. Just being able take my own data and easily go from zero to any kind of accuracy in the ninety percent range is an astonishing development. For many applications, developing a custom model is now reduced to an exercise of gathering and grooming training data. The machine learning technology itself is no longer a barrier.

Less training detail in Create ML also means more room for Apple to experiment under the hood. It’s a separation of concerns between applications and machine learning that will likely prove useful - particularly when you consider Apple’s ability to innovate and optimize its lineup of hardware.

Machine Learning Reproducibility Crisis

2018-05-06T00:00:00+00:00

Pete Warden discusses code, data and software process. When mixed with machine learning, it can be tricky to iterate on results.

“I’m no shrinking violet when it comes to version control. I’ve toughed my way through some terrible systems, and I can still monkey together a solution using rsync and chicken wire if I have to. Even with all that behind me, I can say with my hand on my heart, that machine learning is by far the worst environment I’ve ever found for collaborating and keeping track of changes.”

Creator of Swift Working on TensorFlow

2018-03-15T00:00:00+00:00

Chris Lattner joined Google back in August.

Machine Learning Glossary

2017-12-12T00:00:00+00:00

A list of over 150 machine learning terms, sorted alphabetically.

mini-batch

A small, randomly selected subset of the entire batch of examples run together in a single iteration of training or inference. The batch size of a mini-batch is usually between 10 and 1,000. It is much more efficient to calculate the loss on a mini-batch than on the full training data.

Climbing Out of Uncanny Valley

2017-11-28T00:00:00+00:00

People, it seems, have an emotional reaction to things that look almost but not quite human. There are theories as to why but regardless of the reasons, it’s been a challenging barrier to using photo-realistic computer-generated humans. Something the entertainment industry has been chipping away at for a while.

The makers of Mug Life have released an iOS app that can look at a 2D image of a face and then animate it in three dimensions. The results are eye-opening.

“This innovative technology featuring deep neural networks marries decades of video game expertise with the latest advances in computer vision.”

Technical advancements in this area are accelerating.

MLModel API

2017-10-23T00:00:00+00:00

Generated Code

When you take a small Core ML model like Squeeze Net and drop it into Xcode, you get a SqueezeNet implementation. Without comments, the class itself is a only a few dozen lines in length.

class SqueezeNet {
    var model: MLModel

    init(contentsOf url: URL) throws {
        self.model = try MLModel(contentsOf: url)
    }

    convenience init() {
        let bundle = Bundle(for: SqueezeNet.self)
        let assetPath = bundle.url(forResource: "SqueezeNet", withExtension:"mlmodelc")
        try! self.init(contentsOf: assetPath!)
    }

    func prediction(input: SqueezeNetInput) throws -> SqueezeNetOutput {
        let outFeatures = try model.prediction(from: input)
        let result = SqueezeNetOutput(classLabelProbs: outFeatures.featureValue(for: "classLabelProbs")!.dictionaryValue as! [String : Double], classLabel: outFeatures.featureValue(for: "classLabel")!.stringValue)
        return result
    }

    func prediction(image: CVPixelBuffer) throws -> SqueezeNetOutput {
        let input_ = SqueezeNetInput(image: image)
        return try self.prediction(input: input_)
    }
}

SqueezeNet is not a subclass of MLModel. It’s a wrapper with methods for loading and driving a model property. Xcode generates input and output classes for the model as well.

Feature Introspection

In the end, a class like SqueezeNet is a high-level, type-safe convenience. At a lower level, it’s possible to drive the model directly and generically. This is how a framework like Vision is able to inspect and use an arbitrary model at runtime.

MLModel sits at the heart of Core ML. It’s an abstraction that’s focused on input and output features. MLModelDescription indicates how these features are structured. An MLFeatureProvider delivers input into the model. A corresponding MLFeatureProvider contains output.

Image Processing and Classification

Using the Vision framework, a VNImageBasedRequest turns an image into observations.

VNCoreMLRequest is a type of image-based request that uses a Core ML model. Depending on the output description of the model, the request will automatically produce either classifications, pixel buffers or feature values.

As Matthijs Hollemans mentions in his post, it’s important to match image input to the preprocessing that a model expects. Much of this can be addressed when the model is converted to Core ML. From the modelDescription, the Vision framework is able to convert most images into the pixel format that a model wants.

Another kind of compatibility to be aware of is image orientation. This is particularly true for iOS where captured images typically align with the camera sensor instead of aligning to the orientation of the device. Request handlers in the Vision framework offer a way to specify orientation when applying Core ML requests to camera images.

Conclusion

“A successful book is not made of what is in it, but of what is left out of it.”

— Mark Twain

When using an MLModel, an app is free to focus on input and output. Model implementation is set aside as optimization detail to be navigated by the .mlmodel creator who knows the model and Apple engineers who know their lineup of hardware.

Core ML can generate code for a model that is convenient and easy to use. However if your solution involves plugging in a range potential models, any MLModel can be inspected at runtime to query how its input and output features are structured.

Core ML Store

2017-10-08T00:00:00+00:00

A list of free Core ML models with associated sample code and reference. Curated by Kedan Li.

“Download and use per license. Remember to acknowledge.”

The site currently has eight image filters and roughly twenty classifiers.

NVIDIA Drivers for macOS High Sierra

2017-09-28T00:00:00+00:00

CUDA is an NVIDIA hardware toolkit. Many deep learning frameworks use it to accelerate training. Apple last offered an NVIDIA GPU in its MacBook Pro in 2014. A GeForce GT 750M, which provided a CUDA compute capability of 3.0.

At WWDC, Apple announced official support for external GPU enclosures in macOS High Sierra. This week NVIDIA followed suit.

“After skipping the assorted High Sierra betas, NVIDIA has rolled out drivers for its line of PCI-E graphics cards.”

This gives any Mac with Thunderbolt 3 connectivity the ability to power up CUDA 9.0 with a compute capability of 6.1.