A Dance with Constraints

The year was 2015. Apple announced exciting new features of CloudKit at WWDC. Mostly notably, a very generous free tier. Private Database is free, for App developers anyway. Public Database's free limits grow with users. I am going to spare you from numbers, just say, it is basically free. By the time, Apple started to charge you, you probably don't even care, since you've got so many users.

Nice. No?

Free. Nothing can beat free, right?

Well, the answer is "kinda".

We had this idea. It was great, like most ideas when it was still an idea in your head. The idea is using CloudKit as backend, for free. So people can read code anywhere, and make annotation anywhere. Yes, yes, I know, you need to have data connection to really do it anywhere. Well, anywhere with good cellular coverage. You know what, anywhere with internet connection.

So, we designed it, architected it, and implemented it. Then we noticed a problem. CloudKit is slow, like really slow. It is no where close to DynamoDB. It is no where close to S3. It took many seconds to sync a very small text file into iCloud.

Free became an constraint. You can not pay to speed it up.

It was still useful. But it just became harder to build things on top. Speed shapes what you do, and how you double

Now let's talk about machine learning.

We had ideas, features, and even prototypes. But it was hard to package them into products. It is like you have this awesome recipe. But it must be served within 5 minutes.

It took years for the infrastructure to get there.

We had the idea to use computers to help people write. We released eWriter, Numathic Write in 2017. But AI Editor was what was intended all along. We just could not make it. There were other less capable algorithms before GPT-2, many of them. But the tooling was not there. Packaging a machine learning model into an App was just too complicated for a broader user base.

Torch were the first packages examed. Embedding a Lua interpreter into an App is fine. But Torch requires so many libraries. To package them into an App and make sure they work is quite challenging, not to mention converting algorithms into Lua.

We explored TensorFlow C++ APIs too. Still way too complicated.

Only at beginning of this year thing changed when Swift For TensorFlow was introduced to the public. Now, you have a language that can take Python code from research papers, and also can produce shared libraries, frameworks for macOS, iOS and Linux, probably Windows in the future.

There are constraints on the hardware. Most computers, PC or Mac, are not capable of running recent machine learning models. To show the benefit of simplicity in machine learning, bigger model is required. CPU is having more and more cores. GPU and accelerators are coming. There is this MLIR project that might lift constraints we are facing. A lot of exciting things are happening. It is really an exciting time to be in.

This article is written in AI Editor