Merlyn Mind emerges from stealth with $29M and a hardware and software solution to help teachers with tech

We’ve chronicled, in great detail, the many layers of technology, services and solutions, that have been wrapped around the world of education in recent years — and especially in the last year, which became a high watermark for digital learning tools because of Covid-19. Today, a startup called Merlyn Mind is coming out of stealth with a proposition that it believes helps tie a lot of this together in the K-12 classroom — a “digital assistant” that comes in the form of a piece of custom hardware and software to “read” natural voice and remote control commands from a teacher to control multimedia apps on a screen of choice. Along with this, Merlyn Mind is announcing $29 million in initial funding to build out its vision.

The funding is being led by specialist edtech investor Learn Capital, with other unnamed investors participating. It comes after Merlyn Mind spent about three years quietly building its first release and more recently piloting the service in 50+ classrooms in more than 20 schools.

Co-founded by longtime IBM scientists Satya Nitta (the CEO), Ravi Kokku, and Sharad Sundararajan — all of whom spent several years leading education efforts in IBM’s Watson AI research division — Merlyn Mind is coming to the market with a patented, vertically integrated solution to solve what Nitta told me in an interview he believes and has seen first-hand to be a fundamental pain point in the world of edtech.

In effect, education and technology may have now been merged into a single term as far as the tech world is concerned, but in terms of practical, on-the-ground application, many teachers are not making the most of the tools they have in the classroom. The majority are, he believes, facing “cognitive overload” (which is not to mention the kids, who themselves probably are facing the same: a problem for it to tackle down the road, I hope), and they need help.

To be fair, this problem existed before the pandemic, with research from McKinsey & Co. published in 2020 (and gathered earlier) finding that teachers were already spending more than half of their time on administrative tasks, not teaching or thinking about how and what to teach or what help specific students might need. Other research from Learn Platform found that teachers potentially have as many as 900 different applications that they can use in a classroom (in practice, Nitta told me a teacher will typically use between 20 and 30 applications, sites and tech services in a day, although even that is a huge amount).

Post-Covid-19, there are other kinds of new complications to grapple with on top of all that. Not only are many educators now playing catch-up because of the months spent learning at home (it’s been widely documented that in many cases, students have fallen behind), but overall, education is coming away from our year+ of remote learning with a much stronger mandate to use more tech from now on, not less.

The help that Merlyn Mind is proposing comes in the form of what the startup describes as an “AI hub.” This includes a personal assistant called Symphony Classroom, a kind of Alexa-style voice interface tailored to the educational environment and built on a fork of Android; a smart speaker that looks a bit like a soundbar; and a consumer-style remote that can be used also for navigation and commands.

These then work with whatever screen the teacher opts to use, whether it is a TV, or an interactive whiteboard, or something else; along with any other connected devices that are used in the classroom, to open and navigate through different apps, including various Google apps, NearPod, Newsela, and so on. (That could potentially also include kids’ individual screens if they are being used.)

The idea is that if a teacher is in the middle of a lesson on a specific topic and a question comes up that can best be answered by illustrating a concept through another app, a teacher can trigger the system to navigate to a new screen to find that information and instantly show it to the students. The system can also be used to find a teacher’s own materials on file. The demo I saw worked well enough, although I would love to see how an ordinary teacher — the kind they’re hoping will use this — would fare.

Everyone knows the expression “hardware is hard,” so it’s interesting to see Merlyn addressing its problem with a hardware-forward approach.

Nitta was very ready with his defense for this one:

“I’ll tell you why we built our own hardware,” he told me. “There’s a bunch of AI processing that’s happening on the device, for various reasons, including latency and security. So it’s kind of an edge AI appliance. And the second thing is the microphones. They are designed for the classroom environment, and we wanted to have complete control over the tooling of these microphones for the processing, for the environment, and that is very hard to do. If you are taking a third-party microphone array off the shelf, it’s impossible, actually, you simply cannot.”

The startup’s early team is rounded out with alums from the likes of HP Education, Amazon, Google, Facebook, Broadcom and Roku to help build all of this, knowing the challenges they were tackling, but also the payoff once it would be finished if it all works.

“We have a very, very talented team, and we basically said, right, this is going to be a lot of hard work that will take us three and a half years. We have to build our own piece of hardware… and we ended up building the entire voice stack from from scratch ourselves, too,” Nitta continued. “It means we have end to end control of everything from the hardware all the way to the language models.”

He did point out though that over time, there will be some elements that will be usable without all the hardware, in particular when a teacher may suddenly have to teach outside the classroom again in a remote learning environment.

It’s a very ambitious concept, but where would education and learning be if not for taking leaps once in a while? That’s where investors stand on the startup, too.

The proof will be in how it gets adopted when it finally launches commercially later this year, with pricing to be announced later.