Microsoft introduced Windows Copilot Runtime, a software development platform that gives you all the tools you need in a single product

Microsoft Build 2024 is happening these days in Seattle, and the Redmond-based tech giant has been having a great week so far: the company announced a plethora of new devices (Copilot + PC), Copilot enhancements, including Team Copilot, and many other features coming to Windows, such as the brand-new Recall.

However, at Build 2024, software developers were met with Windows Copilot Runtime, a new platform, meant to encapsulate many tools necessary for building apps. In the official blog post, Microsoft seems very excited about the new platform. Developers should feel that way, too. Why? Because it’s quite versatile.

The new Windows Copilot Runtime is that system. Developers can take advantage of Windows Copilot Runtime in a variety of ways, from higher level APIs that can be accessed via simple settings toggle, all the way to bringing your own machine learning models.

Microsoft

For example, the platform will include many tools making it, as Microsoft says, the end-to-end Windows ecosystem. These include:

• Applications and Experiences: software developers can access apps made by other developers and curated by Microsoft.
• Windows Copilot Library which encompasses over 40 models that ship with Windows, so that developers can use them to build apps.
• AI frameworks such as DirectML, ONNX Runtime, PyTorch, WebNN, along with toolchains like Olive, AI Toolkit for Visual Studio Code. These will be useful to scale the AI-based apps.

Microsoft says that the new features coming to Windows 11 are also going to be quite useful to software developers and they’re also part of the new Windows Copilot Runtime. Windows Recall is one of them, and it’s an AI-based tool that will record everything you do on your PC. This is especially useful when running into error and not knowing why, as you can trace it back.

Other tools such as Cocreator, Restyle Image, Live Captions, or Windows Studio Effects will also be quite useful, as they offer you access to rendering art in seconds, following up with a tutorial in a foreign language, or even designing your apps to follow special Windows Studio Effects actions.

Windows Copilot Library is also quite the rich baggage for every kind of Copilot extension that you might need, and some of them will be released to the platform as soon as next month, Microsoft says:

Windows Copilot Library consists of ready-to-use AI APIs like Studio Effects, Live captions translations, OCR, Recall with User Activity, and Phi Silica, which will be available to developers in June. Vector Embeddings, Retrieval Augmented Generation (RAG), Text Summarization along with other APIs will be coming later to Windows Copilot Library. Developers will be able to access these APIs as part of the Windows App SDK release. 

Microsoft

While some can be integrated using code, others follow a no-code policy, allowing developers to easily integrate effects such as Creative filters, Portrait light, Eye contact teleprompter, Portrait blur, and Voice focus, into their apps.

The Windows Copilot Runtime is basically an entire software development department in one laptop with everything in one place. Not only you’ll be able to build your apps, but you’ll have Copilot extensions and apps, such as Cocreator to come up with design sketches.

If your app implies adding multiple languages, the Live Caption capability can help you translate or come up with a sketch of a translation for it. Plus, assuming you’ll want to build apps for Windows, the Studio Effects integration will help you build them natively for the OS.