With advancements in Speech-to-text AI and on-device AI, the call center industry is approaching a transformative change. We should start rethinking the traditional approach of cloud-based transcriptions, bringing the process directly onto the agents’ devices.

Let’s dive into why this makes sense for modern call centers and BPOs.

1. Cost Benefits: The most immediate benefit of on-device transcription is cost savings. Traditional cloud transcription services are very expensive due to the costs associated with audio data transmission and processing on remote servers. On-device transcription, however, leverages the processing power of the agent’s device, leading to a significant reduction in these costs.
2. Security Enhancement: Security is a top concern in the call center world, especially when dealing with sensitive customer data. With on-device transcription, the audio is not sent to 3rd party services, drastically reducing the risk of data breaches. On-device processing aligns perfectly with stringent data protection regulations, offering peace of mind to both call centers and their customers.
3. On-Device PII Redaction: This method also allows for direct PII (Personally Identifiable Information) redaction on the agent’s device. This enables call centers to use the data appropriately while adhering to customer privacy and industry regulations.
4. Live Transcription: Another exciting aspect of on-device transcription is its application in providing low-latency live transcription for agents. This feature can be a game-changer, enabling agents to see a real-time transcript of the call. When integrated with existing agent-assist systems, agents can instantly receive suggestions and accurate responses, enhancing both efficiency and customer satisfaction.
5. Support for all SoftPhones: Since the transcriptions and recordings are performed on the agent’s device, it can support any SoftPhone, Dialer, or CCaaS solution the call center chooses to work with, making it application-agnostic.
6. Unified Experience and Storage: Many call centers (such as BPOs) must support multiple SoftPhones for their customers. With on-device transcriptions, all transcription and recording data can be saved in a unified storage giving the call center further opportunity to streamline agent experience across different communication applications and as well as easy integration with other systems.

Challenges and Solutions

Historically on-device transcription wasn’t an option for several reasons:

Hardware Limitations: The CPU requirements for on-device transcription could exceed the capabilities of existing hardware in call centers, requiring costly upgrades.

However, this challenge is being rapidly addressed by two factors:

1. In the last years, Speech-to-text technologies have become significantly more efficient in terms of CPU and memory requirements which makes them capable of running locally.
2. The influx of AI-powered laptops and workstations from Intel, AMD and Qualcomm designed to handle such tasks efficiently.

Integration Challenges: Historically, integrating on-device transcription with various call center systems and software was complex, requiring significant technical effort and resources.

However, with the integration of cloud systems in call centers, this is no longer a challenge. The transcriptions, and recordings, generated on agent’s device, can be easily uploaded to AWS, Google, Azure or other cloud storage and then fed into Call Center AI solutions such as CallMiner, Observe AI, AWS Connect, NICE, etc.

Low Transcription Quality: Historically, on-device yielded lower-quality transcriptions since you could only deploy small models on the device and you might not always have access to the latest AI models and updates that cloud-based systems do, leading to less sophisticated transcription capabilities.

However, the progress in Speech-to-text models and the availability of better CPUs makes it possible to successfully deploy and run high-quality efficient models on-device.

System Architecture

At Krisp, we utilize advanced Speech-to-text models that operate directly on-device. These models are not only highly efficient but also produce high-quality transcriptions. Importantly, they are designed to redact PII and are compatible with over 100 call center communication applications.

The following diagram shows how Krisp would typically be deployed in call centers.

• Automatically install Krisp on all agents devices
• Krisp will transcribe and record all agent calls from any CX solution or SoftPhone (e.g. Genesys, TalkDesk, Avaya, etc)
• Krisp will redact PII on-device and upload the final transcription and recording to call center’s preferred storage (e.g. S3, Azura, FTP, etc).
• The transcription and recording will be fed to Call Center AI for further processing

System architecture for deploying on-device transcriptions in call centers

Conclusion

Soon on-device transcription in call centers will mark a practical shift in operational strategy, focusing on enhanced security, cost-effectiveness, and improved efficiency in call centers.

Try next-level audio and voice technologies

Krisp licenses its SDKs to embed directly into applications and devices. Learn more about Krisp’s SDKs and begin your evaluation today.

The post The Power of On-Device Transcription in Call Centers appeared first on Krisp.

The 20 Rising Stars represent young, high-growth and category-leading cloud companies who have raised up to $25 million to date and are poised to join the Cloud 100 ranks.

“The Rising Stars of today are the Cloud 100 companies of tomorrow”, said Byron Deeter, a top cloud investor, and partner at Bessemer Venture Partners. “Many of our past Rising Stars have gone on to become members of the exclusive Cloud 100, and we’re certain many of this year’s honorees will follow suit. As an early partner to many exceptional founders over the years, we are thrilled to celebrate the bright futures of our 2020 Rising Stars.”

“Tech companies have been leaders in innovating the way we live and work for decades, and the world needs that kind of innovative, solutions-oriented mindset now more than ever. The 20 Rising Stars list includes innovative cloud leaders that are creating next-generation enterprise technology,” said Matt Garratt, Managing Partner, Salesforce Ventures. “We are optimistic about these entrepreneurs in making future Cloud 100 lists, and look forward to how they will change industries around the world.” 

“For five years now, we have ranked the best and brightest emerging companies in the cloud sector,” said Alex Konrad, Forbes editor of The Cloud 100. “With so many businesses growing fast in the cloud, from data infrastructure to marketing, it’s harder than ever to make the Cloud 100 list – but with more elite company if you do. Congratulations to each of the 2020 Cloud 100 honorees and the 20 Rising Stars honorees poised to join their ranks!” 

We are honored to be recognized by Forbes, Bessemer Venture Partners, and Salesforce Ventures as one of the Rising Stars in the Cloud 100! 

The post Krisp is Named One of 20 Rising Stars as Part of Forbes’ Cloud 100 List appeared first on Krisp.

This was a challenging and fun journey, including unexpected insight from a dinner, a miracle and some Rick brilliance. We had to go through some challenging steps to make it happen and thought it’s a story worth sharing.

First, what’s Krisp?

Krisp app uses a specially designed Deep Neural Network (DNN) to separate human voice from background noise in real-time. The DNN is collectively trained on 10K+ hours of human voice and background noises and achieves an unprecedented level of quality. The app is used by tens of thousands of users every day so the DNN was in the wild for some time now. 

Our main desktop app (Krisp for Windows and Mac) acts as a virtual microphone device. 

We always wanted to have Krisp running inside Chrome in the form of an Extension. And always thought it’s not possible. 

The dinner that changed everything

Until one beautiful evening we had a dinner with our friends @Discord. Stan, Discord’s CTO, hinted that Chrome lately made some additions (enabled audio filter plugins) and it might be possible to develop an external audio filter now, all in Javascript. 

I was skeptical.

I was so skeptical that didn’t want to engage our core team yet. I went straight to Toptal to find someone who could quickly hack a proof of concept (POC) for us. We Toptal.

A month after that dinner we had a POC of an extension which was able to add an audio filter to a webpage that transforms the microphone stream in real-time by adding a static noise. The filter was working fine for Google Meet, Webex and other apps. It was perfect.

This was exciting. We set out to build Krisp for Chrome and even came up with a cool name for it – KrispX (X for extension).

There was one big challenge though. How do we port our C++ codebase into browser? Well, obviously WebAssembly (WASM) but we didn’t have anyone with such experience. 

WASM and XNNPACK

So Artak, one of our architects, started looking into WASM. When we have no idea about something – we always look at Artak ?

His first version of the port came after 3 days. Yep, he is super fast ?. The version was extremely slow though. It was a hacked-together version which implemented matrix multiplication in the most naive way, in Javascript. So the algorithm was running 10x slower than our main C++ model on the same laptop.

After some research we found about Tensorflow.js and all the awesome work that Google is doing to bring DNNs to the browser. The next finding was XNNPACK. It’s a highly optimized library of floating-point neural network inference operators for WASM (and ARM, x86). 

XNNPACK was great. After some adjustment to the build system we were able to build our code with XNNPACK for WASM. It took us overall 10 days. The result? Almost the same speed as the C++ version. 

To be honest we were mindblown at this point. WASM and XNNPACK were amazing. Good engineering, Mozilla, Facebook and Google, as always ?

But apparently this was just the beginning of our journey.

3ms frames

The way Chrome audio filters are designed is extremely constrained.

Chrome feeds the filter plugin with 3ms frames, while it reads the data from the mic, and the plugin has <3ms time to complete the processing. However our DNN operates on 30ms frames. So we had to bufferize 10 frames (30ms) and then process the 30ms frame with our DNN within 3ms. 

If you don’t process within 3ms Chrome will drop the audio packet and give you the next one. In practice this would mean dropped packets and broken voice.

To be clear, this was simply impossible for us. Our DNN was too big for this.

We needed a miracle.

The miracle

The miracle came from our incredible research team. We had an ongoing project where the goal was to reduce the size/speed of our DNN so that it can be embedded in more constrained environments. Our architect, Stepan, together with the team, had made decent progress there and this progress came amazingly timely.

They have built a new DNN which was 10x smaller than the main model and it had a very comparable quality.

We quickly ported it to WASM and XNNPACK and viola, we had a real-time solution inside Chrome. 

Inspired by this progress, our product team, led by Davit, integrated WASM code into Chrome extension and the QA team has started testing it with real apps. It worked quite well but occasionally there were voice breakups.

Apparently the optimizations were not enough. If the CPU usage was going up, which is common when you have a video call running inside Chrome, the time required by DNN to process a frame would increase so we would pass the threshold of 3ms and this would cause Chrome to drop packets. 

The Rick moment

So we all looked at Artak again ?. And he of course had an amazing idea. We were not surprised. He always has plenty of these.

What if we distribute the DNN processing into multiple parts and process every part within 3ms? We would introduce a bit of latency but that should be negligible for our use case.

He had the algorithm ready in 10 days. Why so long? Well he was on vacation for 7 days.

The new algorithm was perfect. No voice breakups, no noticeable latency even when there is video and the CPU usage hits the fan.

Krisp for Chrome is now live. From idea to going live – 3 months.

You can use it with any app running inside Chrome browser, from Google Meet to Whereby. And it’s running our 30X faster algorithm.

The post We had to squeeze our Neural Network by 30x to run inside Chrome appeared first on Krisp.

NOTE: This blog post was originally published on March 17, 2020. The COVID-19 plan had ended on October 13, 2020 and is no longer available.

What’s happening in the world these days is scary. We understand it’s going to get worse until it gets better. Individuals and companies need to proactively react to help each other through this difficult time.

Self-isolation is the only proven way to contain the spread of the virus and that means people need to work from home (WFH). We need the right tools to stay productive and have less stress during this difficult time. 

In the last two years, our company has made AI-powered technological breakthroughs (noise cancellation, room echo removal and more coming soon) which allow people to work from anywhere they want and these technologies come really handy these days. 

Krisp weekly downloads are increasing by +600% week over week. This tells us one thing: Krisp helps people and removes some of the stress while WFH.

It is our responsibility to step up and do everything to help in such situations, with the extent that is possible. 

While Krisp is growing at rapid pace, we are not yet a large company and thus have limited resources and reach. However, we think that every action matters and even though we are small, we can still have our impact on the world.

With this in mind, we have made some important decisions.

https://youtu.be/nmNGXPxVhmI

What we’re changing today

Effective today and for the next 6 months:

• Krisp introduces a free tier. All users get 120 min/week of free noise cancellation.
• All students, teachers, hospital and government workers, worldwide, get unlimited Krisp for free.
• Krisp Pro and Teams cut the price 30%. The new price is $40/year and $5/month.
  

For more information about these changes and how to apply them, please visit our help center.

Make a dent

This decision has been inspired by Zoom, Google, Webex, Loom and other awesome companies that are taking actions these days.

This is an exemplary quote from Loom’s recent blog post:

“Our hope is that by stepping up, we’re making a dent in the global impact this pandemic is causing, not just by making our product more accessible, but by helping set the kind of example we hope more businesses will follow.”

In light of the current crisis, we’re proud to introduce the Krisp COVID-19 Plan & Pricing Updates.

Please reach out if we can be of any help.

Thank you.

The post COVID-19 Response: New free plan and dropped prices appeared first on Krisp.

Note: The Krisp Phone app will be removed from the app store and will be no longer available for download as of December 26th. However, if you’ve already downloaded it to your mobile device – you can continue to use it until then. As of December 26th, the app will be officially shut down. All existing users will be signed off and the app will no longer function. Please read more about our decision here.

I would like to reveal something. 

The built-in noise cancellation on all our phones is pretty shitty. 

That’s not because Apple and Samsung of the world are not investing energy to make them better. It’s simply because they don’t have the technology to make it better.

All modern smartphones have multiple microphones deployed on the phone. I have talked about it previously on NVIDIA Developer Blog.

https://youtu.be/F-rWA3cf79o

The basic concept is that one microphone is located close to your mouth and the others are designed as far as possible from your mouth. With this setup, noise cancellation works ONLY when we hold our phones close to our mouth. 

However, people increasingly don’t hold the phone close to their mouths anymore when they are in calls. Instead, we wear headsets like AirPod, or we hold it in front of us (for video) or we drive and speak in the car. In all these cases phone noise cancellation STOPS working entirely. Period. 

At Krisp, we have built a revolutionary app to fix this problem. The app is called, not surprisingly, Krisp. 

Krisp gives you the freedom to make calls from anywhere you want using your phone. It uses AI-powered noise cancellation technology to completely eliminate the background noise while you are having your call.

With this app, you can hold the phone anyhow you want and wear any headset you want. 

Krisp doesn’t care. It adds up on whatever noise cancellation you already got on your phone and adds a superpower to it.

You can use Krisp to call any phone number. You can even join conference calls – like Zoom, Webex, Meet, MS Teams, Skype for Business, etc.

So next time you are outside in street, or in an airport or maybe at home but you have your kids screaming in the background (like my kids like to do) – just give Krisp mobile dialer a try and you are going to be amazed. 

And one more thing… Krisp is completely free. So downloading and using it is really a no brainer. 

Enjoy!

The post Why is Krisp a Revolutionary App for iPhone? appeared first on Krisp.