Accuracy
A comparison of Nabla Copilot and Medx1 Copilot, focusing on transcription accuracy and automated medical processes. This analysis uses Word Error Rate (WER) as the key metric.
What is WER?
WER (Word Error Rate) evaluates transcription accuracy and is calculated as:
WER = (Substitutions + Deletions + Insertions) / Total Reference Words
Performance Comparison
Nabla Copilot
- Total Reference Words (N): ~135
- Estimated Errors:
- Substitutions: ~10–15 words
- Deletions: ~10–15 words
- Insertions: ~10–15 words
WER Calculation:
WER ≈ (10 + 10 + 10) / 135 ≈ 35 / 135 ≈ 26%
Medx1 Copilot
- Total Reference Words (N): 134
- Errors:
- Substitutions: 2 words
- Deletions: 1 word
- Insertions: 3 words
WER Calculation:
WER = (2 + 1 + 3) / 134 = 6 / 134 ≈ 4.48%
Comparison Table
| Metric | Nabla Copilot | Medx1 Copilot |
|---|---|---|
| Total Words | 135 | 134 |
| Substitutions | ~10–15 | 2 |
| Deletions | ~10–15 | 1 |
| Insertions | ~10–15 | 3 |
| Word Error Rate | 26% | 4.48% |
Embedded Video Comparisons
Nabla Copilot Performance:
Medx1 Copilot Performance:
Transcript in video
| Correct Transcript | Medx1 Transcript | Nabla Transcript |
|---|---|---|
| Doctor: Hey Salim, how are you? | Doctor: Hey Salim, how are you? | Hey. |
| Patient: مرحبا دكتور كيف حالك؟ I’m good. | Patient: مرحبا دكتور كيف حالك؟ I’m good. | Hi, Celine. How are you? |
| Doctor: Okay, you’re good? So what’s the problem? | Doctor: Okay, you’re good? So what’s the problem? | OK. You’re good. So what’s the problem? |
| Patient: دكتور، my ظهري، my it hurting so much. كنت بسوي deadlift. I deadlifted 300 pounds. | Patient: دكتور، my ظهري، my it hurting so much. كنت بسوي deadlift. I deadlifted 300 pounds. | Patient: Doctor, my lottery. My it’s hurting so much. OK, deadlift. I learned that lifted 300 pounds. |
| Doctor: 300 pounds? Okay, wow. And were you wearing any safety gears when you were deadlifting? | Doctor: 300 pounds? Okay, wow. And were you wearing any safety gears when you were deadlifting? | Doctor: 300 lbs. OK, well. And were you wearing any safety gears when you were adopting? |
| Patient: No, I didn’t wear anything. في البيت ف ما كان معي | Patient: No, I didn’t wear anything. في البيت ف ما كان معي | Patient: No, I didn’t wear anything. |
| Doctor: Wow. Uh and like how is the pain? Like does the pain radiate down your leg or is it localized to your back? | Doctor: Wow. Uh and like how is the pain? Like does the pain radiate down your leg or is it localized to your back? | Doctor: And like, how’s the pain? Like, does the pain radiate down your leg or is it localized to your back? |
| Patient: والله hurt so much و بينزل ل رجلي بعد | Patient: والله hurt so much و بينزل ل رجلي بعد | Patient: Well, I heard so much beans are largely bad. |
| Doctor: Okay, I see. Um yeah so you probably slipped a disc. So what we are going to do is we will send you to MRI and see how bad it is. Um if it’s too much, we can get you a steroidal injection. | Doctor: Okay, I see. Um yeah so you probably slipped a disc. So what we are going to do is we will send you to MRI and see how bad it is. Um if it’s too much, we can get you a steroidal injection. | Doctor: Yeah. So you probably, you probably slipped a disk. So what we are going to do is we will send you to MRI and we will see how bad it is. If it’s too much, we can get you a steroidal injection. |
| Patient: Uh no no no no I want to do the MRI inshallah and uh min shuf uh shomolo. | Patient: Uh no no no no I want to do the MRI inshallah and uh min shuf uh shomolo. | Patient: No, no, I want to do the MRI inshallah and. |
| Doctor: Alright. Okay then uh just stay put, we’ll get someone to take you to the MRI. Uh alright and then let’s see what we can do. If it’s too bad, we might have to go for surgery. | Doctor: Alright. Okay then uh just stay put, we’ll get someone to take you to the MRI. Uh alright and then let’s see what we can do. If it’s too bad, we might have to go for surgery. | Doctor: OK, then just stay put. We will get someone to take you to the MRI. Alright. And then let’s, let’s see what we can do. If it’s too bad, we might have to go for surgery. |
| Patient: Okay shukran doctor. Have a good day. | Patient: Okay shukran doctor. Have a good day. | Patient: Okay. Shukran, Doctor. Have a good day. |
| Doctor: Okay you too. | Doctor: Okay you too. | Doctor: Okay, you too. |
Key Insights
- Medx1 Copilot achieves a dramatically lower WER (4.48%) compared to Nabla Copilot (26%), demonstrating superior transcription accuracy.
- Reduced errors translate to more reliable automated processes and streamlined workflows for healthcare professionals.