What does a computational linguist do at work?

By Julia Berens · · 5 min read

Two people in blue and black work with a laptop and papers at a honey-colored wooden table beside a sunlit window.

What does a computational linguist do when a speech recognizer drops the word four from “set a timer for four minutes”? Before changing any code, they might put on headphones and check what the speaker actually said. Then they can compare that clip with others, look for a pattern, and test a change on recordings they have not used to develop the fix. This example is hypothetical, but it shows how the job can combine close listening, programming, and statistics.

What does a computational linguist do with a bad transcript?

Imagine a team reviewing a speech recognizer that turns spoken timer requests into text. The linguist opens a recording next to two transcripts: the system’s output and a human-written reference used to score it. The reference says “set a timer for four minutes.” The system says “set a timer for minutes.” The linguist listens to the clip to confirm that the speaker did say four. A reference can contain a mistake, too.

Next comes a small but consequential decision: how should the team handle “4” versus “four”? If the test is about recognizing the spoken words, the team might agree to treat those spellings as equivalent. If it is about the exact text shown on screen, formatting might need a separate test. Either way, “set a timer for minutes” is missing information, not just using a different spelling. In a review table, the linguist marks it as a deletion and records where in the clip it occurred.

Look for a pattern beyond one recording

One missing number is a clue, not a diagnosis. The linguist can use a script to find more reference transcripts with a number before minutes or seconds. Then they can listen to a sample of successful recognitions as well as failures. Does the system drop other numbers? Does it recognize four in “four minutes left,” where the preceding word is different? Do the failures show up mostly in recordings with background noise?

The sequence for four gives the linguist one hypothesis to check. In many varieties of English, for and four can sound alike when both are stressed. In everyday speech, though, for is often unstressed and may sound closer to “fer.” The spelling alone cannot tell the linguist how a particular speaker said the phrase. Listening can reveal whether the two words sounded very similar, whether one was especially short, or whether noise covered part of the phrase.

Counts help put those observations in perspective. If most of the recordings are noisy, the linguist might find more errors in noisy clips even when the error rate is no higher there. They would compare the share of missed numbers among noisy clips with the share among quiet clips, counting only clips where a number was actually spoken. A difference can guide the next investigation without proving what caused it.

Try a change and test it on fresh clips

Suppose closer review suggests that the system often drops numbers in timer requests, even in clear audio. Some recognizers can give extra weight to expected words or phrases when choosing a transcript. If this system supports that feature, the linguist might work with an engineer to try giving more weight to duration phrases when the interface already expects a timer request. A rule that simply inserts four after for would not fix the underlying problem. It could turn “set a timer for five minutes” into a different request.

To assess the change, the team runs the original and modified systems on the same set of clips, kept separate from the examples used to investigate the errors and tune the settings. One standard measure is word error rate: the number of substitutions, deletions, and insertions needed to match the reference, divided by the number of words in the reference. The NIST speech recognition scoring toolkit is one resource for comparing system output with reference transcripts.

The team also measures whether each requested duration is recognized correctly. Changing four to five is only one substitution, but it changes the timer setting. The test should include other numbers and requests with no spoken duration, such as “cancel the timer,” to check whether the change introduces numbers that do not belong. If the system recovers more missing numbers but also guesses extra ones, the linguist reports both results. They can share example clips and counts so the team can decide whether the trade-off is acceptable or whether to revise the change.

Build the skills with a small transcript project

You do not need to train a speech recognizer to practice this work. Write a set of short timer requests and record yourself saying them. Save the text from a speech-to-text tool, then listen back and write a reference for what you actually said. If you read “five seconds” when your script said “five minutes,” the reference should say seconds. Use recordings you have permission to analyze, and check where a tool sends or stores audio before uploading anyone’s voice.

  • Coding: Put the reference and output in a CSV file, with one row per clip. Write a short Python script to read the rows, convert text to lowercase, and print clips where the transcripts differ. Decide explicitly how to handle digits and punctuation.
  • Statistics: Count how many requests contain a duration and how many of those durations the tool got right. Report the count and the total alongside the percentage. Getting 3 out of 4 right tells you less than getting 30 out of 40 right, even though both give 75 percent.
  • Language analysis: Listen to each mismatch before assigning a label. Separate a missing number from a wrong number or a wrong unit. Compare errors with similar requests the tool got right, and write down alternative explanations before settling on a pattern.

If you can change a setting, save some clips for a final comparison rather than checking that setting only on errors you have already seen. For practice explaining a pattern from evidence without writing code, try the ILC training and practice problems. To see how language analysis fits into college coursework, read what linguistics majors study.

Frequently asked questions

Do I need to know machine learning to start?

No. Start by comparing transcripts, labeling errors, and writing a script that counts them. Machine learning becomes useful when you want to understand or change how the recognizer makes predictions.

Can word error rate improve while timer requests get worse?

Yes. A system could fix several other words but start confusing “four” and “five” more often. That is why a team can measure duration accuracy separately.