MIS 752 · Lab 5 · Mikey Atwood
Speak the Caller's Language: Searching 911 Calls by Meaning
Callers in a crisis describe emergencies in their own words. This page tests whether a small, free AI model can match those words to the right kind of call when a keyword search can't.
100%
semantic search: right call ranked first when callers used their own words
0%
keyword search on those same queries
2 of 36
queries where keyword search returned nothing at all
$0
runs on a laptop CPU: a 22.7M-parameter model, no API, no data leaves the building
The problem
A frightened caller does not talk like a CAD system. They say "he says it feels like an elephant sitting on his chest"; the call-taker's record says chest pain, difficulty breathing. A keyword search over past calls, protocols or incident notes only finds the records that happen to share the caller's words.
Semantic search compares meaning instead of spelling. A small embedding model turns each sentence into 384 numbers so that sentences describing the same situation land close together, even with no words in common. This page tests whether that holds up on 911 calls.
Does it actually find the right call?
When the caller happened to use the record's own vocabulary, all three methods ranked the right call first. When the caller described it in their own words, keyword search and BM25 found nothing useful, and semantic search still put a correct call at the top.
Try it: four real searches
Queries typed into the notebook and the top three calls semantic search returned. The misses are included on purpose: it can only return what is in the corpus.
Caller says: "My vehicle just vanished sometime while everyone in the house was asleep last night."
- I just noticed my bike is gone from the garage, someone must have taken it while we slept. (law15, similarity 0.60)
- My car was stolen from the driveway overnight. (law14, similarity 0.60)
- My car just caught fire in the parking lot. (fire11, similarity 0.50)
Top two are the right kind of call (a theft overnight) with no words in common with the query.
Caller says: "my neighbor's house is on fire"
- There's a huge grass fire spreading toward the houses. (fire14, similarity 0.64)
- There's smoke pouring out of my neighbor's roof. (fire1, similarity 0.63)
- My car just caught fire in the parking lot. (fire11, similarity 0.53)
All three are fire calls; #2 is the closest match.
Caller says: "my husband is choking on a hotdog"
- My husband is clutching his chest and can barely breathe. (ems1, similarity 0.54)
- He's blocking the front door and I can't get my kids out. (law9, similarity 0.36)
- There's a man trying to break into my neighbor's house right now. (law1, similarity 0.34)
A miss: the corpus has no choking call, so it found the nearest breathing emergency, then drifted.
Caller says: "a car crashed into my brick mailbox"
- My car was stolen from the driveway overnight. (law14, similarity 0.49)
- Someone just smashed my car window and grabbed my bag. (law11, similarity 0.48)
- My car just caught fire in the parking lot. (fire11, similarity 0.42)
A miss: no traffic-collision calls exist in the corpus, so "car" pulled in car-related calls.
Does the model understand the calls?
Every sentence compared with every other. Labels show the sentence id and voice (cal = caller, ctk = call-taker). Hover any cell to read the two sentences.
The numbers
| Method | precision@k | recall@k | nDCG@k | MRR |
|---|---|---|---|---|
| keyword (word overlap) | 0.24 | 0.31 | 0.31 | 0.33 |
| semantic (embeddings) | 0.47 | 0.72 | 0.78 | 1.00 |
| BM25 | 0.22 | 0.28 | 0.29 | 0.33 |
Every query, every method
| query_type | query | method | retrieved | p_at_k | recall_at_k | rr |
|---|---|---|---|---|---|---|
| caller guessed our words | a man is trying to break into the house next door | keyword (word overlap) | law1,law2,law5 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | a man is trying to break into the house next door | semantic (embeddings) | law1,law2,law9 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | a man is trying to break into the house next door | BM25 | law1,law5,law9 | 0.333 | 0.5 | 1.0 |
| caller's own words | Something is scraping against the back entrance and it doesn't look like the man has noticed me at the window. | keyword (word overlap) | law1,law10,law11 | 0.000 | 0.0 | 0.0 |
| caller's own words | Something is scraping against the back entrance and it doesn't look like the man has noticed me at the window. | semantic (embeddings) | law4,law5,law12 | 0.333 | 0.5 | 1.0 |
| caller's own words | Something is scraping against the back entrance and it doesn't look like the man has noticed me at the window. | BM25 | law1,law10,ems5 | 0.000 | 0.0 | 0.0 |
| caller's own words | He looped toward the rear of the property and seems to be testing the glass panes now. | keyword (word overlap) | law2,fire13,fire14 | 0.000 | 0.0 | 0.0 |
| caller's own words | He looped toward the rear of the property and seems to be testing the glass panes now. | semantic (embeddings) | law5,law4,law17 | 0.333 | 0.5 | 1.0 |
| caller's own words | He looped toward the rear of the property and seems to be testing the glass panes now. | BM25 | fire13,law2,fire14 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | my ex won't leave and keeps yelling at me | keyword (word overlap) | law7,law6 | 1.000 | 1.0 | 1.0 |
| caller guessed our words | my ex won't leave and keeps yelling at me | semantic (embeddings) | law6,law7,law9 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | my ex won't leave and keeps yelling at me | BM25 | law7,law6 | 1.000 | 1.0 | 1.0 |
| caller's own words | The man is standing in the doorway and I can't get the children past him to safety. | keyword (word overlap) | law1,law20 | 0.000 | 0.0 | 0.0 |
| caller's own words | The man is standing in the doorway and I can't get the children past him to safety. | semantic (embeddings) | law9,law1,fire17 | 0.333 | 0.5 | 1.0 |
| caller's own words | The man is standing in the doorway and I can't get the children past him to safety. | BM25 | law1,law20 | 0.000 | 0.0 | 0.0 |
| caller's own words | The man just flung an object across the living room and it feels like he might come after me next. | keyword (word overlap) | law1,ems4 | 0.000 | 0.0 | 0.0 |
| caller's own words | The man just flung an object across the living room and it feels like he might come after me next. | semantic (embeddings) | law10,law1,law4 | 0.333 | 0.5 | 1.0 |
| caller's own words | The man just flung an object across the living room and it feels like he might come after me next. | BM25 | ems4,law1 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | someone smashed my car window and stole my bag | keyword (word overlap) | law11,law12,law14 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | someone smashed my car window and stole my bag | semantic (embeddings) | law11,law12,law14 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | someone smashed my car window and stole my bag | BM25 | law11,law19,fire11 | 0.333 | 0.5 | 1.0 |
| caller's own words | My vehicle just vanished sometime while everyone in the house was asleep last night. | keyword (word overlap) | law20,law1,law3 | 0.000 | 0.0 | 0.0 |
| caller's own words | My vehicle just vanished sometime while everyone in the house was asleep last night. | semantic (embeddings) | law15,law14,fire11 | 0.667 | 1.0 | 1.0 |
| caller's own words | My vehicle just vanished sometime while everyone in the house was asleep last night. | BM25 | law20,law1,fire12 | 0.000 | 0.0 | 0.0 |
| caller's own words | The guy sprinted off toward the north end of the block in a dark coat. | keyword (word overlap) | law3,law16,fire13 | 0.000 | 0.0 | 0.0 |
| caller's own words | The guy sprinted off toward the north end of the block in a dark coat. | semantic (embeddings) | law13,fire15,law5 | 0.333 | 0.5 | 1.0 |
| caller's own words | The guy sprinted off toward the north end of the block in a dark coat. | BM25 | fire20,fire13,law16 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | there's a guy checking door handles on cars in the parking lot | keyword (word overlap) | law16,law17,fire11 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | there's a guy checking door handles on cars in the parking lot | semantic (embeddings) | law16,law17,fire12 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | there's a guy checking door handles on cars in the parking lot | BM25 | law16,law17,fire11 | 0.667 | 1.0 | 1.0 |
| caller's own words | The man dropped low next to a light-colored car the moment he noticed me. | keyword (word overlap) | law1,law11,law14 | 0.000 | 0.0 | 0.0 |
| caller's own words | The man dropped low next to a light-colored car the moment he noticed me. | semantic (embeddings) | law16,law11,law10 | 0.333 | 0.5 | 1.0 |
| caller's own words | The man dropped low next to a light-colored car the moment he noticed me. | BM25 | law1,law15,fire11 | 0.000 | 0.0 | 0.0 |
| caller's own words | It feels like that one vehicle keeps circling back near my place again and again this evening. | keyword (word overlap) | law3,law12,ems4 | 0.000 | 0.0 | 0.0 |
| caller's own words | It feels like that one vehicle keeps circling back near my place again and again this evening. | semantic (embeddings) | law20,law15,fire11 | 0.333 | 0.5 | 1.0 |
| caller's own words | It feels like that one vehicle keeps circling back near my place again and again this evening. | BM25 | ems4,fire12,law3 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | my husband is holding his chest and struggling to breathe | keyword (word overlap) | ems1,ems2,ems4 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | my husband is holding his chest and struggling to breathe | semantic (embeddings) | ems1,ems2,ems9 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | my husband is holding his chest and struggling to breathe | BM25 | ems1,ems4,ems2 | 0.667 | 1.0 | 1.0 |
| caller's own words | He describes a crushing weight resting right on top of his ribcage. | keyword (word overlap) | (nothing) | 0.000 | 0.0 | 0.0 |
| caller's own words | He describes a crushing weight resting right on top of his ribcage. | semantic (embeddings) | ems4,ems1,ems16 | 0.333 | 0.5 | 1.0 |
| caller's own words | He describes a crushing weight resting right on top of his ribcage. | BM25 | (nothing) | 0.000 | 0.0 | 0.0 |
| caller's own words | The color is draining from his mouth area and he is soaked through with perspiration. | keyword (word overlap) | law2,law12,law14 | 0.000 | 0.0 | 0.0 |
| caller's own words | The color is draining from his mouth area and he is soaked through with perspiration. | semantic (embeddings) | ems5,ems10,ems9 | 0.333 | 0.5 | 1.0 |
| caller's own words | The color is draining from his mouth area and he is soaked through with perspiration. | BM25 | fire7,fire4,law2 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | my son had a seizure and he won't wake up | keyword (word overlap) | ems6,ems7 | 1.000 | 1.0 | 1.0 |
| caller guessed our words | my son had a seizure and he won't wake up | semantic (embeddings) | ems6,ems7,ems8 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | my son had a seizure and he won't wake up | BM25 | ems6,ems7 | 1.000 | 1.0 | 1.0 |
| caller's own words | Every limb was jerking uncontrollably and afterward he just seems completely dazed. | keyword (word overlap) | (nothing) | 0.000 | 0.0 | 0.0 |
| caller's own words | Every limb was jerking uncontrollably and afterward he just seems completely dazed. | semantic (embeddings) | ems9,ems20,ems10 | 0.333 | 0.5 | 1.0 |
| caller's own words | Every limb was jerking uncontrollably and afterward he just seems completely dazed. | BM25 | (nothing) | 0.000 | 0.0 | 0.0 |
| caller's own words | He clamped down hard on the inside of his mouth and he is now bleeding a little. | keyword (word overlap) | ems5,fire12,fire18 | 0.000 | 0.0 | 0.0 |
| caller's own words | He clamped down hard on the inside of his mouth and he is now bleeding a little. | semantic (embeddings) | ems10,ems9,ems5 | 0.333 | 0.5 | 1.0 |
| caller's own words | He clamped down hard on the inside of his mouth and he is now bleeding a little. | BM25 | fire18,ems5,fire12 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | my mom took too many pills and isn't responding much | keyword (word overlap) | ems11 | 1.000 | 0.5 | 1.0 |
| caller guessed our words | my mom took too many pills and isn't responding much | semantic (embeddings) | ems11,ems12,ems13 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | my mom took too many pills and isn't responding much | BM25 | ems11 | 1.000 | 0.5 | 1.0 |
| caller's own words | Her respiration has significantly dropped off and her eyelids won't lift at all. | keyword (word overlap) | fire20 | 0.000 | 0.0 | 0.0 |
| caller's own words | Her respiration has significantly dropped off and her eyelids won't lift at all. | semantic (embeddings) | ems14,ems19,ems11 | 0.333 | 0.5 | 1.0 |
| caller's own words | Her respiration has significantly dropped off and her eyelids won't lift at all. | BM25 | fire20 | 0.000 | 0.0 | 0.0 |
| caller's own words | She's slumped on the tile near the sink and there's a drained container of her medicine beside her. | keyword (word overlap) | law1,law3,law16 | 0.000 | 0.0 | 0.0 |
| caller's own words | She's slumped on the tile near the sink and there's a drained container of her medicine beside her. | semantic (embeddings) | ems15,ems19,ems12 | 0.333 | 0.5 | 1.0 |
| caller's own words | She's slumped on the tile near the sink and there's a drained container of her medicine beside her. | BM25 | ems19,ems14,ems11 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | one side of his face is drooping and his words are slurred | keyword (word overlap) | ems16,law3,ems17 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | one side of his face is drooping and his words are slurred | semantic (embeddings) | ems16,ems17,ems9 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | one side of his face is drooping and his words are slurred | BM25 | ems16,ems17,law3 | 0.667 | 1.0 | 1.0 |
| caller's own words | He has no strength at all on one whole side and can't get that limb to move. | keyword (word overlap) | law3,ems9,ems16 | 0.000 | 0.0 | 0.0 |
| caller's own words | He has no strength at all on one whole side and can't get that limb to move. | semantic (embeddings) | ems18,ems9,ems16 | 0.333 | 0.5 | 1.0 |
| caller's own words | He has no strength at all on one whole side and can't get that limb to move. | BM25 | ems9,ems16,law3 | 0.000 | 0.0 | 0.0 |
| caller's own words | The woman with blood sugar issues just went down near the stove, her fingers won't stop trembling, and her sight has gone fuzzy. | keyword (word overlap) | law3,law5,ems3 | 0.000 | 0.0 | 0.0 |
| caller's own words | The woman with blood sugar issues just went down near the stove, her fingers won't stop trembling, and her sight has gone fuzzy. | semantic (embeddings) | ems19,ems20,ems14 | 0.667 | 1.0 | 1.0 |
| caller's own words | The woman with blood sugar issues just went down near the stove, her fingers won't stop trembling, and her sight has gone fuzzy. | BM25 | fire20,law5,ems10 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | there's smoke pouring out of my neighbor's roof | keyword (word overlap) | fire1,law1,law16 | 0.333 | 0.5 | 1.0 |
| caller guessed our words | there's smoke pouring out of my neighbor's roof | semantic (embeddings) | fire1,fire2,fire14 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | there's smoke pouring out of my neighbor's roof | BM25 | fire1,law1,fire20 | 0.333 | 0.5 | 1.0 |
| caller's own words | Now there's real fire poking out of the roof openings above the loft space. | keyword (word overlap) | fire1,fire14,law1 | 0.000 | 0.0 | 0.0 |
| caller's own words | Now there's real fire poking out of the roof openings above the loft space. | semantic (embeddings) | fire4,fire14,fire16 | 0.333 | 0.5 | 1.0 |
| caller's own words | Now there's real fire poking out of the roof openings above the loft space. | BM25 | fire1,fire14,fire19 | 0.000 | 0.0 | 0.0 |
| caller's own words | The pet next door keeps yapping nonstop but nobody seems to be out front. | keyword (word overlap) | law9,law2,law12 | 0.000 | 0.0 | 0.0 |
| caller's own words | The pet next door keeps yapping nonstop but nobody seems to be out front. | semantic (embeddings) | fire5,law9,law4 | 0.333 | 0.5 | 1.0 |
| caller's own words | The pet next door keeps yapping nonstop but nobody seems to be out front. | BM25 | law9,law17,law12 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | I smell a strong gas odor coming from my kitchen | keyword (word overlap) | fire6,fire7,fire10 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | I smell a strong gas odor coming from my kitchen | semantic (embeddings) | fire6,fire7,fire10 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | I smell a strong gas odor coming from my kitchen | BM25 | fire6,fire7,fire10 | 0.667 | 1.0 | 1.0 |
| caller's own words | Little bursts of electricity keep popping from the socket in the plaster. | keyword (word overlap) | law12,law14,law15 | 0.000 | 0.0 | 0.0 |
| caller's own words | Little bursts of electricity keep popping from the socket in the plaster. | semantic (embeddings) | fire9,fire10,law10 | 0.333 | 0.5 | 1.0 |
| caller's own words | Little bursts of electricity keep popping from the socket in the plaster. | BM25 | ems5,law14,fire2 | 0.000 | 0.0 | 0.0 |
| caller's own words | The power blinked for a second and now a scorched odor is drifting down the corridor. | keyword (word overlap) | fire7,fire16 | 0.000 | 0.0 | 0.0 |
| caller's own words | The power blinked for a second and now a scorched odor is drifting down the corridor. | semantic (embeddings) | fire10,fire4,fire16 | 0.333 | 0.5 | 1.0 |
| caller's own words | The power blinked for a second and now a scorched odor is drifting down the corridor. | BM25 | fire16,fire7 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | my car caught on fire in the parking lot | keyword (word overlap) | fire11,law16,fire12 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | my car caught on fire in the parking lot | semantic (embeddings) | fire11,law14,law11 | 0.333 | 0.5 | 1.0 |
| caller guessed our words | my car caught on fire in the parking lot | BM25 | fire11,fire12,law16 | 0.667 | 1.0 | 1.0 |
| caller's own words | The flames are jumping to the cars sitting right next to it. | keyword (word overlap) | law16,ems4,fire3 | 0.000 | 0.0 | 0.0 |
| caller's own words | The flames are jumping to the cars sitting right next to it. | semantic (embeddings) | fire13,fire11,fire14 | 0.333 | 0.5 | 1.0 |
| caller's own words | The flames are jumping to the cars sitting right next to it. | BM25 | ems4,fire3,law16 | 0.000 | 0.0 | 0.0 |
| caller's own words | A gust came through and now the blaze has leapt over to the other side of the street. | keyword (word overlap) | law2,law3,ems16 | 0.000 | 0.0 | 0.0 |
| caller's own words | A gust came through and now the blaze has leapt over to the other side of the street. | semantic (embeddings) | fire15,fire14,fire11 | 0.667 | 1.0 | 1.0 |
| caller's own words | A gust came through and now the blaze has leapt over to the other side of the street. | BM25 | ems16,fire4,law2 | 0.000 | 0.0 | 0.0 |
| caller guessed our words | we're trapped upstairs because the stairs are on fire | keyword (word overlap) | fire16,fire17,fire11 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | we're trapped upstairs because the stairs are on fire | semantic (embeddings) | fire16,fire17,fire4 | 0.667 | 1.0 | 1.0 |
| caller guessed our words | we're trapped upstairs because the stairs are on fire | BM25 | fire16,fire17,fire19 | 0.667 | 1.0 | 1.0 |
| caller's own words | There's a pair of grown-ups and a young one who can't get out of the building. | keyword (word overlap) | law1,law16,ems10 | 0.000 | 0.0 | 0.0 |
| caller's own words | There's a pair of grown-ups and a young one who can't get out of the building. | semantic (embeddings) | fire18,fire16,fire17 | 0.667 | 1.0 | 1.0 |
| caller's own words | There's a pair of grown-ups and a young one who can't get out of the building. | BM25 | fire1,law1,ems10 | 0.000 | 0.0 | 0.0 |
| caller's own words | That beeping device in the hallway won't quit even though nothing looks like it's actually burning. | keyword (word overlap) | fire10,ems20 | 0.000 | 0.0 | 0.0 |
| caller's own words | That beeping device in the hallway won't quit even though nothing looks like it's actually burning. | semantic (embeddings) | fire20,fire10,fire4 | 0.333 | 1.0 | 1.0 |
| caller's own words | That beeping device in the hallway won't quit even though nothing looks like it's actually burning. | BM25 | fire10,ems20 | 0.000 | 0.0 | 0.0 |
Where this matters
Call-taking and CAD search: surfacing similar past calls, the right protocol card, or prior incidents at an address, from whatever words the caller uses.
Records and policy: letting officers search training manuals and general orders by situation rather than by the exact term the manual uses.
Where not to use it: exact identifiers such as plate numbers, case numbers, statute citations and names must still be matched exactly. "Close in meaning" is dangerous there; a real system combines both.
How this was measured
60 teaching sentences written for this lab in two voices (911 callers and call-taker CAD notes) across 3 disciplines: emergency medical services, fire, law enforcement. No real calls. Embedded with all-MiniLM-L6-v2 (384 dimensions) and compared by cosine similarity. Retrieval was scored against 36 test queries with hand-written correct answers; whether a query shares vocabulary with its answers is computed from the text, not labelled by hand. Mean reciprocal rank (MRR) is 1.0 when the first correct answer is ranked first, 0.5 when second, and 0 when none appears in the top 3.