Turns out that if I enter "Frankfurt, Germany" into Apple Maps (which I assume is what the package tracker app does), it takes me to "Frankfurt (Oder), Germany" instead of "Frankfurt am Main, Germany".

All these thresholds or ranking factors seem to come intuitively to humans (I would guess a good intuition for them is actually a sign of intelligence), but it seems to be incredibly hard to capture them in ranking.

As others have pointed out, a solution here would be to make Siri more conversational. A simple "Which London?" could've removed the ambiguity and given Siri the opportunity to learn something about that particular person (that London, England is more important to him than London in Canada).

Reality is hard. And with machine learning (especially proprietary, remotely-hosted machine learning) there's rarely a way to pinpoint a line of code and say: "this is what happened and why you're now frustrated and firing hypothetical personal assistants".

Before lockdown I even had it disabled entirely because it would get activated randomly from time to time, even if nobody in the vicinity said anything remotely close to "Hey Siri".

However, are you planning to visit your parents in London, Canada this weekend? Then an assistant who would still answer with the time of London, UK would maybe also not be the smartest?

So really context is everything and making broad statements that if an assistant was to answer with anything but London, UK should get fired is something that someone would say, who IMHO should get fired. shrug

Also, IMHO, if someone doesn't know that machines don't have human context and therefore doesn't know to ask their digital assistant "What is the time in London, UK" when they want to know the time in London, UK, then maybe they should get fired from their tech job. shrug

If the person asking the question lives in Ohio, they may actually be talking about London OH (or Dublin OH). Some people in neighbouring states may mean the same, though they will be more likely to mention the state. However, how close should you be to London, OH even within the state to mean the Ohio one and not the UK one? How close is close enough? Is a few hours of driving close enough? A 3 hour flight? What if I'm roughly at 6 hours from London OH and 7 hours from London UK?

Further, if the person is a British expat in Ohio, especially if they are working for a multinational business (or not), they would more likely mean London UK. German expats, though? Russians? Or an Irish person who lives in Amsterdam having some relatives in Ohio US, looking to book a flight to Dublin. Etc. etc.

There are so many contextual layers here that even human assistants can occasionally get it wrong, and without the context the task becomes insurmountable for the "AI" algorithms. That is not to say virtual assistants are useless, just that selling them as "AI" is a big lie, bigger than even those who market these algorithms as "AI" think it is.

Here comes my favourite brain freeze moment - recently my parents asked me to explain this to them. How do you construct a good search phrase? My brain blanked. I HAVE NO IDEA. It seems I have learned fluent Goonglish without noticing, and now can't explain the grammar or vocabulary of it.

Let users create and share small commands. Create a simple natural language for commands that are easy to program, extend, and remember, and narrow the scope of inputs the voice engine has to deal with.

It was so beautiful and effective and just light years ahead of what we're getting.

Microsoft gets a special mention for lost potential here. Their voice system in Windows could be a way to navigate the layered menus of the OS, but it is mostly focused on answering general queries. Voice is a great replacement for the program launcher, except it's not customizable, but that's about the extent of how much you can control the system with it. Let me do anything buried in the control panel, show me everything you know about a process when I ask, solve that first, then worry later about telling me how big the moon is. You make an OS, don't forget what that is.

I mean, if anything, just appreciate how amazing humans are at differentiating these ambiguities.

But that fails completely when you get to names like London (or Paris or Moscow or Cairo).

But it happens with people, too. I'm from Mississippi, though I haven't lived there since I left for college. I now live in Houston. At a family reunion many years ago, I ran into a cousin I hadn't seen since we were kids. She asked where I was living, and I told her.

"Oh, isn't it terrible about that wreck?" she asked.

Baffled, I asked for more information. "Oh, you know, that wreck over on 406!"

I did not know. "I'm sorry, Houston's really huge. I don't know what wreck you mean."

"Oh, did you mean you live in Houston, TEXAS? I thought you meant Houston, MISSISSIPPI!"

I was, at the time, about 30. I grew up in that state, and lived there until I went to college. And until that moment, I had never even HEARD of Houston, Mississippi (a metropolis, it turns out, of about 3600 people in the misbegotten northeast corner of the state).

Sometimes you can ask a question and watch it be perfectly transcribed in real time, but then receive a nonsensical answer from it. Ask the exact same question immediately after on the same device, transcribed exactly the same way, and get the correct answer.

Where does such unpredictability come from? How can Siri transcribe the words correctly but fail to deliver the right answer?

After a couple of weeks of this, I somehow got the idea that it was related to the devices' configured locations. And sure enough, telling the Home that I lived in the next city over fixed the problem.

So I started a binary search and eventually found that the issue was limited to my ~10x20 block Seattle neighborhood - basically the outline shown when I search for its name in Google Maps. I then also realized that it applied to weather queries on my phone as well, but since the phone uses GPS rather than a specific location setting, I could only reproduce the broken and working behaviors by crossing one of the neighborhood boundary streets.

Turns out it was some long-standing configuration issue with Knowledge Graph's entry for my neighborhood, and some recent code change in location-based weather queries began butting heads with it. Luckily I worked at Google at the time and was able to track down and pester people that could help fix the issue.

90% of the time works fine, and it's essentially all I use Siri for. It's very convenient when cooking and my hands are dirty. But 10% of the time I get something along the lines of...

"I'm sorry, but you don't have the Timer app installed".

"I'm sorry, but you don't have the Timer app installed".

"I'm sorry, but you don't have the Timer app installed".

"I'm sorry, but you don't have the Timer app installed".

It's infuriating because I know Siri is dumb so I use the same exact simple phrases to avoid confusion. Sometimes it works, sometimes it doesn't. It always transcribes the command accurately though! I've actually lost my temper and smashed an Apple Watch before over this. This is in my house, on a very reliable network, always with my phone within a reasonable distance.

I only have one smart lock, which works perfectly, and it is called "FRONT DOOR" in HomeKit.

When I ask Siri about my FRONT DOOR she responds that she cannot find it.

When I ask Siri about the status of my DOOR, she responds with "The FRONT DOOR is locked/unlocked".

I'll then say 'Alright Siri you literally just used the phrase "FRONT DOOR" five seconds ago and the text transcript on the screen says "FRONT DOOR" hey Siri is my FRONT DOOR locked'

Siri: WTF are you talking about? You don't have a FRONT DOOR.

"Hey Siri is my door locked"

Siri: Your FRONT DOOR is locked.

Google and Alexa handle things flawlessly.

Until they have real metrics around how often Siri fails they will continue to think that their correct response rate is great.

I was caught off guard and told her I'd prefer the one in Massachusetts. I did not fire her. She was young, had a poor general education, and had never traveled outside her home state. Those things do not make her stupid.

Me: "Hey Siri, play Radiolab podcast"

Siri: Which Radiolab podcast, Radiolab or Radiolab: More Perfect"

Me: "Radiolab"

Siri: Which Radiolab podcast, Radiolab or Radiolab: More Perfect"

Me: "Radiolab"

Siri: Which Radiolab podcast, Radiolab or Radiolab: More Perfect"

...

Me: "The first one"

Siri: "I don't know >the first one<"

Me: "Siri you're useless"

Siri: "That's not nice"

Me: "Could be but it's true"

I filed a request to NOT pre-fill the starting city to a place in Kentucky and got a polite but firm reply that this was the default list provided by the 3rd party online booking engine and cannot be customized. Sigh.

But that would make it almost as smart as an Infocom game from 1981. Something, something, doesn't scale, mumble, something...

Instead of looking at people, you can also scrape websites to get the relations. But here you may get a recursive problem because if a website speaks of "London", you might not know in advance which London they speak of.

I think the most damning part is how, at the bottom, he list a handful of other "smart" assistants which correctly list London UK's time... for now.

But, his point about consistency and slowness is exactly why I never use these shitty voice assistants. If I'm going to be interacting with some pedantic robot, I generally want to be able to edit the text of my request.

TLDR; voice assistants suck.

You actually need to imagine additional context to make any other answer plausible.

If you hear the sound of hooves clip-clopping nearby, you think horses, not zebras.

Judging from the published pictures [0], it looks like Google Maps may lead to the same result as Apple Maps for "Frankfurt".

[0] https://talksport.com/football/529808/benfica-fan-wrong-fran...

But yes, Siri is the worst.

This is supposed to be a personal assistant. And I have a whole list of what it could do for me, personally. But it doesn't.

I've been trying to figure out how to hook Google's speech recognition and voice into other apps, since they're great and it's 99% of what I need, hands-free control and feedback. Maybe they should make that easy, preferably offline and let other people create their own personal assistant modules or something.

I've got no idea about Siri, but the android one ties into your Google account to get your calendar and mail so it can get context about up coming travel etc.

I would seriously doubt this assumption. Why on earth should someone living in a state specifically ask for the local time in a different location within that same state?

On the contrary, this context information would make it much more likely that the person actually meant "London, England". Except if there is a timezone border going through the state, of course.

However, I obviously agree with your general point regarding the severe limitations of what we currently call "AI" and how little "intelligence" there actually is.

IMO I would be very disappointed if Siri started asking clarifying questions at a significantly higher rate. Siri is already a bit too chatty, and I never feel like having an extended conversation with her.

I’d rather she just say the wrong thing (but make it clear that the answer is for a specific London, e.g. “The time in London Ontario is...”) and I can correct her. It’s the same number of conversational “turns”, but in the happy path when she actually gets it right the first time, it’s one-shot and done.

It’s a lot harder to get signal on this for learning, but I feel like there are ways around this as well. (Maybe saying “thanks” can signal she got something right, and prefixing the next utterance with “no” could signal it was wrong...)

You can't reply with "no Siri, not that London" and have it remember. It doesn't learn your voice among the people who normally use your Siri in your household.

"Artificial intelligence" is always going to make mistakes, as do real humans. Humans can perform unsupervised learning - in fact it's one of the key skills that employers like to select on! Until AI can learn in context it's going to be very limited.

I think it's possible that general acceptance of these non-AI gimmicks being referred to as "AI" will end up pushing genuine progress in true AI further into the future.

Recently Google got much better in understanding full sentences and there are tons of SEO optimized pages for certain phrases. Nevertheless, using keywords is what I imagine advanced users do.

The problem reminds me of the difficulty of programming in applescript. In applescript, articles like "the" can be inserted optionally in the code, and there are lots of equivalent ways to write things, i.e. "if x equals y" is the same as "if x is equal to y". As a result I never remember the syntax, and error messages are less helpful.

You will be surprised on how many cities are named 'San francisco' in the world : https://en.wikipedia.org/wiki/San_Francisco_(disambiguation)

That’s kind of weird. It’s not that Siri is especially bad. It’s that “Siri” is something different depending on how you query it. Other online search systems aren’t like that, and integration and consistency are typically Apple’s forte.

Voice assistants generally use both the text transcription and a bunch of contextual metadata as input. That metadata could include things like what's currently visible on the screen, your location, your recent queries, etc.

So even though the underlying algorithms powering the assistant may be deterministic, the input data between two seemingly identical queries could vary quite a bit.

For instance, Siri almost certainly has context around the previous questions you've asked. It would be reasonable to assume that if an assistant received two identical questions back-to-back the initial answer was wrong.

In that scenario, the assistant might decide to use the a different answer (perhaps one that had a lower ranking) in an attempt to get it right.