Searching for meaning...

The technology space is littered with big ideas that never made it into the real world, especially ideas involving the internet and what people might want to use it for. Semantic search is one such "big idea" but I can't help thinking that the jury is still out on a) if it's really feasible and b) can it make money.

Semantic search companies aim to provide people with a mechanism to ask "real-world" questions and get sensible answers (ideally correct ones!) This is different from keyword based search (like Google) where words are typed in and the search engine returns a list of WEB pages that contain those words. A good illustration of the difference would be if you typed in the question "is pink married" into Google you'd get a selection of pages back regarding the colour pink, the pop-star pink and the subject of marriage but probably not a straightforward answer to your question. If you typed the same thing into a semantic search engine you (should) get the answer "yes" because the software has "understood" the context (meaning) of your question and has an underlying database of facts from which it can answer the question correctly.

One company that is trying to achieve this is a Cambridge based firm company called "True Knowledge", there are others, for example "wolfram alpha" and the mainstream players, like Google and Microsoft are also experimenting with semantic concepts in their search engines.

Out of interest I typed the following question into the true knowledge engine, "is Richard Dawkins religious", a resounding "yes" was the answer! 100% wrong but I can see how they might think that. Religion is probably a highly frequent term within documents that also mention "Richard Dawkins" but obviously the answer is completely wrong (in any meaningful sense, even though some apologists would like to bend the meaning of these words)

You can see the problem, in order to answer random questions phrased using human language requires so much more than facts, more often than not it requires actual life experience. This is a really vexing computer science problem, i.e. how do we get software to really understand us without us having to learn alien languages that computers can understand. The True Knowledge engine gets the question about pink being married correct (but then that is one of their stock examples), Wolfram Alpha doesn't clearly not recognising the fact that in the context of this particular question pink means the pop-star pink.

The second issue facing companies trying to solve this problem is how can it be monetised? Whilst it's easy to see how someone wanting to win a pub quiz could potentially use such technology what value is it outside of academic interest, or put another way who would pay for it? Google famously solved this problem with advertising and I suppose these new search engine companies could try the same thing, but that would seem difficult to me since there is such a momentum around keyword search; so how about businesses? Let's say I'm a sales person wanting to find out if the company True Knowledge is a suitable prospect for my product or service, that's intelligence I might consider paying for so a typical question would be "how many people work for true knowledge". I tried this in the true knowledge system, it couldn't answer the question, then I tried the same question in Wolfram Alpha and got the answer 3 million (it clearly misunderstood what I meant).

On both fronts of practicality and commercial value there is clearly still much work to be done.