Semantics

Latest news on semantics

The future of search

The Internet has had an enormous impact on people's lives around the world in the 10 years since Google's founding. It has changed politics, entertainment, culture, business, health care, the environment and just about every other topic you can think of. Which got us to thinking, what's going to happen in the next 10 years? How will this phenomenal technology evolve, how will we adapt, and (more importantly) how will it adapt to us? We asked ten of our top experts this very question, and during September (our 10th anniversary month) we are presenting their responses. As computer scientist Alan Kay has famously observed, the best way to predict the future is to invent it, so we will be doing our best to make good on our experts' words every day. - Karen Wickre and Alan Eagle, series editors.I am a search addict. I?m naturally inquisitive ? I?ve always liked finding things out. Plus, I?ve worked at Google on search for the past 9 years and 3 months. Of course I search - a lot. Yet I would guess that on any given day, I only do about 20% of the searches that I could. This past Saturday, I kept track of the things that came up in conversation that I wanted to search for right then but couldn?t:Are "fab," "goy" and "eely" words? (There was a Scrabble game going on.) What time does J.C. Penney open on Saturday? Which school has a team called the Banana Slugs? What is the team mascot for San Jose State? How much power does that hydroelectric dam generate? What do you call a group of turkeys? What time does Tropic Thunder show? What?s the name of that great Irish flute player, first name James? What?s the name of the largest city in Russia after Moscow and St. Petersburg? Which is older, a redwood or a cypress? What?s the oldest living thing and how old is it? Who sings ?Queen of Hearts?? What kind of bird is that flying over there? Is the "LF" in San Francisco on Union Square or Union Street? What are the dance steps to the Charleston? What day of the week was The Lawrence Welk Show on? What are the lyrics to ?In the Mood?? How does Coumadin differ from aspirin in its blood thinning effects? What was the story behind the naming of the number "googol"?And those are just the ones that I remember. Looking at this list, two things are very clear: (1) I could do a lot more searches and (2) search still has a lot of opportunity for innovation, change, and progress. There are lots of ways that search will need to evolve in order to easily meet user needs. Let?s look at some of my unanswered questions from Saturday and consider how search might change over the next 10 years.ModesFirst, why couldn?t I do these searches right then, when I needed to? Because search still isn?t accessible enough or easy enough. Search needs to be more mobile ? it should be available and easy to use in cell phones and in cars and on handheld, wearable devices that we don?t even have yet. For example, when the topic of the oldest living thing came up during a boat ride, everyone in the conversation was curious about it, but no one wanted to break out an awkward, slow device to do a search. It would be much nicer if we had a device with great connectivity that could do searches without interruption. One far-fetched idea: how about a wearable device that does searches in the background based on the words it picks up from conversations, and then flashes relevant facts?This notion brings up yet another way that ?modes? of search will change ? voice and natural language search. You should be able to talk to a search engine in your voice. You should also be able to ask questions verbally or by typing them in as natural language expressions. You shouldn?t have to break everything down into keywords.Further, why should a search be words at all? Why can?t I enter my query as a picture of the birds overhead and have the search engine identify what kind of bird it is? Why can?t I capture a snippet of audio and have the search engine identify and analyze it (a song or a stream of conversation) and tell me any relevant information about it? Services that do parts of that are available today, but not in an easy-to-use, integrated way.In the next 10 years, we will see radical advances in modes of search: mobile devices offering us easier search, Internet capabilities deployed in more devices, and different ways of entering and expressing your queries by voice, natural language, picture, or song, just to name a few. It?s clear that while keyword-based searching is incredibly powerful, it?s also incredibly limiting. These new modes will be one of the most sweeping changes in search.MediaThen there?s the media aspect. The 10 blue links offered as results for Internet search can be amazing and even life-changing, but when you are trying to remember the steps to the Charleston, a textual web page isn?t going to be nearly as helpful as a video. The media of the results matters.Universal search, which we released last May, was an important first step that included images, videos, news, books, and maps/local information in our main Google search results. Yet our presentation is still very linear (the results are just a list) and even (no one result is more important or larger than the next). What if the results page began to transform radically to really harness these different types of results into something that felt much more like an answer rather than just 10 independent guesses? What if results pages pulled the best media together and laid it out such that the most useful content was not only first but largest? What if we laid out content in columns to use more of the width available on newer, wider screens?We?ve barely scratched the surface with universal search, but it?s an important first step to exploring the full range of what we can do with rich media. For the past year, our goal has been to take advantage of these new types of results and evolve the interface design and user experience in response. You?ll see the fruits of this experimentation in the coming months, but even these changes are just the beginning. The face of search will change dramatically over the next 10 years. Maybe it should contain even more videos and images, maybe it should sharply differentiate the relative weight and accuracy of the results more, maybe it should be more interactive in terms of refinements? We?re not sure yet, but we do know that the one thing that the search experience can?t be - especially in the face of the online media explosion we?re currently experiencing - is stagnant.PersonalizationSearch engines 10 years from now will be a lot better than the ones we have now. We know this because Google itself gets a little better each day. We?re constantly writing and revising new notions of search relevance, and we release improvements almost daily. Those improvements add up for us and for other search engines, so it follows that search engines 10 years from now will be markedly better. Therefore, the real question is not will search be better, but rather how will it be better?One answer is clear: search engines of the future will be better in part because they will understand more about you, the individual user. Of course, you will be in control of your personal information, and whatever personal information the search engine uses will be with your permission and will be transparent to you. But even with the most rudimentary user information, search engines can and will provide drastically better search results. Maybe the search engines of the future will know where you are located, maybe they will know what you know already or what you learned earlier today, or maybe they will fully understand your preferences because you have chosen to share that information with us. We aren't sure which personal signals will be most valuable, but we're investing in research and experimentation on personalized search now because we think this will be very important later.Location Your location is one potentially useful facet of personalized information. Looking at my questions, the answers to a number of them (What time does J.C. Penney open? How much power does that hydroelectric dam generate? What time does Tropic Thunder play?) require the search engine to know that I was in Yankton, South Dakota and Crofton, Nebraska when I asked. Since location is relevant to a lot of searches, incorporating user location and context will be pivotal in increasing the relevance and ease of search in the future.SocialAnother element of personalization is social context. Who am I friends with, and how do I relate to them? How can I harness their knowledge more efficiently? For example, I have a friend who works at a store called LF in Los Angeles (hence, the question about LF in San Francisco). By itself, ?LF? is a very ambiguous acronym. According to the first page of search results on Google, it could refer to my friend?s trendy fashion store, but it could also refer to Leapfrog Enterprises, low frequency, Lebhar-Friedman, Li & Fung Investment Group, LF Driscoll Construction Management, large format, or a future concept car design from Lexus. Today, the person typing ?LF? has to figure out which is the right result ? to ?disambiguate? the ambiguous term ? but this is something that the search engine needs to get better at. Perhaps we?ll understand the semantics of the question about where LF in San Francisco is, and infer that LF is a store. Or maybe, search could analyze my social graph and realize that one of my friends works at LF, that I saw that friend this weekend, and that in that context ?LF? refers to her place of employment. Algorithmic analysis of the user?s social graph to further refine a query or disambiguate it could prove very useful in the future.In addition, there are searches where actually asking a friend helps. I was having a hard time finding out the answer to the question about aspirin versus Coumadin because I was spelling it ?cumitin? and Google wasn?t correcting me. A quick email to a doctor friend, and I was back on the right track - equipped with the right spelling and his explanation of the difference, so I could search and learn even more about how these two drugs are used to thin blood. There?s a lot of expertise, knowledge, and context in users? social graphs, so putting tools in place to make ?friend-augmented" search easy could make search more efficient and more relevant.LanguageThe above examples show how modes, media, and various forms of personalization have the potential to vastly improve search ? but what about language? We know there are cases where an answer exists on the web, but not in a language you read. This is why Google is investing in machine translation. We want to be able to unlock the power of web search for anyone speaking any language. The basic concept is ? if the answer exists online anywhere in any language, we?ll go get it for you, translate it and bring it back in your native tongue. This is an incredibly empowering idea that could really change the way that users experience the web and communicate with each other, particularly in languages where not a lot of native content is available. You can see our early explorations in this space here, by visiting our cross-language information retrieval tool.Conclusion We?re all familiar with 80-20 problems, where the last 20% of the solution is 80% of the work. Search is a 90-10 problem. Today, we have a 90% solution: I could answer all of my unanswered Saturday questions, not ideally or easily, but I could get it done with today?s search tool. (If you?re curious, the answers are below.) However, that remaining 10% of the problem really represents 90% (in fact, more than 90%) of the work. Coming up with elegant, fitting and relevant solutions to meet the challenges of mobility, modes, media, personalization, location, socialization, and language will take decades. Search is a science that will develop and advance over hundreds of years. Think of it like biology and physics in the 1500s or 1600s: it?s a new science where we make big and exciting breakthroughs all the time. However, it could be a hundred years or more before we have microscopes and an understanding of the proverbial molecules and atoms of search. Just like biology and physics several hundred years ago, the biggest advances are yet to come. That?s what makes the field of Internet search so exciting.So what's our straightforward definition of the ideal search engine? Your best friend with instant access to all the world?s facts and a photographic memory of everything you?ve seen and know. That search engine could tailor answers to you based on your preferences, your existing knowledge and the best available information; it could ask for clarification and present the answers in whatever setting or media worked best. That ideal search engine could have easily and elegantly quenched my withdrawal and fueled my addiction on Saturday. I?m very proud that Google in its first 10 years has changed expectations around information and how quickly and easily it should be able to be retrieved. But I?m even more excited about what Google search can achieve in the future.And here, in order, are the answers to my Saturday questions.Are fab, goy, and eely words? Yes, yes, and yes, according to Merriam-Webster:Search: [fab site:m-w.com ]Result: http://dev.m-w.com/dictionary/fabSearch: [goy site:m-w.com]Result: http://dev.m-w.com/dictionary/goySearch:[eely site:m-w.com ]Result: http://dev.m-w.com/dictionary/eelyWhat time does J.C. Penney open on Saturday? 10 a.m.Search: [jc penney yankton ]Hours on results page: http://www.google.com/search?q=jcpenney+yanktonWhich school has a team called the Banana Slugs? University of California, Santa CruzSearch: [banana slugs]Result: http://en.wikipedia.org/wiki/University_of_California,_Santa_CruzWhat is the team mascot for San Jose State? The San Jose State SpartansSearch: [san jose state mascot]On results page: http://www.google.com/search?q=san+jose+state+mascotHow much power does that hydroelectric dam generate? $35M of electricity annuallySearch: [hydroelectric dam crofton yankton]Search: [gavins point dam]Result: https://www.nwo.usace.army.mil/html/Lake_Proj/gavinspoint/welcome.htmlWhat do you call a group of turkeys? A rafter of turkeysSearch: [group of turkeys]On results page: http://www.google.com/search?q=group+of+turkeysWhat time does Tropic Thunder show? 7 p.m.Search: [movies yankton mall]Result: http://www.moviefone.com/theater/carmike-cinemas-yankton-mall-5/9346/showtimesWhat?s the name of that great Irish flute player, first name James? James GalwaySearch: [irish flute player james]On results page: http://www.google.com/search?q=irish+flute+player+jamesWhat?s the name of the largest city in Russia after Moscow and St. Petersburg? NovosibirskSearch: [largest Russian cities]Result: http://en.wikipedia.org/wiki/List_of_cities_and_towns_in_Russia_by_populationWhat?s older, a redwood or a cypress? Cypresses (4500 years old is oldest known) are older than redwoods (2200 years old is oldest known)Search: [cypress tree age]Result: http://www.payvand.com/news/08/apr/1253.htmlSearch: [redwood tree age]Result: http://www.sempervirens.org/sequoiasemp.htmWhat?s the oldest living thing and how old is it? The bristlecone pine, living for 5,000-11,000 yearsSearch: [oldest living thing]Result: http://waynesword.palomar.edu/ww0601.htmhttp://hubpages.com/hub/Oldest_living_thingWho sings ?Queen of Hearts?? Juice NewtonSearch: ["queen of hearts" song]On results page: http://www.google.com/search? =%22queen+of+hearts%22+songWhat kind of bird is that flying over there? A turkey vultureSearch: [turkey vulture flying] on Google image searchPictures that match on results page: http://images.google.com/images?q=turkey%20vulture%20flyingIs the LF in San Francisco on Union Square or Union Street? 1870 Union StreetSearch: [lf san francisco]Address on results page: http://www.google.com/search?q=lf+san+franciscoWhat are the dance steps to the Charleston? Show in video belowSearch : [Charleston dance demonstration]Video result: http://uk.youtube.com/watch?v=zzyg7l6qxNQWhat day of the week was The Lawrence Welk Show on? SaturdaySearch: [lawrence welk show]Result: http://en.wikipedia.org/wiki/The_Lawrence_Welk_ShowWhat are the lyrics to ?In the Mood???In the mood, that's what he told me,In the mood, and when he told me,In the mood, my heart was skippin',It didn't take me long to say "I'm in the mood now".?Search: [?in the mood? lyrics]Result: http://www.lyricsdepot.com/glenn-miller/in-the-mood.htmlHow does Coumadin differ from aspirin in its blood thinning effects? Aspirin is an anti-platelet agent that prevents clotting. Coumadin also prevents clotting but the mechanism is different. Both thin the blood, but Coumadin is stronger and much more effective in certain instances like atrial fibrillation.Search: [aspirin Coumadin how different]Result: http://www.stmaryhealthcare.org/body.cfm?id=250What was the story behind the naming of the number "googol"?Search: [number googol named]Result: http://en.wikipedia.org/wiki/Quintillion#The_googol_familyPosted by Marissa Mayer, VP, Search Products & User Experience

Microsoft Live Search gets Powerset boost

Google-bashing semantics Microsoft yesterday revealed how it has begun integrating Powerset?s semantic internet search technology into its own Live Search product.?

Microsoft and Immersion settle settlement settlement

The semantics of force feedback Microsoft and the force-feedback technology firm, Immersion, appear to have come to final terms after six years of suing each other.?

Last Call: XProc: An XML Pipeline Language

2008-08-14: The XML Processing Model Working Group has published the Last Call Working Draft of XProc: An XML Pipeline Language. This specification describes the syntax and semantics of XProc: An XML Pipeline Language, a language for describing operations to be performed on XML documents. A pipeline consists of steps. Like pipelines, steps take zero or more XML documents as their inputs and produce zero or more XML documents as their outputs. Comments are welcome through 26 September. Learn more about the Extensible Markup Language (XML) Activity. (Permalink)

Games Without Frontiers: Fun Way to Lose Weight: Turn Dieting Into an RPG

A friend of mine recently slimmed down on Weight Watchers. She joined two months ago, and in just a couple of weeks, she'd shed 10 pounds. She'd been trying for a year to lose weight, but nothing worked -- until now. Why did Weight Watchers work so well? For a really fascinating reason: because it isn't a normal diet. It's something more. Something fun. It's an RPG. The Weight Watchers program is designed precisely like a role-playing dungeon crawler. That's why people love it, stick to it and have success with it. And it points to the way that we could use game design to make life's drudgery more bearable. Let me unpack this a bit. When I asked my friend to see how Weight Watchers works, she showed me the ingenious system. "The best part is the web tool," she said, pulling it up on her laptop. When you first log in to Weight Watchers, it determines how much food you'll be allowed to eat that day, expressed as a number of "points." My friend gets 23 points per day. Each time she eats a piece of food, she enters it into the online database, and it calculates how many points she's used. A small apple is one point; a piece of fried chicken is seven points. When she first started the program, she was stunned at how quickly she burned through her daily points. A single bagel was six points -- more than 25 percent of her daily quota. "How the hell am I going to do this without starving?" she wondered. But pretty soon she learned to hack her daily eating to suit the system. She snacked on vegetables that took zero points -- like bell peppers -- or only one or two points, like a tasty brand of microwave popcorn. Then she'd save up the big points for a really decent dinner. Better yet, Weight Watchers assigns her an extra 35 bonus points per week that she can use if she goes over her daily limit. Or she can bank them for a big blowout restaurant meal on the weekend. What makes this point-counting possible is Weight Watchers' elegant online tool. Type in any food you can think of -- including brand-name snacks or boxed meals -- and Weight Watchers has already calculated the points for you. If she makes a special sandwich at home, she can calculate the ingredients, save it with a custom name, and then drag and drop it into her day every time she eats one. As I watched her poke around on the screen, managing inventory, calculating points, staying within her range, it hit me: Weight Watchers is an RPG. Think about it. As with an RPG, you roll a virtual character, manage your inventory and resources, and try to achieve a goal. Weight Watchers' points function precisely like hit points; each bite of food does damage until you've used up your daily amount, so you sleep and start all over again. Play well and you level up -- by losing weight! And the more you play it, the more you discover interesting combinations of the rules that aren't apparent at first. Hey, if I eat a fruit-granola breakfast and an egg-and-romaine lunch, I'll have enough points to survive a greasy hamburger dinner for a treat! Even the Weight Watchers web tool is amazingly gamelike. It has the poke-around-and-see-what-happens elegance you see in really good RPG game screens. Accidentally snack on a candy bar and ruin your meal plan for the day? No worries: Just go into the database and see what spells -- whoops, I mean foods -- you can still use with your remaining points. And those 35 extra points you get every week? They're like a special buff or potion -- a last-ditch save when you're on the ropes. Indeed, I'm in awe of the sheer brilliance of Weight Watchers in adopting the word points as its metric for measuring food. The word immediately shoves the user into the semantics -- and fun -- of gameplay. You regard losing weight as an intriguing challenge, as opposed to a mere grind. This puts me in mind of the talk that Jane McGonigal -- a brilliant and pioneering alternative-reality game designer -- gave at this year's South by Southwest conference. She argued that game designers ought to put their skills to use in the real world by reshaping dull, everyday activities into fun challenges. Why not a game that gives you points for walking your dog or jogging? "Games are an incredible language and system. They should be everywhere," she said. "Why are we making games only for the bound pages for a computer screen or console? Why aren't we doing that to help people navigate and understand the world around us?" She couldn't be more right. As McGonigal points out, there are already some witty attempts -- like Chore Wars, Wii Fit or Seriosity's system that tries to limit corporate e-mail overload by forcing people to "spend" virtual totems to send a message. I can think of tons of things I'd love to see turned into a game: doing my taxes, dealing with my inbox backlog, being stuck in traffic. And this stuff is clearly possible, because if Weight Watchers can turn something as unpleasant as dieting into a playful activity, the sky's the limit. Just ask my friend. - - - Clive Thompson is a contributing writer for The New York Times Magazine and a regular contributor to Wired and New York magazines. Look for more of Clive's observations on his blog, collision detection.

Fox News' Jarrett falsely claimed Obama is making his "first trip to the Middle East"

During the July 19 edition of Fox News' America's Election HQ, anchor Gregg Jarrett falsely claimed that Sen. Barack Obama's overseas trip, which includes planned stops in Iraq, Israel, and Jordan, constituted his "first trip to the Middle East." In fact, as Media Matters for America has documented, Obama made a January 2006 Middle East trip that included stops in Iraq, Israel, and Jordan. From the July 19 edition of Fox News' America's Election HQ: JARRETT: All right, Inga, let me begin with you. What are your listeners -- INGA BARKS (talk radio host): Hi, Gregg. JARRETT: -- saying about Obama's first trip to the Middle East? Are they expecting sort of the usual political spin that you get from politicians? BARKS: Semantics, semantics, semantics. No matter how this turns out, Gregg, if -- he's going to say -- he's going to spin it to say that there's -- that it is consistent with his original policy. JARRETT: Yeah. Joe -- BARKS: Right. I mean, no matter what they say it's going to be consistent with whatever he originally said. He was right all along. JARRETT: Well, but you know what? Maybe some of it is. Because, Joe, [Iraq Prime Minister] Nouri al-Maliki, today -- JOE MADISON (talk radio host): That's right. JARRETT: -- calling Obama's 16-month withdrawal plan, quote, "the right timeframe for withdrawal." So, I mean, look -- MADISON: With -- with -- yeah. BARKS: Yeah. JARRETT: -- do your listeners think this trip -- MADISON: No. JARRETT: -- may actually dispel concerns that he's not ready to be commander in chief? What do you think, Joe?

POWDER Formal Semantics First Working Draft Published

2008-07-09: The Protocol for Web Description Resources (POWDER) Working Group has published the First Public Working Draft of Protocol for Web Description Resources (POWDER): Formal Semantics. This document underpins the Protocol for Web Description Resources (POWDER). It describes how the relatively simple operational format of a POWDER document can be transformed through two stages, first into a more tightly constrained XML format (POWDER-BASE), and then into an RDF/OWL encoding (POWDER-S) that may be processed by Semantic Web tools. The formal semantics of POWDER are best understood after the reader is acquainted with the Description Resources and Grouping of Resources documents. Learn more about the Semantic Web Activity. (Permalink)

State of Speech Reco and Synthesis

There?s a very detailed (and long) article on the state and future of speech recognition and speech synthesis in the New York Times from late June. Although the prognosis is not that positive, it is written almost with the challenge of a Turing test for speech recognition, i.e., a computer recognizing the semantics of human [...]

Mark Lucovsky visits the Gillmor Gang

Google's Mark Lucovsky talks about search APIs, cache semantics, cloud computing, Live Mesh, and his fondness for Microsoft Outlook on the Gillmor Gang.