Google, Kosmix and the Exploration of the Deep Web

Google, Kosmix and the Exploration of the Deep Web Welcome to the Deep Web, the portion of the Internet not accessible to traditional search engines. Social networks, media-sharing sites for photos and videos, library catalogs, airline reservation systems, phone books, and all kinds of scientific databases lurk inside the Web, practically invisible today’s search tools. The volume of this hidden content is enormous: some estimates have pegged the size of the Deep Web at up to 500 times larger than the slice of the Web we see on search engines today.Ironically, the Deep Web hides some of the richest content on the Internet. The Web 2.0 revolution has enabled an explosion of sites dedicated to user-generated content, including Wikipedia, YouTube, Flickr, TripAdvisor and Yelp. Content on these sites grows so rapidly that it’s nearly impossible for Web crawlers to keep up. Social media sites like Facebook, MySpace, and Twitter pose the same challenge, and often add privacy protections that permit only “friends” to view certain information. In this session, Kosmix Co-Founder Dr. Anand Rajaraman and Dr. Alon Halevy of Google Labs will explore the mysteries of the Deep Web.  Prominent experts in this field, Rajaraman and Halevy will examine how the Deep Web will change the business of Search, offer insight into what content creators should expect as the Deep Web becomes more accessible, and comment on what this shift might mean for the end user. They will also address the question of whether the Deep Web will someday be fully exposed, and if not, why not.   Agenda The now well established Search SIG Agenda format will be used: 6:30-7:00pm – Registration / Food & Drink 7:00–7:05pm – A few words about the Search SIG 7:00-9:00 pm – Program Price $15 at the door for non-SDForum members No charge for SDForum members Date:  November 12, 2009 Location: Cubberley Community Center 4000 Middlefield Rd., RM H-1 Palo Alto, CA Agenda The now well established Search SIG Agenda format will be used: 6:30-7:00pm – Registration / Food & Drink 7:00–7:05pm – A few words about the Search SIG 7:00-9:00 pm – Program

Search UX Innovations with Bing and Google — SDForum Search SIG

On Tuesday, September 29th, join Bing, Google, Yahoo! and your fellow Search SIG members and guests for a lively conversation on innovations in search user experience.

Time: 6:30 – 9 pm. Location: Microsoft Silicon Valley Conference Center, 1065 La Avenida Building 1, Mountain View, CA

Free for SDForum Members. $15 at the door for non-members.

Dinner will be provided at no additional charge by Microsoft.

Confirmed Panelists:

Sean Suchter [quasi-German for "searcher"] runs Microsoft’s Search Technology Center – Silicon Valley. Sean has nearly eleven years experience in web search and previously ran the Yahoo! Search Technology team, where he was responsible for all aspects (engineering, research and product management) of the algorithmic side of Yahoo! web search. Previously to working on web search, Sean did speech recognition research. Sean holds a Bachelor of Science in Engineering and Applied Science from the California Institute of Technology in Pasadena, CA.

Johanna Wright is the Director of Product Management responsible for Web Search.  She oversees the development of Google’s search user interface and features and was the initial product lead on Universal Search, having driven it from inception to launch. Prior to Google, Johanna worked at a number of software start-ups in New York City. Johanna holds an MBA from UCLA and a BA in Mathematics from Barnard College.

Dr. Larry Cornett is the vice president of the Yahoo! Search consumer products division. In this capacity, he is primarily responsible for the Yahoo! web search experience and heads up a multinational team of product managers, designers, and developers focused on creating a world-class search experience.

Prior to this role, Dr. Cornett was a Director at eBay, where his team focused on multiple products for Tailored Shopping Experiences, Platform, and International sites. Earlier in his career, he was a designer at Apple Computer, where he worked on the Finder, OS, and International software; and IBM, where he worked on database and development tools. He was also the principal consultant for MindSpan Design, an interaction design agency where he worked on desktop, web, and mobile solutions for a variety of clients.

Dr. Cornett received his Ph.D. from Rice University, where he designed and developed a coaching system for training software users. He holds multiple patents, which include design work on web-based products and hardware solutions.

Safa Rashtchy

Moderator Safa Rashtchy is well known for his expertise on search and for hosting the teen panel at Web 2.0 Summits and for his prescient investor analysis on the search space as Piper Jaffrey’s Managing Director for Global Internet Media.

Real-Time Search. May 12th Search SIG. SDForum

The basic premise of real time search is to answer the simple yet important question: “What are other people saying about my query right now?” The result is an intriguing blend of news, trends, navigation and discovery, all powered by information users are creating in ever-increasing quantities. Some recent examples of real-time search utility have been around news events (“plane in hudson“), national sentiment (“AIG“) and research on products (“Canon EOS“). As the usage of real-time search starts to vector towards the mainstream, there are important questions to be answered. These range from its integration with web search to eliminating spam and improving overall relevance. This panel features experts in the field who will attempt to provide an overview of the space and its future evolution.

Some areas that will be covered:

- What are the primary use cases of real-time search?

- How does it fit in with classical web search?

- What kind of relevance mechanisms work well for this medium?

- Will it remain a niche service or will it reach the mainstream?

- Will traditional search advertising work in this medium?

Robert Scoble will moderate.
Facebook and Yahoo! are confirmed panelists. Invited panelists include Twitter and Google.

Date:	Tuesday, May 12, 2009
Time:	6:30pm – 9:00pm
Location:	Cubberly Community Center
Street:	4000 Middlefield Rd., Room H-1
City/Town:	Palo Alto, CA

Agenda

The now well established Search SIG Agenda format will be used:
6:30-7:00pm – Registration / Food & Drink
7:00–7:05pm – A few words about the Search SIG
7:00-9:00 pm – Program

Price

$15 at the door for non-SDForum members
No charge for SDForum members

The New York Times on How Patients are Using the Internet to Get Medical Advice

Earlier this week, the New York Times had an article that was very relevant to next week’s Search SIG on the “Search for Better Health.“  Their article “Logging on for a Second (or Third) Opinion” had a lot of interesting data points including (from their Pew Internet and American Life Project reference)

• “of those with a high-speed connection, 1 in 9 do health research on a typical day.
• And 75 percent of online patients with a chronic problem told the researchers that ‘their last health search affected a decision about how to treat an illness or condition”

According to this New York Times piece, “reliance on the Internet is so prevalent, said the report’s author, Susannah Fox, the associate director at Pew, that ‘Google is the de facto second opinion‘ for patients seeking further information after a diagnosis.”

In addition to describing all the useful health information that one can find on the Internet, it’s also intriguing to hear about the patient-to-patient information sharing. The New York Times shares an example of a lady in Oregon who used Twitter during her operation and recuperation. She attributes Twitter to helping her stop taking a certain antinasuea drug due to a friend’s warning her that her teeth chattering might be a side effect of the drug. She also put here surgery and tumor photos on Flickr.

Can we view someone in Oregon as American mainstream or are they still too close to the Valley?

The article also lists out a lot of sites that include

• General Interest ( WEbMD, Discovery Health, the Mayo Clinic)
• Medical Research ( PubMed, Clinicaltrials.gov, Psycinfo, Nccam.nih.gov )
• Patient Sites ( acor.org, e-patients.net, trusera.com)
• Disease-specific sites ( americanheart.org, cancer.org, diabetes.org, breastcancer.org, diabetesmine.com )
• Web Tools (sugarstats.com, drx.com, yourdiseaserisk.com )

Oct 9th. The Search for Better Health: Empowering Consumer Health

Panelists:

• Yahoo’s Director of Product Management, Don Chennavasin
• RightHealth/ Kosmix’s RightHealth Product Manager, Saumil Mehta
• Healthline’s Chairman and CEO, West Shell

Moderator: Matthew Holt, The Health Care Blog and The Health 2.0 Conference

Date:	Thursday, October 9, 2008
Time:	6:30pm – 9:00pm
Location:	Microsoft Silicon Valley Conference Center
Street:	1065 La Avenida, Building 1
City/Town:	Mountain View, CA

From health search engines and widgets to social networks and web med journals, consumers have access to more medical information than ever before. A recent PEW report estimates 75-80% of all Internet users have searched online for health info. In fact, 75% of e-patients with a chronic condition say their last health search affected a decision about how to treat an illness or condition.

How consumers search for health data and interact with information in cyberspace is changing the way we live. From developing healthier habits to asking better questions with their doctors and controlling online medical records, the web is empowering patients around the globe.

In this discussion, Health 2.0 experts from Kosmix’s RightHealth, Healthline, Yahoo! and others will discuss the rising popularity of online health information sites, and its impact on the medical field. This panel, moderated by Matthew Holt of the Health 2.0 conference, will also provide insight into health user behavior, lessons learned and plans for the future.

Search SIG Tomorrow Night.

At least one of the panelists, plans to make a major announcement tomorrow night…

Join the lively discussion with Facebook, FriendFeed, Mahalo, and Wikia Search.

Date: July 8, 2008

Time: 6:30pm-9:00pm

Location: Cubberly Community Center, Room H-1, 4000 Middlefield Road, Palo Alto

Pizza will be served.

New Search SIG Co-Chair

Please join me in welcoming Aditya Agarwal as the new co-chair for the Search SIG. Aditya is Director of Engineering at Facebook, has deep experience in search, and brings great enthusiasm to the Search SIG.

Robert Tri, Co-Chair

July 8th Event. Social Search.

Social Search

Will Community Wisdom Take Search and Discovery to the Next Level?

With your moderator Safa Rashtchy, join a lively discussion with

• Facebook’s Ari Steinberg,
• FriendFeed’s CEO Bret Taylor,
• Mahalo’s CEO Jason Calacanis, and
• Wikia Search’s Jeremie Miller

on what up-and-comers are doing to provide improved search and discovery capabilities through the wisdom of the crowd and how it will impact the overall consumer web search marketplace. We will address questions such as:

• What is social search and discovery? Why and how does it enhance the user experience?
• What are the potentials for social search –what are its limitations?
• When will social search reach an inflection point?
• What are the best platforms/networks or technologies for social search and discovery?
• Who are the most successful existing platforms on social search and discovery?
• Will social search supplement or replace current search and discovery processes?
• What types of players are best positioned to capture the largest market share?
• What are the roles of social networks and “traditional” search sites in this new area?

Confirmed Panelists:

 

Moderator Safa Rashtchy is well known for his expertise on search and for hosting the teen panel at Web 2.0 Summits and for his prescient investor analysis on the search space as Piper Jaffrey’s Managing Director for Global Internet Media.

 

Date: July 8, 2008

Time: 6:30pm-9:00pm

Location: Cubberly Community Center, Room H-1, 4000 Middlefield Road, Palo Alto

Registration: http://www.sdforum.org/index.cfm?fuseaction=Calendar.eventDetail&eventID=13050

Search SIG Tonight!

Using the Social Graph / Social Platforms to Enhance Search & Discovery

When: Tuesday, Feb. 12, 2008
Where: Yahoo! Main Campus, Sunnyvale, CA – Building C. Classroom 5.

Kevin Marks from Google
Aditya Argarwal from Facebook
Kent Brewster of Yahoo!
Eve Phillips CEO of Chirp

With Dave McClure moderating. This should be a good one!

http://www.facebook.com/event.php?eid=8507947786

http://upcoming.yahoo.com/event/421044/ 

Register here: http://sdforum.com/index.cfm?fuseaction=Calendar.eventDetail&eventID=12986

Search SIG Feb. 12, 2008 – The Social Graph & Search

Hey, I know it’s been a long time. But we’re getting back up to speed with an awesome event.

Dude, Where’s My Car?  Using the Social Graph / Social Platforms to Enhance Search & Discovery

When: Tuesday, Feb. 12, 2008
Where: Yahoo! Main Campus, Sunnyvale, CA -  Building C. Classroom 5.
Who: Dave McClure will moderate! (more to come soon)

topics to cover:
- platform overviews / highlight unique features of each platform (API overview / features)
- explore social graph search / friend list lookup — as API feature  (on platform vs embedded external)
- discuss targeted search vs serendipitous discovery (search box vs news feed)
- enabling friend lists / social graph lookup from email / IM / other data sources
- opportunities for 3rd-party developers — distribution / viral acquisition / monetization
- review privacy mgmt / data portability — high priority or not?  alpha geek issue, or mainstream?

Upcoming: http://upcoming.yahoo.com/event/421044/

Register for the event (these fill up fast!):  http://sdforum.com/index.cfm?fuseaction=Calendar.eventDetail&eventID=12986