Recent years have seen a dramatic growth of natural language text data, including web pages, news articles, scientific literature, emails, enterprise documents, and social media such as blog articles, forum posts, product reviews, and tweets. Text data are unique in that they are usually generated directly by humans rather than a computer system or sensors, and are thus especially valuable for discovering knowledge about people’s opinions and preferences, in addition to many other kinds of knowledge that we encode in text. This course will cover search engine technologies, which play an important role in any data mining applications involving text data for two reasons. First, while the raw data may be large for any particular problem, it is often a relatively small subset of the data that are relevant, and a search engine is an essential tool for quickly discovering a small subset of relevant text data in a large text collection. Second, search engines are needed to help analysts interpret any patterns discovered in the data by allowing them to examine the relevant original text data to make sense of any discovered pattern. You will learn the basic concepts, principles, and the major techniques in text retrieval, which is the underlying science of search engines.
Lesson 6.4: Future of Web Search
Loading...
Из курса от партнера University of Illinois at Urbana-Champaign
Text Retrieval and Search Engines
383 оценки
Из урока
Week 6
In this week's lessons, you will learn how machine learning can be used to combine multiple scoring factors to optimize ranking of documents in web search (i.e., learning to rank), and learn techniques used in recommender systems (also called filtering systems), including content-based recommendation/filtering and collaborative filtering. You will also have a chance to review the entire course.
Познакомьтесь с преподавателями
ChengXiang ZhaiProfessor
Department of Computer Science
[SOUND].
This lecture is about the future of web search.
In this lecture, we're going to talk about some possible future trends
of web search and intelligent information retrieval systems in general.
In order to further improve the accuracy of a search engine,
it's important that to consider special cases of information need.
So one particular trend could be to have more and more specialized than
customized search engines, and they can be called vertical search engines.
These vertical search engines can be expected to be more effective than
the current general search engines because they could assume that
users are a special group of users that might have a common information need,
and then the search engine can be customized with this ser, so, such users.
And because of the customization, it's also possible to do personalization.
So the search can be personalized,
because we have a better understanding of the users.
Because of the restrictions with domain, we also have some advantages
in handling the documents, because we can have better understanding of documents.
For example, particular words may not be ambiguous in such a domain.
So we can bypass the problem of ambiguity.
Another trend we can expect to see,
is the search engine will be able to learn over time.
It's like a lifetime learning or lifelong learning, and this is, of course,
very attractive because that means the search engine will self-improve itself.
As more people are using it, the search engine will become better and better, and
this is already happening,
because the search engines can learn from the [INAUDIBLE] of feedback.
More users use it, and the quality of the search engine allows for
the popular queries that are typed in by many users allow it to become better,
so this is sort of another feature that we will see.
The third trend might be to the integration of
bottles of information access.
So search, navigation, and recommendation or filtering might be
combined to form a full-fledged information management system.
And in the beginning of this course, we talked about push versus pull.
These are different modes of information access, but these modes can be combined.
And similarly, in the pull mode, querying and the browsing could also be combined.
And in fact we're doing that basically, today, is the [INAUDIBLE] search endings.
We are querying, sometimes browsing, clicking on links.
Sometimes we've got some information recommended.
Although most of the cases the information recommended is because of advertising.
But in the future, you can imagine seamlessly integrate the system with
multi-mode for information access, and that would be convenient for people.
Another trend is that we might see systems
that try to go beyond the searches to support the user tasks.
After all, the reason why people want to search is to solve a problem or
to make a decision or perform a task.
For example consumers might search for
opinions about products in order to purchase a product,
choose a good product by, so in this case it would be beneficial to
support the whole workflow of purchasing a product, or choosing a product.
In this era, after the common search engines already provide a good support.
For example, you can sometimes look at the reviews, and then if you want to buy it,
you can just click on the button to go the shopping site and directly get it done.
But it does not provide a, a good task support for many other tasks.
For example, for researchers,
you might want to find the realm in the literature or site of the literature.
And then, there's no, not much support for finishing a task such as writing a paper.
So, in general, I think, there are many opportunities in the wait.
So in the following few slides, I'll be talking a little bit more about some
specific ideas or thoughts that hopefully,
can help you in imagining new application possibilities.
Some of them might be already relevant to what you are currently working on.
In general, we can think about any intelligent system, especially intelligent
information system, as we specified by these these three nodes.
And so if we connect these three into a triangle,
then we'll able to specify an information system.
And I call this Data-User-Service Triangle.
So basically the three questions you ask would be who are you serving and
what kind of data are you are managing and what kind of service you provide.
Right there, this would help us basically specify in your system.
And there are many different ways to connect them depending on
how you connect them, you will have a different kind of systems.
So let me give you some examples.
On the top, you can see different kinds of users.
On the left side, you can see different types of data or information, and
on the bottom, you can see different service functions.
Now imagine you can connect all these in different ways.
So, for example, you can connect everyone with web pages, and
the support search and browsing, what do you get?
Well, that's web search, right?
What if we connect UIUC employees with organization documents or enterprise
documents to support the search and browsing, but that's enterprise search.
If you connect the scientist with literature information
to provide all kinds of service, including search, browsing, or
alert of new random documents or mining analyzing research trends,
or provide the task with support or decision support.
For example, we might be, might be able to provide a support for
automatically generating related work section for
a research paper, and this would be closer to task support.
Right? So then
we can imagine this would be a literature assistant.
If we connect the online shoppers with blog articles or product reviews
then we can help these people to improve shopping experience.
So we can provide, for example data mining capabilities to analyze the reviews,
to compare products, compare sentiment of products and to provide task support or
decision support to have them choose what product to buy.
Or we can connect customer service people with emails from the customers,
and, and we can imagine a system that can provide a analysis
of these emails to find that the major complaints of the customers.
We can imagine a system we could provide task support
by automatically generating a response to a customer email.
Maybe intelligently attach also a promotion message
if appropriate, if they detect that that's a positive message, not a complaint, and
then you might take this opportunity to attach some promotion information.
Whereas if it's a complaint, then you might be able to
automatically generate some generic response first and
tell the customer that he or she can expect a detailed response later, etc.
All of these are trying to help people to improve the productivity.
So this shows that the opportunities are really a lot.
It's just only restricted by our imagination.
So this picture shows the trend of the technology, and also,
it characterizes the, intelligent information system in three angles.
You can see in the center, there's a triangle that connects keyword queries
to search a bag of words representation.
That means the current search engines basically provides search support
to users and mostly model users based on keyword queries
and sees the data through bag of words representation.
So it's a very simple approximation of the actual information in the documents.
But that's what the current system does.
It connects these three nodes in such a simple way, or
it only provides a basic search function and doesn't really understand the user,
and it doesn't really understand that much information in the documents.
Now, I showed some trends to push each node toward a more advanced function.
So think about the user node here, right?
So we can go beyond the keyword queries, look at the user search history,
and then further model the user completely to understand the,
the user's task environment, task need context or other information.
Okay, so this is pushing for personalization and complete user model.
And this is a major direction in research in,
in order to build intelligent information systems.
On the document side, we can also see, we can
go beyond bag of words implementation to have entity relation representation.
This means we'll recognize people's names, their relations, locations, etc.
And this is already feasible with today's natural processing technique.
And Google is the reason the initiative on the knowledge graph.
If you haven't heard of it, it is a good step toward this direction.
And once we can get to that level without initiating robust manner at larger scale,
it can enable the search engine to provide a much better service.
In the future we would like to have
knowledge representation where we can add perhaps inference rules, and
then the search engine would become more intelligent.
So this calls for large-scale semantic analysis, and
perhaps this is more feasible for vertical search engines.
It's easier to make progress in the particular domain.
Now on the service side,
we see we need to go beyond the search of support information access in general.
So search is only one way to get access to information as well recommender
systems and push and pull so different ways to get access to random information.
But going beyond access,
we also need to help people digest the information once the information is found,
and this step has to do with analysis of information or data mining.
We have to find patterns or convert the text information into
real knowledge that can be used in application or
actionable knowledge that can be used for decision making.
And furthermore the knowledge will be used to help a user to
improve productivity in finishing a task, for example, a decision-making task.
Right, so this is a trend.
And, and, and so basically, in this dimension, we anticipate
in the future intelligent information systems will provide intelligent and
interactive task support.
Now I should also emphasize interactive here, because it's important to optimize
the combined intelligence of the users and the system.
So we, we can get some help from users in some natural way.
And we don't have to assume the system has to do everything when the human,
user, and the machine can collaborate in an intelligent way, an efficient way,
then the combined intelligence will be high and in general,
we can minimize the user's overall effort in solving problem.
So this is the big picture of future intelligent information systems,
and this hopefully can provide us with some insights about
how to make further innovations on top of what we handled today.
[MUSIC]
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
