Welcome to the course “Augmented Reality & Video Service Emerging Technologies.” The level of AR (Augmented Reality) and advanced video & multimedia technology included in a product is what determines the level of value and luxury. The objective of this course is to teach all important technologies that are used in state-of-the-art AR, Skype, and YouTube video and multimedia products and services. This includes the advanced video and real-time multimedia delivery mechanisms based on H.264/MPEG-4 AVC, MPEG-DASH, CDN, and mobile CDN. If you have knowledge of these core technologies, you can understand the operations that are used in every advanced video and multimedia system in the World. As the future World of business and products are driven to be more and more video and multimedia oriented, having knowledge of these core technologies will enable you to lead your company to become the true World leader in AR and video multimedia technology products, services, and business. Thus, I cordially welcome you into the beautiful and powerful World of advanced AR and video multimedia!
2.2 AR Feature Detection & Description Technology
Loading...
Из курса от партнера Yonsei University
AR (Augmented Reality) & Video Streaming Services Emerging Technologies
11 оценок
Yonsei University
11 оценок
Курс 4 из 4 — Specialization Emerging Technologies: From Smartphones to IoT to Big Data
Из урока
AR Technology
The second module “AR Technology” focuses on AR (Augmented Reality) technologies, operation workflow, and Cloud support technologies. First, the features of AR technological components, and the role of AR feature detection/description technology and the IPD (Interest Point Detection) process is introduced. Second the advantages of AR cloud cooperative computation and AR cloud offloading is covered. In addition, the types of AR feature extraction descriptor types, feature detector requirements, and influencing factors are covered.
Познакомьтесь с преподавателями
Jong-Moon ChungProfessor, School of Electrical & Electronic Engineering
Director, Communications & Networking Laboratory
[MUSIC]
Augmented reality, feature detection and description technology.
Now we're going into the details of how actually augmented reality is done.
In order to do something, in order to display information or
put an object in a certain location, on a certain entity that is in
your real world view, you first have to detect that object.
You first have to decide what you are going to choose in
the real world view to put text or a certain image or a character on top.
In order to do that, you need feature detection and description technology.
Augmented reality feature detection.
Characteristic primitives of an image are identified.
Identification is done by highlighting unique visual cues,
and various feature detection techniques exist.
The objective is to conduct efficient and
effective extraction of stable visual features.
What does this mean?
Well, if the computation processing task is
too heavy that you need to compute a lot.
Or it doesn't really compute it the right way,
such that you get errors in what you actually compute.
Then augmented reality cannot be used.
And due to these reasons, augmented reality was
not made possible in older times, in former devices.
And this is why augmented reality is now very capable.
Because augmented reality is at a level in which we can be efficient.
We can process it very quickly with low computation,
saving the battery consumption of your augmented reality device.
In addition, what you find is very reliable, very accurate,
and the information that you display on it is very effective.
Therefore, overall, the effectiveness and
the efficiency is there to enable augmented reality applications.
And that is why augmented reality is so popular now and
is going to become so much more popular in the near future.
Augmented reality feature detector requirements.
Robust against changing imaging conditions.
What does that mean?
Well, think of it this way.
This is not virtual reality, where the environment and
all the characters within the environment are make believe.
This is augmented reality.
Like on the glasses that you're using right now,
you get a view of the real world.
But then again, the probability that you're going to stay exactly fixed
at a location looking at an object is zero.
No, you are going to move around and look at that at various angles.
As you conduct your daily work,
as you conduct your daily pleasure of playing or relaxing.
In that case what that means is that your view of that certain object or
the environment is going to consistently change.
Because this is the real world and
everything is displayed on your smartphone or on your glasses or
head mounted display or other devices like your car windshield.
Then in that case the way an angle, the light as well as
the surrounding environment is going to consistently change.
So therefore, based on these changing imaging conditions, we need
the augmented reality feature detector technology to remain robust, reliable.
Such that even if you look at an object under a different lighting,
or a different environment, or at a different angle,
then still it will detect the same object the same way, and
provide your desired information in the right way.
It needs to satisfy the application's Quality of Service requirements.
This means that quality of service as in terms of device lifeline,
how long you can use it on a certain device.
Because if the device, no matter how good it is,
if you can only use the device for five minutes, then it won't sell.
In addition is the accuracy there?
Is the information displayed within the given amount of time?
Because people that are using their glasses are not going to stay looking at
the same object until the information pops up in an augmented reality fashion.
No, they're just going to go and
look at whatever they need to look at at that time.
The information, augmented information or the augmented object that is
going to be added on to the real world view has to be displayed immediately.
As in terms of how immediate, well,
down to the level of maybe 10 to 50 milliseconds.
It needs to be that fast or even faster.
Then you must be able to complete the entire process of augmented reality and
serve it in that way.
Before the user looks in a different position, and
a new view comes into their vision range.
Therefore, the device has to be very accurate, very fast,
with limited, very little power consumption.
These type of quality of service requirements must be satisfied for
that product that is going to be used in a specific way.
In terms of AR feature detection influencing factors, they include
the environment, of course, because the environment will consistently change.
Changes in viewpoint, even if it's in the same environment,
based on what angle I'm looking at you,
will give me a different view of the overall object that I'm looking at.
Image scale, in other words, when you look close and
you look far, then the object will be large and
it will be small based on looking at it through a magnifying range or
a macroscopic view that has an overall landscape.
The object that you're looking at will be extremely enlarged or
reduced, as well as the resolution of that object image will be varied significantly.
You need your augmented reality device to be able to work properly
at magnified scales as well as reduced, compact scales.
And that is where the image scale comes in that your device needs to be able to
support.
Resolution, your camera may have different resolutions.
The resolution may change based on the light setting.
As well as the resolution may change based on the amount of
energy that is left in your device.
The resolution may change based on the network connection.
If the network is becoming slow, the throughput is becoming reduced.
The data rate is becoming smaller.
And there's more and more queueing delay.
But you need to use the networking support to get augmented reality services.
Then the video resolution that intakes the image may be reduced.
That's the resolution issues.
Lighting is also another thing.
Is it bright or dark?
Because as in terms of dark, if it becomes really dark, then if
the device is equipped with infrared then you may go into infrared mode.
And the infrared camera that intakes the information, the image,
may not be at the same resolution that the camera for a bright day would be at.
In that case the resolution will drop as it goes into infrared
mode because the lighting is not supportive.
We will look at these type of things in our augmented
reality smartphone testing project, which is the last module of this lecture.
So hang in there, and we'll test a couple of things on some interesting apps.
The interest point detected in the interest point detection process,
the IPD process, can be used to obtain feature descriptors.
A circular or
square region centered at the detected interest points is basically it.
As you can see there are circles in this image and that image over there.
In addition, you can see that the size of the circles fit
the size of the regions based on their scales.
So there's big circles and small circles.
Circles are used in this figure as well as that figure as well.
However, the size of the regions, as well as using squares,
sometimes instead of circles you'll see these squares that
represent the object that is being identified.
Feature detection, image features with unique interest points are detected.
Feature descriptors characterized image features using a sampling region.
Exactly how is this done?
Well, we will talk about this in further detail in this lecture and
the following lectures.
So hang in there.
Sampling based on an image patch around detected interest
points is where the feature detection engine works most.
As in terms of methods, one method is to use spectra descriptors
that are generated by considering the local image region gradients.
Some of the techniques supporting this are SIFT and SURF,
which we will look in further details, of course.
A gradient, what is a gradient?
Because this is used for the local image
region processing in feature detection in method 1.
Well you can think of it like this,
a gradient is a multi-variable generalization of the derivative.
A derivative is a derivative of a single variable, normally a scalar.
A gradient is done on multiple variables and
therefore normally represented in terms of a vector.
So using gradients in method 1 is what will enable
us to generate the spectra descriptors.
Another method is method 2, which uses local binary features that
are identified using a simple point-pair pixel intensity comparison.
This here, as in terms of the point-pair pixel intensity comparison,
are based on local binary features.
And what we're trying to do is, instead of using the gradient,
which is the differentiation process,
these type of methods are much simpler and less computation burdening.
So therefore you can do them quicker and
the energy consumption of the device is reduced significantly.
That is the purpose of the these techniques.
And the algorithms that enable you to use this in feature detection are BRIEF,
ORB and BRISK, which of course we will study in further detail.
Feature detection and description schemes.
Schemes that do both feature detection and description are SIFT, SURF, ORB and BRISK.
BRIEF does feature description only.
And for a feature detector scheme there is FAST.
And we will study about these algorithms in detail.
AR scheme mix matching.
What do you mean by mix matching?
Well, feature detection results in a specific set of pixels at
specific scales identified as interest points.
Different feature description schemes can be applied to different interest points
to generate feature descriptions.
This is because some schemes have specific benefits that they
are extremely good at when they are processing in their certain way.
Such that you may want to mix and match a certain sequence to enable a certain
feature, while achieving a certain level of reliability, robustness and accuracy.
And at the same time being sufficiently fast and sufficiently low power consuming.
Application and
interest point characteristic based scheme selection is possible.
You will see some mixing and matching to create a certain algorithm.
Something like ORB, which I will soon explain in further details.
These are the references that I used and I recommend them to you.
Thank you.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2019 Все права защищены.
