This course will show how one can treat the Internet as a source of data. We will scrape, parse, and read web data as well as access data using web APIs. We will work with HTML, XML, and JSON data formats in Python. This course will cover Chapters 11-13 of the textbook “Python for Everybody”. To succeed in this course, you should be familiar with the material covered in Chapters 1-10 of the textbook and the first two courses in this specialization. These topics include variables and expressions, conditional execution (loops, branching, and try/except), functions, Python data structures (strings, lists, dictionaries, and tuples), and manipulating files. This course covers Python 3.
Bonus: Leonard Kleinrock - The First Two Packets on the ARPANET
Loading...
From the course by University of Michigan
Using Python to Access Web Data
14536 оценки
From the lesson
Networks and Sockets (Chapter 12)
In this section we learn about the protocols that web browsers use to retrieve documents and web applications use to interact with Application Program Interfaces (APIs).
Meet the Instructors
Charles SeveranceAssociate Professor
School of Information
[MUSIC]
When you walk into the back entrance of Boelter Hall on the UCLA campus,
you may notice a seemingly random pattern of floor tiles in the entrance.
If you spend a little time looking at the pattern,
it might dawn on you that the tiles represent zeros and ones, and
then you might even figure out that the tiles represent ASCII characters.
The characters in the floor tiles spell out 'lo and behold' to commemorate
the building where L and O were the first two packets ever sent on the ARPANET
from UCLA to Stanford Research Institute on October 29th 1969.
[MUSIC]
>> ARPA wanted a network so that they could share the large computing resources
they had given to their researchers across the country.
University of Utah had a terrific graphics operating system.
SRI had database, we had simulation.
University of Illinois had high-performance computing.
And every time ARPA brought on a new researcher, they'd offer to buy him
a computer, fine,
but the researcher'd say I want the same capability all those other guys have.
I want the graphics, the database, and all the rest.
And ARPA said we can't afford that.
If you want to do graphics,
you log on to the machine at Utah through a network that we think we're gonna make.
So the need for the network was to do resource sharing and
not to protect the United States against a nuclear holocaust.
When Bob Taylor came in as the net director, and he recognized this need for
sharing resources.
By the way, notice the phrase I'm using.
Sharing resources is exactly what I built into the network design.
Now, they wanted to share the external resources.
Same idea.
You have it. You're not using it.
Somebody else should be able to.
So, they brought in Larry Robertson, another classmate of mine,
in fact an office mate of mine at MIT, to manage this project.
He came to me cuz he knew my work.
He watched me do the simulation.
in fact I used his compiler on the TX2 computer,
and said Len we need to know if this thing's going to work.
He knew that I had the theory so that I could show it to him, it's going to work.
In fact he even says he would never have decided to spend millions of dollars of
the US government's money
if he wasn't sure this thing would work.
So the design began to be laid out by a few of us in 1967.
In '68, they sent out a request for a proposal.
At the end of '68, Bolt, Beranek and Newman, a Cambridge Massachusetts firm,
won the contract to produce the first switch of the ARPANET.
And we became the network measurement center early on so we could test it out.
During the design phase, some great people were there, throwing their ideas out.
Herb Baskin was there, a time-sharing expert.
And he said, if this network can't deliver short messages within a half a second,
I can't use it for time sharing.
Specification, half a second.
By the way we got 200 milliseconds.
>> [LAUGH] >> It came along.
And Wes Clark said, set up the computer for communications.
I said, look, if this is gonna be an experiment, and
I was also interested in the research and the experimentation,
we have to build software in so we can run experiments.
Artificial traffic generators, measurement hooks,
a place where the measurements can be evaluated. Put that software in.
So Howie Frank began to talk about network reliability.
He said this network, if anything fails, the network shouldn't collapse.
So we didn't say there should be five nines of up time.
We went much more pragmatic.
We said if any single thing fails, everybody else can still talk.
So to do that you need something called a two-connector topology,
two independent paths between every pair of nodes. Built it in.
So all those specs went to BBN, they built the darn thing.
They delivered the switch here at UCLA
on schedule. Eight months after they got the contract,
they were to deliver this new technology, new applications, new device.
They did it on time, on budget, it came here,
we plugged it in, and bits began to move back and forth between our time-shared
machine and that switch on the day after Labor Day, September 2nd, 1969.
But that was just a one-node network.
The schedule was that another one of these switches would be delivered at
Stanford Research Institute 400 miles to the north.
And they would connect that to their machine and that happened in October.
So in October we had a two-node network.
My machine, my switch, another switch 400 miles away, and the SRI host.
And it wasn't one single line.
It was a gang of 4.8 kilobit per second lines.
So now what do you do?
You have a two-node network, well now you can do something.
So we decided one night late in October, programmer Charlie Klein and
I went into the room and said look let's communicate between these two machines.
So we got ahold of Bill Duvall, their programmer up there, and we said
let's simply log in from a terminal connected to our host to that machine.
The idea is these are both time-sharing systems.
They expect terminals to connect in and use the services of the machine.
The big thing was, sit at terminal here, log onto your machine here,
and through this wonderful network, log on here as if you're a local user.
Well, that's easy enough.
So, we got all set.
Got Charlie down at the terminal over here.
And just to be sure this worked, we had a telephone handset.
In fact, I actually think I've got the, here it is.
[LAUGH] I just happen to have it.
That was the telephone set.
>> That's the telephone.
>> That was the telephone.
We plugged it in.
We derived a- >> You weren't using Skype?
>> [LAUGH] Afraid not.
>> [LAUGH] >> Plugged it in.
We used a piece of the high-speed line for the phone connection.
But what the interesting thing is, we were using the defunct circuit switching
technology to prove out the new packet switching technology.
And it really helped us, so we could see what was going on.
So Charlie typed the L and said, did you get the L?
Bill said yep, got the L.
Typed the O, you get the O?
We're trying to do L-O-G for login.
You get the O? Got the O.
Typed the G. You get the G? Crash!
So the first message ever on the network was lo, as in lo and behold.
Now that's especially interesting because if you go outside this hallway
here down into the alley, and come in through another entrance to this building,
and I just discovered this about a week ago, you walk on a platform, and
there's a mosaic of tiles down there.
And they're a strange pattern.
It turns out, it's the ASCII code for lo and behold.
I have no idea who did that.
It's about a year and a half now, some very clever person put that in.
That was the first message, October 29th 1969 at 10:30 at night.
[SOUND] You're entering 1969 right now.
We reproduced this room to look as it did, and
smell and feel like it did some 40-odd years ago.
And if you look over here,
you're looking at the first piece of equipment ever on the Internet.
This is that first interface message processor, IMP number one at UCLA.
A Honeywell minicomputer, adapted by Bolt, Beranek, and Newman, BB&N,
to operate as a switch for channel-building functionality.
This is the same physical four square feet where it served as the opening
note of the Internet, the first piece of equipment ever on the Internet.
>> And that's the actual one.
>> That's it, I kept it for years. They tried to throw it away many times.
Most of the people who had IMPs have tossed them.
This is the one of two left in the world, but this is number one.
This is the first piece.
If you open this machine, you would be privileged to smell it.
It's got an unusual odor and just brings you right back.
The emotion, it's great.
You can't smell it through that.
This is a military hardened machine.
This machine was essentiality a state-of-the-art
minicomputer which was adapted by BB&N.
And I first saw it in 1968, at one of the joint computer conference shows.
Thousands of people in a big exhibit floor,
and you see these sky hooks up here?
They had one of these machines hanging from the ceiling,
swinging in the air, running.
And there was a guy, big guy, stripped to the waist,
oiled skin, with a sledgehammer and he was going whack,
whack, to show that >> It was military.
[LAUGH] >> And it was.
But the most important document of the Internet,
the most important document of the entire Internet is right here.
You talked about who was working with me.
Well one of my software programmers was Jon Postel.
In fact here's his picture.
And he was not a hippie.
Even though he appears to be.
He was the one who basically disciplined my staff to do things properly,
keep records.
He said, we have to keep a record of what's going on.
So beginning in October, basically a month after the IMP arrived,
we started keeping an IMP log.
You know, this is an engineer's log.
This is not a Madison Avenue piece of document.
Just scribbles.
We used an old SDS log.
And in here, we kept a record of what's going on.
And the most important entry happened to be right here.
On October 29th 1969 at 10:30 at night, Charlie Klein,
the programmer who was in the room with me, made this entry.
Talked to SRI host-to-host.
This is the only record of the very first message ever on the Internet, right here.
You know, we had the technology, we started making measurements.
We were the first experimental node.
So we saw things happening.
How come?
We have a 50-kilobit-per-second line.
The routing procedure either goes one way or the other, one at a time.
Now how can you get more than 50 kilobits per second between two nodes?
If there's only one path at any time?
And we said oh, it's obvious.
That path is on now, when it gets backed up, you change paths.
So this guy's emptying its packed backup while you're sending this way, then you can
get close to 100 kilobits per second.
So I think, but we could break it, as I said.
And every time we did we would call BB&N and say, fix it.
We did this.
Fix it. Cuz they wouldn't give us the code.
They kept it proprietary until ARPA said, we paid for that code.
You have to open it.
They did.
We saw it.
And it would take them six months to fix that thing.
We discovered a fault, this time we got the code.
We showed them how to fix it.
It still took six months.
One of the things I was very much interested in with design,
was distributed control.
Why?
I was a student of Shannon, and
Shannon's great work came when he had a lot of things interact.
Long code words, for example.
That's when these emerging properties arise.
So I said, I want to design large networks.
Because on a large network
you can't have a single point of control.
You have to distribute it.
So what does it mean to distribute control?
You're delegating authority to all the peers.
When ARPA started funding, the principal investigators,
they had the same philosophy.
They said look, you're a smart guy, here's some money, go do the thing you do best.
We're not gonna sit on top of you. Make good things happen.
So here we are, I'm a recipient of that kind of money, what do I do now?
I've got my graduate students. They're brilliant kids.
Look, we need a host-to-host protocol. Here.
I'm not gonna sit on top of you, I'm gonna run with it.
That is not a product mentality right?
That is a research and development and creative mentality, and
it worked so well.
[MUSIC]
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
