Introduction to HTML 5

Transcript:

Brad Neuberg: My name's Brad Neuberg and I'm with Google Developer Programs. Today I'm going to give an introduction to HTML 5. I want to encourage this talk to be interactive — if you have a question, if you have a rant, please raise your hand during the talk so we can answer questions or hear rants, if they're short.

Before I jump into HTML 5 proper, I want to set the context of where we are today, why the arrival of HTML 5 is exciting and interesting. I think it's no secret that the last two years have been very interesting ones on the web. Things are really accelerating. If you look at this graph, what you see at the top, the black line is the abilities of native platforms. Those have stayed roughly consistent the last few years. But over the last two years, the web platform has kind of woken up again; it's a renaissance. We see greater, and greater, and greater abilities in the web browser really making it possible to build web apps that have almost the same degree of fidelity as native apps. And really, it's not that hard to imagine that soon, those two lines will converge, and you can start building web apps that are even more powerful than what we have today.

Some of the new abilities that we'll touch on in HTML 5 are solving real developer challenges. One of the primary ones that I'll talk about is graphics — people want to get outside of the standard text that you can do on the web today, and do graphics. Location — it's a real challenge. People want to build location aware applications, for example, for social networking and maps. Storage. I like to say that we've got these unbelievable machines now and we're using them like they're a Commodore 64 in terms of the web; we don't really use their abilities. So when it comes to both storage and speed, in terms of JavaScript being much faster, let's take advantage of the machines that we have in front of us today.

In terms of one of the real trends of the last two years is we have more developers when it comes to open source and alternative browsers. Just the last two years, if you look at the number of check-ins in terms of Firefox, Chrome, WebKit, you see a huge, huge increase in the number of active people contributing to these open source browsers. Then you see a massive increase in the number of users. When Mozilla and Firefox first came out with an open source browser, I think it was an open question whether you could build a consumer open source application. I think at this point, the answer is yes. It's been a really amazing thing that now users really will use an open source browser; we're at about 30 per cent in terms of the combined alternative and modern browsers, as we like to say. That's a roughly linear trend and it's only increasing.

One of the other big changes in the last two years is that JavaScript has gotten much faster; about a 100 times increase in the performance of JavaScript. This is opening up building more advanced applications in the browser, and I'll show you later on some really cool demos of things like image recognition being done in JavaScript in the browser, and it actually working. This is really due to the fact that JavaScript has gotten much, much faster in the modern browsers.

All of these things come together. The fact is that we're in a different world today than we were in the late '90s when the last version of HTML came out. HTML 4 was roughly around 1997, and the world is different in terms of some of the things you saw earlier, so it really is time for a new version of HTML to really rev things.

I want to just touch on something: it's been really amazing what folks have been able to do the last few years just using the latent abilities of the browser. I like to ask: when were the basic technologies behind Ajax invented? I mean, the answer's up there.

[audience calls out responses]

Brad: It's actually a little earlier. '97.

Audience member 1: '95.

Brad: Did it come in '95? Well, it showed up...

Audience member 1: '95.

Brad: OK.

[laughter]

Brad: So what was the first application that used the XML HP request object?

Audience member 1: Outlook web base outlook.

Brad: Outlook web access, I heard two people. Right. That was the first app that was really pushing dynamic HTML, XML HP request in 1997. Unfortunately it took about eight years, until about 2005, for it to show up in enough browsers for developers to become aware of the fact that hey, this stuff is everywhere. Many people in 2004 were saying dynamic HTML is dead if you were a part of this, and suddenly people realize no, you can do really amazing stuff, and that renaissance kind of began in 2005. So this time around we have a chance to do things differently, we have to chance to consciously say: what do we want from the web? Let's rev it, and let's move forward.

The talk today is going to be broken into several different parts. HTML 5 has a lot of stuff in it, and we would be here the whole day if we went through all the different things. So I'm going to focus on a few pieces that are really interesting. We're going to talk about graphics in the browser; I'm going to talk about the canvas and something called SVG, or Scalable Vector Graphics. We're going to talk about video in HTML 5. I'll talk about the Geolocation API. I'll go into something called the AppCache and the database, and then something called Web Workers. For every one of these I'll give a basic introduction, demos, some sample source code, and I'll talk about where you can use this today; where can you actually deploy this?

Jumping in, let's start with graphics. Until recently, you really couldn't draw on the web every easily. You could use Flash and things like that to draw on the web, but they didn't integrate very well with the web stack so even if you did get some images on the page you couldn't make them very interactive with JavaScript, it was pretty basic. There were a variety of options that are present — you could use Vector Markup Language on Internet Explorer, Flash and Silverlight to do these things, but these are kind of trapped in a box, they don't integrate that well with the rest of the web technologies — HTML, the DOM, CSS. The nice thing about canvas and SVG, and I'll give you a basic introduction to both, is that they integrate as first class citizens with the rest of the web stack; they're not trapped in a box.

I'm going to talk about both, but I want to talk about SVG first. SVG is like HTML for graphics. It's been around for several years. One of the things that's happened in the last two years is it's suddenly showed up and it's on all the browsers except for Internet Explorer.

[laughter]

Brad: Not casting any aspersions there. It's finally at a state where people can start to do interesting things. It stands for Scalable Vector Graphics, and like I said, it's just like HTML — you have a form tag, you have a table tag, and then you have a circle tag, and a rectangle tag. Let's look at an example of this. To draw a rectangle you say 'rect', put it on the screen at some location, give it a width and a height, give it a fill and a stroke, change the outline. And I love this — this right here makes the corners rounded. Everyone always jumps through hoops to do rounded corners on the web, but that's how you do it, with SVG.

Audience member 1: RX, RY.

Brad: Yeah, so you could make different kinds of rounded corners. That just makes them the same.

This is the part that's interesting: we can set an ID so the JavaScript can interact with this just like you do with HTML, and a class, so you could centralize your style information using CSS. That right there is what is drawn from what you just saw. The reason I bring this up today is because one of the cool new features of HTML 5 is that it now allows you draw an SVG into normal HTML pages. Right here, the first thing you see is that doc type at the top — that is the doc type for HTML 5. When you drop that into one of your pages, that tells the browser this is an HTML page, and to use the HTML 5 parsing and abilities. Then, one of the new features of HTML 5 is SVG becomes just a part of HTML. There you see us dropping the rectangle right in the page, and you just surround it with the SVG tag.

[Audience member applauds]

Brad: [laughs] You're excited. Here's what's cool about this. SVG, years before this, you had to use XHTML, which… I don't like getting into big discussions about XHTML versus HTML, but let's just say it's not used on 99.9 per cent of the web that's out there. The nice thing about this is you can now draw it up into a normal HTML pages and just get graphics done.

Audience member 2: What's going to happen to previous versions that don't support this?

Brad: The question is, what's going to happen with all the previous versions that don't support this. What do you mean by previous versions?

Audience member 1: Or you could put namespace, so SVG then namespace, which renders as a separate thing.

Brad: The comment was you could put namespaces on the SVG. With HTML 5, you don't have to put the namespaces, but you can put them on there if you want to work with browsers that might exist before this parsing algorithm.

I mentioned that what makes canvas so cool, and what makes SVG so cool, is that it's a part of the web. Here we are grabbing the rectangle off the screen using normal JavaScript, just like we would with HTML. There we are changing the color. I mean, you don't have to learn significant new skills. There we are making it be interactive, so when you click it we say hello. One of the things that's unique about SVG — it might be hard on here, but — on the left you see a bitmapped image, and it's blurry. On the right is a vector image. So whether you're looking at something with SVG inside HTML on a 40-inch monitor or a small iPhone screen, it's going to look legible, it's going to look beautiful. That's one of the nice side effects of vector graphics.

Yes?

Audience member 3: Can you apply CSS styles to the objects?

Brad: The question is can you apply CSS styles to the objects. Yes, you can. There's something really cool about that. We did an experiment where we took one of these New York Times infographics — you know, you see an article, it says how many people think blah, and there's little graphs in there and text. That was a 50K JPEG. We turned it into SVG, made some CSS so you could centralize, make all my bar graphs look like this, all my styles look like that. What do you think the size of the SVG file was using CSS with gzip compression on?

Audience member 1: You mean the source file?

Brad: The source file that you would end up serving to someone.

Audience member 1: It's smaller, because you're reusing it.

Brad: One comment was it's smaller because you're reusing it. It's definitely smaller.

Audience member 4: 4K.

Brad: 4K? 1K. Actually, I think 1.5K. So about an order of magnitude reduction in size. Using CSS can help with that as well.

So, the Canvas API — I like to think of that as a scriptable image tag. Instead of always having to give the source of what you're looking at, you can draw your own image. You predominantly drive it with the JavaScript API. To use it, the first thing you do is you drop the canvas tag into your page, you give it an ID, and you set a width and a height. Then, in your JavaScript, you grab a handle onto your canvas and you get what's known as a context, and that gives you a drawing service. You see the 2D there — I always love that because that opens up the possibility of a 3D drawing service, which has begun to emerge. I think WebGL, which is already just arrived, one that builds on Firefox and Safari.

Audience member 5: I think Firefix 3.5 ships with it.

Brad: Firefox 3.5 ships with it? I think Firefox 3.6 has this experimentally. But you can say 3D and then it'll hand you back a drawing surface where you use something like OpenGL. But that's not part of HTML 5, that's something else. HTML 5 gives you a 2D drawing surface, then once you have that drawing surface you can start to do things. You set your fill style, your stroke style. Here we are changing how things are filled. Then you can re-use that. Here we are doing a rectangle; that's just saying XY with height. Here we are changing the style again, but adding some opacity, so that's an RGBA to make it 50 per cent opaque, and then we're drawing another rectangle. That is what that results in. It's a really straightforward API; it's really nice.

I'm going to touch on when you want to use one or the other, but I want to show you demos first.

This is the Firefox Mozilla download center. I really like this one, because it shows how to use both these technologies together to take advantage of their strengths. This page shows, in real time, people downloading Firefox all around the world. The page keeps a persistent connection open to the server, and shows a yellow dot wherever someone's downloading. Here's what I love about this: the map in the background is SVG. That makes a lot of sense — they probably grabbed the map from Wikipedia which has a huge repository of these, and they brought it into the page. The dots are drawn in real time with JavaScript using canvas. I like this. It's a nice integration of the two technologies.

This is a really creative use of the canvas tag. Everyone here has played Doom, or s first-person shooter. This is a first-person gifter.

[laughter]

Audience member 6: Aww.

Brad: This is a great use of the canvas tag. You might not be able to notice but I'm using the keyboard to do the navigation. This is a performance sensitive application, and I'm not using the mouse. Canvas is especially suited to games.

Audience member 1: That's a two and a half, right?

Brad: Yeah, they're faking that. They're faking this. They're probably using what's called 'draw image', and changing that based on the perspective and applying a transform.

I've got some special demos that I wanted to show. I went to a conference last weekend and Vincent Hardy showed some super cool demos that were using YUI 3, where he slightly changed it to work with SVG, and he took all of the YUI 3 animation functions and then applied it to SVG. I was really blown away by these, so I want to show you two of these — I got permission from him to show you all.

The first one grabs images from Flickr. All of those transforms are being done by the YUI 3 transform functions, but it's applying them to SVG elements instead of HTML. Then as we go over, this is taking pictures from the Flickr interesting-ness.

Audience member 1: The scaling is done with SVG? And the transformations?

Brad: The question is, is the scaling done with SVG? Yes. All the scaling is done with SVG.

Audience member 1: And which part is the canvas part?

Brad: The question is what's the canvas part? This particular one is just using SVG.

Audience member 1: Oh, OK. Wow, it's fast.

Brad: Everything you're seeing here works on Firefox, Safari, and Chrome. This is a nice use of SVG. I'll touch on these — it's mouse driven, you're doing scalable transforms of things, rotations. This is a good use case of SVG.

Audience member 7: What kind of CPU do you have?

Brad: The question is what kind of CPU do I have. That's a good question. This is a not a particularly new laptop, it's several years old. Intel.

Audience member 1: Core 2?

Brad: It probably is.

Here's one more. These are animated lyrics, and this is, again, all done with SVG, also using HTML 5 audio in the background. What you're going to see is a song, and you're going to see lyrics that are cued to the song, and all the lyrics are just plain text so they could be translated in real time, although we're not doing that here.

[demonstration plays]

Audience member 1: Is this using SMIL?

Brad: It is using SMIL. SMIL allows you to do animation…

Audience member 1: With a time sequence.

Brad: …with time sequences. And I actually have to check; I'm not sure if it's using JavaScript. I think this works on Firefox as well.

Here's another really, really cool use case of the canvas tag. This is called Bespin, from Mozilla. Here's the thing I love about this: the folks at Mozilla wanted to create a really world class code editor, but in the browser. They didn't want to have to deal with the text area element, so they built everything to just go through the canvas tag, and they built a full code editor directly in the canvas tag. It has collaborative editing and so on. I love this because it's forcing the canvas tag to do some really nifty stuff.

A natural question is, when canvas or SVG? SVG is higher level. When you draw something on it, you have a tree. When I draw a circle or a rectangle, the computer still knows: hey, that's a circle, that's a rectangle, and that's either something that helps me or something that hurts me, depending on what I'm trying to do. If I want to know what things are on the screen, and I want to have mouse interactions, for example, then that helps me. If want import and export I can use illustrator to import and export this. You could use Inkscape, which is an open source tool, to import and export. Then that makes sense.

Canvas is lower level. If I just want pixels, if I just want a scriptable image tag, keep that in your mind, then canvas is the right way to go. Generally you'll know — do I not need to know where everything on the page is? If I have 10,000 bouncing balls, and don't need the computer to know, hey, each of these is a bouncing ball, then canvas makes a lot of sense because then I won't have the overhead of keeping track of where everything is. So if you have a lot of animation, if you're JavaScript-centric, if you're more pixel oriented so you don't need things to dynamically scale based on the size, you can force each of these to do what the other does. I like to say, as computer developers, it's like a link list and a hash table — you can force one to do the other, but they're better suited for different tasks, so you choose the right tool for the job. You can force canvas to do what SVG does, but you'll end up recreating everything that SVG does, and vice versa. If you're building a side scrolling game you shouldn't be using SVG, you should be using the canvas tag. The nice thing is that HTML 5 gives you both of these options for different types of use cases.

Audience member 1: So generally canvas has lower resource usage?

Brad: It's roughly 20 per cent faster because it's not keeping track of where everything is on the page. If you end up recreating all the infrastructure to keep track of everything on the page, you'll probably slow it down by about 20 per cent.

Audience member 1: How about memory uses?

Brad: The question is about memory usage. By default, because you're not keeping track of everything, canvas would have lower memory usage. But if you started actually having objects… Again, it depends on your application.

You actually had your hand first — yes?

Audience member 8: Can you talk about computer presentation techniques in terms of accessibility?

Brad: The question was talking about this in terms of accessibility. As defined, SVG is more accessible, potentially. It's markup just like HTML, it has a title element, a description element. It has accessibility baked in. None of the screenreaders currently hook in to that accessibility, but when they do, if you create content, it will be accessible. Canvas is currently just a bunch of pixels, there's no current accessible mechanism, but there's a lot of discussion and brain-storming happening about how we can use canvas and make it accessible. At the end of the day, I think both are going to be roughly accessible, but the screenreaders need to catch up.

Audience member 1: I suppose anyone who likes DOM would like [xx]?

Brad: Yes. People have added JQuery bindings on top of SVG so it looks and feels just like JQuery, that's right. SVG is like HTML, it's markup, it's got a tree.

Audience member 1: It's easier to write an API.

Brad: Yeah, yeah.

So where are they supported? Both canvas and SVG are supported on all the modern browsers. The direct…

[ laughter as image on screen appears]

Brad: I know, I love that. It's like a little passive aggressive term. The direct SVG in the HTML syntax is still being finalized in HTML 5. Firefox 3.6 will have this; that's still coming down the pipe. But raw SVG support, as you saw, is in all the browsers, as is canvas.

We don't have a lot of time, but I want to touch briefly on how we get canvas to work in Internet Explorer, how we get SVG to work in IE. There's a really nice JavaScript library called XCanvas which can help you run the canvas tag on IE for certain use cases. I've also been working with other people in the open source community to do the same for SVG. We've got a library called SVG Web — it's currently in alpha — which gives the illusion of native support for SVG in Internet Explorer with no downloads by the user. It's just a JavaScript library. I've got a short video here, it's a one minute video, with house music of course.

There's going to be a download, and it works on almost 100 per cent of the browsers out there. Here we see native SVG in Firefox. Then we look at Internet Explorer — this is IE7 — and it works with the same fidelity.

Audience member 1: Is it using VML?

Brad: I'll touch on how it works in a second. It's not just static but it's scriptable SVG as well. We've also got basic video and audio support. This is IE again. What's called declarative animation, so you can make animation easily. We've got font. We've got that resolution independence so you can take advantage of the size of the monitor. That's a little basic tetris. The idea here is the library will use the native support when its present on all the other browsers, and then will do this magic on Internet Explorer. That's where you can grab the alpha currently, as a developer.

I like to say: do you want the red pill or the blue pill? The blue bill is that you, the developer, will feel like you've got native support, you can script it with JavaScript, you drop SVG right into your HTML page. The red pill is we use flash to do the rendering, because wherever IE is, we know that flash is going to be there about 97 per cent of the time. Then we use something else called Microsoft Behaviors. But the developer doesn't have to know this, it just looks and feels like there's native support, and the user doesn't have to download anything. The whole goal of this is we go from about 30 per cent SVG support to close to 100 per cent to help developers actually be able to create stuff.

This last weekend we just hosted the SVG Open Conference at Google to talk about some of this work, and we've been working with Wikipedia as well, who have a quarter of a million SVG images all under creative commons licenses which can be really cool when you're building apps, and we're working with them to get them displaying on IE using this stuff, and doing some cool stuff. Yes?

Audience member 9: What are the Google properties that use SVG and canvas?

Brad: SVG is used in Google Docs when you drop an illustration into the page. It's also used in the Google Visualization API for some of the APIs. The canvas tag is used…

Audience member 10: Doesn't Maps use it for polygons, polylines.

Brad: Google Maps actually has some complicated switching where, depending on the platform it's on, it will either use canvas to draw polygons…

[Audience member 10 makes inaudible comment]

Brad: Yeah. They didn't need the overhead of SVG for their application, they just did raw polygons using the canvas tag. And they have to use VML on Internet Explorer. There's a lot of developer tools and graphing tools. There's something called the Performance Understander which uses canvas to do all sorts of cool graphing in real time, so you can see performance aspects of your application. So good question.

Let's look at HTML 5 video. I like to say it's 2009, it's clear that multimedia is here to stay, it's clear that video and audio is here to stay- so again, this should be a base part of the platform by this time. One of the cool things about HTML 5 is… Imagine if in the late '90s you had to use an object tag to embed an image. Yeah, you laugh right? The same thing's true with video — video's becoming so common on the web, it's easy to produce, it's easy to host now, it's time there was a video tag to match the image tag. HTML 5 has a video tag, you just use the video tag, and you have something roughly similar to setting the source to where it is.

I'll show you how that works. The most simple example is to use the video tag, set the source to where your video file is, and then you can drop the attribute controls on there which will cause the browser to drop a little set of play controls for starting and stopping. On the inside you can drop fallback content that will get displayed if the browser doesn't support... You could fallback, for example, by dropping a flash object in there, for browsers that don't have the native support. The video tag really starts to shine when you're dealing with scripting, and I'll show you that.

One other thing I want to show you is: you might have multiple versions of the video, and unfortunately the situation on the web right now, the formats for video are probably going to converge on what's called Ogg and H.264. Generally you have to throw up two versions, and here's how you can point to both — there we point to the Og version, and then the H.264 MP4 version. The cool thing about the video tag is it has a whole set of ways to script it. Here we are grabbing it off the page and then playing it, but there are callbacks — you can get callback as the video plays, so you could synchronize and cue changing your user interface. Maybe this is a video showing an historical timeline on Wikipedia and you could change the UI to fade in and out different bits of information. So you can really break outside of the box that we had before.

I'll show you some demos. This is the basic player.

[video plays]

Brad: It's a promotional video. Now, here's the cool thing — we can break outside the box. That right there is just having CSS transforms applied to it.

This is a mock up of YouTube using the HTML 5 video tag. This video right here is being directly served to you without a plugin. And each of these little thumbnails right here is the HTML 5 video tag, and all they have is an onmouseover and onmouseout. You can imagine, the onmouseover just says play, and onmouseout says stop. I want to crack this page open, because the nice thing about this page is it's using a lot of HTML 5 features so it's a good example of HTML 5 in general. We see the HTML 5 doc type at the top. Then we see one of the new HTML 5 tags header, which just says: this is stuff at the top of my page, and there we drop our header in…

Audience member 10: Does that scroll independently, or does that position independently of the page?

Brad: The question is does that scroll independently of the page? Not by default, but you could imagine some CSS that says 'header position:fixed'. By default, some of these tags don't have any native browser behavior, they're just a better way to structurally say: here are the things that are actually on my page, which will make search engine's jobs easier, which will make reading data out of the page for sites mashing each other up easier. And it makes, frankly, authoring easier.

We've got the nav tag, which, as you imagine, basically is where you would put your navigation for the site. Then we have the article page, which kind of cues and says: here's where the real content is, everything else is just chrome, essentially, surrounding my article. Again, something that can help search engines, something that can help a browser, for example, when you zoom it would just zoom in to show the article so browsers could start taking advantage of these tags.

Articles are composed of sections. Here we are with the main content — that's the large video — and then there's where we see one of our video tags. That's the main one pointing over to the large video, and you see a property there that says auto buffer. Here's what auto buffer does — even before you press the play button, that tells the browser hey, go ahead and download this in the background and have it buffered, so when the user actually starts playing it it'll start instantaneously. That's what the auto buffer property does.

You see the progress tag, and this gives you a progress indicator. Right there is value, that's where you could set that through script, and the browser will style that for you. So you could imagine calling that with an interval as things get loaded. Here's a meter tag. Having five little stars, for example, that a user can grade — that's everywhere — that's what the meter tag does.

Audience member 1: Can you do a time sequence with the time tag?

Brad: The question is, can you do time sequence…

Audience member 1: I saw a time tag. Is it the static time?

Brad: Yeah, oops, I missed that one, thanks. The question is the time tag.

Audience member 1: What's the duration?

Brad: This is another new tag, and this just marks up when you have time in your document.

Audience member 1: So you can just use JavaScript to do that [xx] sequential?

Brad: Mm hmm. These are some of the other tags that come along — I'm not going deep into these. Dialog tag, you could surround that around comments, for example. At the very end, there is a footer tag, which is roughly similar to the header tag. Pretty straightforward.

Audience member 1: And super, super search engine friendly.

Brad: Yes, and also friendly for content management systems, for example.

Audience member 11: What advantage does the time tag have?

Brad: All of these tags were derived in an interesting way. Ian Hickson got a dump of the whole web and created a script that could determine what were the most used CSS class names. They were words like footer, header, time, article, section, and that showed… There's a theory called "pave the cowpaths" — and some developers will do this — where you make a neighborhood and you don't pave anything, and you see where the grass has been worn down. You come back a few months later and that's been relaid down the sidewalk. That was like a cowpat that said hey, people are actually marking up most of their pages with footers and headers, so we want to make the web easier to use for newcomers who arrive, and more straightforward, so let's solidify those into actual tags since they're already being used. They're going to make your CSS, for example, more readable, if I can just say 'style of footer' if something's inside the footer.

Yes?

Audience member 11: The thing with time, is it like VCAL where you have textual content, and then there's a machine readable representation?

Brad: The question is, is time like VCAL where there's a human readable and then a machine readable…

Audience member 11: Yeah, the content of the element is human readable, and you have an attribute which allows you to attach a machine readable component to that.

Brad: The comment is, essentially, both human readable and an attribute for a machine readable.

Audience member 11: Yeah.

Brad: The time tag itself has been in a fair degree of flux. I haven't looked at the latest representation. My understanding is it's generally to markup human readable, and you can provide an optional machine readable. Time gets complicated really fast, for anyone who's in internationalization. It's meant to hit the 80 per cent of time, and not turn into an ugly monster spec itself, which can happen quickly. So I don't know the exact state, but I do believe you can put a machine readable representation in there.

One more question on this, we've got to move along. Yes?

[Audience member 12 makes inaudible comment]

Brad: The question is, do the footer and header have any visual impact, essentially, on how they're rendered. They're essentially block level elements currently. Footers used to be treated a little differently, but that's been cleaned up in the spec based on feedback from visual designers. Right now they're kind of visual neutral, they're like DIVs essentially, and you would have to apply styling.

Let's jump on because we've got other areas. The video tag is in all the modern browsers. That one wrinkle is they support slightly different codecs, but that's essentially, at this point, two different codecs, so it's probably going to converge on both of those in terms of what folks have to support.

Geolocation — this one's really exciting. Once you have that on the web, and it's a secure way for users, there's all sorts of interesting stuff you can do. I'm waiting for someone to really integrate this deeply, in a good way, for social networking, obviously for games, for photos… I think people are going to do stuff with this that are unexpected and very surprising. The cool thing is that browsers are now location enabled; it's on the latest version of the iPhone, it's on Firefox 3.5. So it's in enough areas that you can start doing really interesting stuff.

I like this one because it's really straightforward to use. The navigator object has always been a global object. You get a new object called geolocation, and you have a method called 'getCurrentPosition'. That method takes a JavaScript callback that will get called with a position object when the position has been determined. Browsers can do it in different ways, but many times they'll kind of ask: do I have GPS? Use that. Do I have WiFi, or cell phone tower triangulation, or reverse lookup? I'll use that. But the developer doesn't have to know those details, they just get a location and accuracy.

They get back this position object which has a lot of stuff on it, actually, you can go pretty deep. But at its basics, you can grab a latitude and a longitude. Then, for example, you could call some method that you've defined called 'show location', which would draw that on a map. You could get, like I said, the accuracy — so maybe draw a circle around that, to show 'here's how sure we are about your position'. That's a really straightforward API.

Here's a demo of Google Maps using this. Again, this is just a prototype. The first thing you see is a gold bar that says hey, this page wants to know my location — I can make it permanent but we'll make it one time — and then it figured out my location and mapped it on the Google Maps using the Google Maps API.

Audience member 13: Did it call Geolocation?

Brad: It called Geolocation, got back that position object.

Audience member 10: From what, in this case? GPS?

Brad: That's a really good question. Not GPS, I don't have that on my computer. Obviously not cell phone ID. Using a WiFi database that Google makes available for free, that Firefox taps into, which can reverse an IP address into a rough location.

[Audience member 1 makes inaudible comment]

Brad: That's right. At that point, the last callback you have is IP reverse address lookup, which is very rough grained. Good question.

Audience member 1: This thing's pretty good.

Brad: Yeah, it can be surprisingly good or wildly bad.

[laughter]

Brad: It luckily has never blown up for me on a demo. This is supported on all the modern browsers. It is not yet supported on the desktop version of Safari, only on the iPhone version.

Alright, let's look at the app cache and the database. Both of these work together to let you build offline web applications. If we want web apps to really be able to be the place that we do work, they need to be able to work on airplanes, when the internet connectivity is sluggish or slow, when you go to a conference and suddenly the WiFi drops out for several hours but there's still a presentation you need to give. A lot of people say why do I ever need offline? You need it when you need it. You will sometimes have troubles with the network, and that's when you don't want to be locked away from your data and your apps. The database and the app cache, these work locally to let you build offline web apps.

I'm going to show you a demo, first, of these working together. This is a sticky note demo. I can put sticky notes in here, and then if I quit Safari and then come back, they're still there. I like this, this little swoosh — that is called a CSS animation. It's a nice little effect. So what's going on here? If you want to build an offline web application, you need two things — you need to be able to have your user interface when you're offline, your JavaScript, your HTML, your CSS, and you need to be able to have your user data. The app cache is the first part. I like to think of this as a super browser cache. Why do you think we can't just depend on the browser cache? Why couldn't I just fiddle around with the browser cache to try to get my stuff to be there when the user goes offline?

Audience member 13: It's ultimately up to the browser.

Brad: Yeah. So you can't really depend on the browser — things can time out, they may not be in there, they may have gotten thrown away. Maybe the user never went to that part of the user interface. Maybe it's a dialog that needs to show up later. The app cache is a simple file where you list all of the resources that you want to have be offline. The first line has to be the magic words 'cache manifest', and you have to set your web server to serve it up with a special MIME type. Inside of there you list all of your resources, and then in the body tag in your HTML, you set the manifest attribute to where this file is. Once you do that, the browser will prompt the user to take these things offline, and then will download all these resources in the background, and will periodically go back to see if that file has changed, so you can update and version your application.

Audience member 11: Is there a size limit?

Brad: I think that the HTML 5 spec recommends… I know there's a recommendation. This number keeps changing. I think the last time I saw it there were 500K or 3 megabyte recommendation for the database. I can't remember if there's a limit, that's a good question.

Audience member 11: Can there be binary data?

Brad: Yes, there can be binary data.

Audience member 14: Will it load all the stuff when there is, for example, a slow connection?

Brad: You don't have a lot of control over when this is loaded. That just tells the browser here's what I need to be offline, and the browser will make these resources available offline. But on a slow connection it either grabs them all or it doesn't, so you don't end up in a halfway, inconsistent state.

Audience member 11: Do any of these resources need to be served with particular cache headers?

Brad: No. They don't need to be necessarily using HTTP caching, but that means that the browser will think they always change, if you don't. I don't want to get deeply into the algorithm, it's a little complicated, but what the browser does if it wants to know if anything's changed it first looks at the cache manifest and says: has this file changed at all? If the file has changed, it goes and revalidates every single one of the resources. That's so things are efficient. You can do comments for example — you could do a comment that says 'version=1', and if you just bump that up the file will change, and that will tell the browser to revalidate all of these. It'll revalidate them, and then if any of them have changed it'll re-download those.

Audience member 11: Is that any mechanism that makes sure that if you have a version of the file from an older version of the manifest, it won't be loaded into the application with a version of another file under the version to manifest? Is there any atomic switchover from one down the manifest to another?

Brad: So the question is, essentially, is there an atomic update? Yes, there is, because a lot of this was written with the feedback from the gears team who needed to do these kinds of offline things for Gmail, and needed atomic updates. We've got to move on.

The other half is the database. This is a full SQL database, generally using SQL 8 and the browsers that support it. Again, it's pretty straightforward. You open your database and give it a name, and a version for schema migration, if you ever need to do that. You give it a name that will show up. Some tools like Safari 4 let you browse all the local data; that's the name that will show up. Then you can give a suggested starting size. Right there we're asking for 200K, so the user will be asked if they want to give permission to store this. You want to ask all at once, so the user isn't constantly pestered. So we're saying we're probably going to use a total of 200K here, and just ask the user one time.

Now, let's talk a little bit about the method. You saw those post it notes — let's say we want to save these and make them persist in the database. Generally we would probably be saving an IED, some text, a timestamp, where they are in the page, and a Z index, because you saw they were stacked. Let's see how you would save that. The first thing you do is, once you've got your database object, you ask for a transaction. Whenever you're working with the database, everything has to be inside of a transaction. The reason is the web is an incredibly unforgiving environment. As you'll see we'll be running SQL here, and if the user closes their browser, leaves the page, the browser crashes, you want to make sure things are still solid in your database. Whenever you touch the database it's all inside of a transaction, so things either work, or they're not applied.

The first thing you do is you say give me a transaction and you give it a JavaScript function, the transaction is handed to you with a callback, and then you start executing your SQL, and we use normal SQL. Here we are inserting into a table called WebKit Sticky Notes. There are our columns, which pretty much map to what we have up there. We give our values with the question marks, and then those get filled in. I always like to ask, why do you think we use the question marks, instead of directly building up a giant string there and pasting them in?

Audience member 11: Injection attacks.

Brad: One answer is injection attacks. Injection attacks are not a concern on the client side, because you need to treat everything in the database as untrusted. When you replay this stuff from the database back to the server, you shouldn't be trusting that input just because it comes from the database, you should be rerunning it through whatever on your server side vets input, just like a user added to it in their form.

On the server side we do this for injection attacks, but on the client side the primary reason is what if one of the values has a single or double quote in it? Basically, that makes sure that things get encoded correctly. Because if you have a big string and a user enters a double quote, it'll break that. It's basically there as a convenience, because the code would be kind of unreadable if you built up a giant string.

Audience member 13: I could mimic that and send it manually without a browser to the server, if it's not server validated.

Brad: The comment is that you could do this manually if it wasn't server validated, is that correct?

Audience member 13: Yeah.

Audience member 11: You can craft an HTTP request.

Brad: Yeah. That would be outside of the standard.

Audience member 11: I understand that, but I disagree with the assertion that SQL injection isn't a problem. You can turn SQL injection into an XSS, for example, if something is taking JSON out and [xx] it.

Brad: That's a good point.

Audience member 11: The SQL injection, in that case, you're laundering.

Brad: That's true. A lot of people commonly ask, what's the security of this…

Audience member 1: It must always be validated on the server side, regardless.

Brad: Well, you need to protect yourself against cross-site scripting attacks in general. All of these use the same domain policy, so only the site that dealt with something can write to it. But if you pull in JavaScript from all over the web, that has access to the database, so you need to make sure you don't have cross-site scripting attacks. If you're getting data from a third party source and you're bringing it in, that would give you susceptibility to a cross-site scripting attack just like pulling in arbitrary JavaScript. That's a good point. You do need to be aware, but only because you need to be aware of cross-site scripting attacks in general, in web browsers. If you pull something from a cookie it's the same kind of issue so again, this doesn't introduce new insecurities, you just have to be really careful about the existing cross-site scripting attacks. That's a very good point.

HTML 5 database is supported everywhere, though unless this is changed it's currently supported on Opera mobile but not desktop.

Audience member 15: It'll be in the next version.

Brad: It'll be in the next version?

Audience member 15: Of the desktop, yep.

Brad: We only have a few more minutes so let's jump to Web Workers. Now that we have all these utilities, we have a lot more JavaScript running in the page, and with great power comes great responsibility. You can now really easily hose the browser. Most browsers are written so that the JavaScript runs in the same thread as the user interface, so if the JavaScript is doing a lot, the browser locks up, has a bad experience. Web Workers lets you make some JavaScript that runs away from the browser so that the browser can say responsive.

Audience member 16: Is it a threaded environment?

Brad: It's not quite threads, because threads are kind of complicated in JavaScript. It's message passing. I'll go through the API, but you basically give them a message, they run, and then they come back and give you a message.

Let me show you an example first. This is an example that finds prime numbers. When we press go, nothing happens; the browser completely locks up. This is, by the way, not using Web Workers, it's traditional. Nothing happens and I literally have to kill the browser. Now if we run it with the good one and press go, we see results coming in from the Web Worker and the browser stays responsive — well, it's supposed to stay responsive — and numbers continue to come, so I could continue browsing, for example, and so on. On machines that have multiple cores, it'll be using your other core.

Audience member 17: Nice.

Brad: Now let's put these pieces together. It's not too often that you want to find prime numbers, unless you're doing cryptography. This is a demo that combines a lot of the pieces that you've seen. You're going to see HTML 5 video playing, you're going to see a Web Worker that's running and using the fact that you can request every frame of your video and bring that in to the canvas tag. It'll get the raw data, it'll process over on a Web Worker who is moving on the screen, send the results back, and then use the canvas tag to draw where everyone is on the screen.

[Demonstration plays]

Those boxes are drawn dynamically, so JavaScript figured out where this person is, whenever she moves. Again, Web Workers are doing that with JavaScript in the background. Never would have thought you'd see video motion detection in JavaScript on the client side. I wouldn't have, I would have thought you were crazy if you'd told me that.

Let me quickly show you what the code looks like. You put all your code that will run on the worker in a separate JavaScript file, and you would instantiate it like this. Once you get your worker you give a message handler that will receive the results whenever the worker messages you. This method right here will get the results, and the data is held in the event 'objects.data'. Over on the worker side, we first have, let's say our algorithm, like finding prime numbers. We start it up and you get a magic method called post message — it actually lives in the window at 'window.postmessage'. And that's where you can, as you have each of your results, post them back, which will go back to the listener function. As each of those primes came in, we were just putting them out on the screen. You could imagine the results of analyzing the video.

People commonly ask what you can do on a worker. You can't touch the user interface, because then you'll hit the main thread. But you can do XMLHTTP request, you can touch the database, you can do quite a lot on the workers. Again, these are supported by all the modern browsers. They're currently on Opera mobile. They're in the next desktop version of Opera.

This brings us to the end. Do I have time for two more questions? Yeah? If you want to download everything you've seen today, there it is as a pdf. Does anyone have any questions?

Audience member 1: What about cross-site XHR?

Brad: Question was about cross-site XHR…

Audience member 1: Header permission base.

Brad: Header permission base? HTML 5 has a way to do messaging between domains that Internet Explorer 8 actually supports, and actually Facebook uses that.

Audience member 1: How about the other browsers? I think Firefox 3.5 uses it.

Brad: I think Firefox 3.5 supports it. I'm not sure if it's in Safari 4 yet.

Audience member 1: How about Chrome?

Brad: Chrome tracks Safari, so if it's in WebKit it's either there or not, I'm not sure. Let me get some questions from the back — did anyone have questions up the back? Yes?

Audience member 15: Some of your demos were in Safari, and others were in Firefox. Was there a reason why you did that?

Brad: I was cheating. The YouTube demo, for example, uses H.264 encoding, so that only works in Safari 4. But they could have written that to also serve up Og versions that would work on Firefox. Certain things just work better on Safari, it's a little faster for some demos. The Firefox 3.5 demo, that was mostly because of the codecs. Safari has the fastest… Actually, Safari and Opera might be neck to neck, but Safari has the fastest SVG support, so that's why I showed those demos in Safari.

Yes?

Audience member 16: Just a quick question: you talked about video tagging, but is there also audio tagging?

Brad: There is a HTML 5 audio tag that works similarly to the video tag. You can take a look at it. Two more question, yes?

Audience member 17: What about the considerations for file size, as far as video streaming? I didn't see anything up there.

Brad: The question is about considerations of file size. Currently, streaming generally requires server side support. At this point that's out of the HTML 5 spec, and the browser would just try to go and fetch that codec. Currently it just grabs the whole thing, but a particular codec could allow you to show some of the video before it's all downloaded. That's generally fake, it's faking streaming.

I believe some people have been talking about formal streaming, but that's a little outside of my domain. I've just been keeping a bit aware of it. But some folks have talked about formal streaming on the HTTP standard that's supported. Yes?

Audience member 18: Does that mean that there's no seek capability in the video tag? You can't suddenly say: stop circling from this portion instead of…

Brad: The question is about seek capability. That's a good question. The seek capability can be from streaming. You could also imagine doing a range request using HTTP to do seeking. I'm not specifically clear if Og… I'm sure that H.264 probably supports that. HTTP can say: get me just this byte range. Again, I haven't specifically looked at that. If you have an answer it'd be interesting to learn, if you can look it up.

Audience member 18: You kind of answered it by saying there's self-streaming, because nothing ever starts to truly play in the video. [?]

Brad: Yeah. And again, if you have what are called range offsets with HTTP, that would require the server side to support range offsets. But that is technically a part of the HTTP standard.

Audience member 18: For it to work with something like H.264, you have to hit on exactly the key frame or something.

Brad: Yeah, yeah. I know there's work being done in this area; I've seen it go by. But I don't know the specifics.

Audience member 19: Use browser bisection to try to arrange then bisect exactly where the key frame is. It's quite a slow process.

Brad: That's right. Key frames can fake this — you can set the number of key frames for a particular video to some granularity level that allows someone to jump around. I'm sure that Og supports key frames, I'd be surprised if it didn't.

One more question. Or not. Thanks everyone!

[applause]