I’m at Black Hat, hanging out at the Amazon booth with my colleagues, and quite amazed at the stir created by the coding question I put up on the board.

For those of you that haven’t had a chance to see the code, or need more time to figure out what’s wrong, I’m posting the code here. Sadly, no prizes for those of you who aren’t able to come to the booth, and I won’t be accepting posts commenting on the code until after the conference is over.

Consider it a theoretical exercise in code review – how would you handle this Java code arriving in front of you at a code review?

public static boolean isDifferent(
    final MyBean oldBean,
    final MyBean newBean)
{
      return
        oldBean != newBean &&
        oldBean != null &&
        !oldBean.equals(newBean);
}

Post your comments, and after the conference is over, I’ll open them up.

For those of you that think in C++ instead of Java (and there were many at the conference), here’s a version in that language:

public: static bool isDifferent( const someClass const * oldObject, const someClass const * newObject) { return oldObject != newObject && oldObject != NULL && !oldObject->equals(newObject); }

There are no ‘cheat’ issues – code not present in the sample is assumed to be perfect.

Some simple statements about Cross Site Scripting / XSS / HTML Injection (all terms for the same thing):

• Ignore the term “Cross Site Scripting” as a confusing anachronism. Think “HTML Injection” whenever you hear “Cross Site Scripting”, and you won’t go wrong.
• The problem of XSS can be summed up quite simply as:
  • XSS allows an outsider to re-write your vulnerable web page(s) before your end users see them.
    • That means any part of the web page can be re-written to do anything your web page could do to the end user
    • That means any attack on your users will be trusted as much as your users trust you
    • That means your users’ credentials, session cookies and other secret information is controlled by an attacker
  • XSS is hard to detect, and in some cases impossible for you, the server owner, to detect an attack on your users.
  • XSS is easy to prevent, but requires your developers to be aware of, and work to prevent, the problem.
  • The presence of XSS vulnerabilities may void your compliance with regulatory standards such as PCI, SOX, etc.
    • Allowing outsiders to rewrite your web page may mean that you cannot state that you adequately control access to the collection of financially significant data.
    • PCI requires you develop applications with regard to a recognised standard framework, and specifically points to OWASP. OWASP lists XSS and Injection flaws as two of the “top ten” vulnerabilities to prevent. This is as close as PCI gets to outright stating that XSS prevents compliance from being achievable.
• All XSS attacks have four components:
  • The Injection
    • Where the attacker provides bad data, either directly to the user, or through a ‘store and replay’ mechanism.
  • The Escape (from “text” / “data” to “code”)
    • The escape is optional – but that option is under the control of the page’s author
    • If the page author requires an escape, the attacker cannot attack without correctly escaping
  • The Attack
    • The attack is mandatory – an XSS attack requires an attack component. Duh.
    • However – where an escape is required by the web page, the attack must follow the escape.
    • There are numerous possible attacks – and more being developed daily. You cannot list all possible attack patterns.
  • The Cleanup
    • The cleanup is optional, at the choice of the attacker, and hides from the user the fact that they have been exploited
    • Many attacks are fast enough to use your users’ credentials that the cleanup is not required. By the time the user notices (often only a second or so), the attacker has stolen the account, made purchases or read off credit card information.
• Given this component approach, you as a web page author have limited options:
  • You can’t block the cleanup, because it’s too late at that point – the attack has occurred, and besides, the attacker may not be interested in the cleanup. The cleanup is really only useful in demonstrating that subtle attacks can take place, giving attackers months to clean out a credit card.
  • Blocking the attack is a leaky measure – not that it isn’t worth doing in some cases. For instance, browser protections against XSS often block the attack measure because they don’t control the web site, and can’t reasonably block anything else.
  • Blocking the escape is a guaranteed measure. If attacker-supplied code cannot be viewed by the browser as code, there is no possibility of attack.
  • Blocking the injection is also a guaranteed measure, with two caveats – when you block an injection, you have to make sure that the injection hasn’t already occurred, and that it cannot occur from any other sources.
    • This means that if you block injection of code into, say, a database of message forum posts,you have to scan the existing posts to ensure that the injection hasn’t already taken place.
      • Clean your database when you recognise an injection or attack pattern.
    • Any time there is a database that accumulates information for later display, there will be more than one source vying to put data into the database. If injection is possible in one source, injection is likely from other sources, too.
      • Funnel all sources through one injection filter.
    • Preventing an escape from text into code works whether or not all the injections are blocked.
      • Never rely on injection prevention alone.
• No matter how innovative attackers get, Cross Site Scripting attacks need not occur.
  • Learn how browsers differentiate between text and code.
  • Use that knowledge to ensure that text remains text, and is never seen as code.
  • Let the browser and your web server platform and libraries help you
    • Use an appropriate document object model function to set text values on fields.
    • Instead of composing HTML in a string to include user supplied text, use a <span> or a <div>, and set its innerText property.
  • Don’t sacrifice safety for speed.
    • Don’t build HTML “on the fly”, either while generating the page or displaying it.
      • InnerHTML is a bad thing to set.
  • Don’t try to “test out” XSS vulnerabilities.
    • You (or your testers) are not Ash Ketchum, and therefore cannot “catch ‘em all”.
  • Develop XSS vulnerabilities out of your application
    • Anything you didn’t specifically write as code, is attacker-supplied untrustworthy data.
    • Prevent injections as much as possible by validating that input is correctly formed.
      • Note that you can’t completely validate all input in all cases.
      • The obvious “bad example” is that of a bug reporting system into which security vulnerability reports are placed. If I have to report an issue with an XSS flaw, I’m going to be deliberately and respectfully pasting XSS attack code into your application. So in that case, you cannot possibly validate my information and remove “invalid” code without removing the valuable part of the text!
      • Validation at the Client is for Convenience only – it allows you to quickly say “that’s not right” and have the user correct it.
      • Only validation at the Server is for Security. Anything you ask the client browser to do on your behalf may be ignored.
    • If you have limited resources, focus on preventing the ability to escape from text into code.
      • Do this by encoding output so that each level of processing can accurately and uniformly distinguish between code and data.
        • This means you must know what each level of processing uses as encoding or escaping of character sequences.
        • This also means you must know how many layers of processing there are between you and the user.
• If you do ONE THING to prevent XSS on your site, output encoding is the one thing to do.
  • Defence in depth says “never do just one thing”. One thing will one day fail to be implemented properly.

As you can see from the IPv4 Exhaustion Counter to the left (snapshot taken 7/16/2010, 7:30 pm PDT), IPv4 addresses are dwindling.

OK, so that’s perhaps a rather simplistic description of what’s going on – these are a count of the IANA blocks that have not yet been handed out to other providers. Usually what happens is that the IANA hands blocks out to regional network block managers, and they hand them out, piece by piece, to the local providers, who hand them out one (or more for larger organisations) at a time to consumers.

So, even when all these blocks have been handed out (known as “X-Day”), there will still be addresses to be handed out to consumers – but it is a key indicator to follow in realising that IPv4 is a dead-end strategy, and something else needs to be investigated.

What, again?

Yes, we’ve had these calls to move to a new addressing format for years – back in 1993, when I first got on the Internet, there was a lot of discussion about “IPng – Internet Protocol, the Next Generation” (STNG was current then, you must understand).

Later, as IPv6 came along, NATs (Network Address Translators) were brought out as the saviour to IPv4. The idea was that we’d all use the same internal addresses as one another (so each company has their own local 10.* netblock behind the NAT), and a single external IP address for each NAT. To put it mildly, this is not a solution to the problem, it merely postponed it a little – if anything the fact that we have to use NATs is indicative that we have already run out of IPv4 addresses. Until you look into it, you really have no idea how much work we have already put into changing our applications to work with NATs in an effort to prop up IPv4, when we could have spent that time in adopting IPv6.

So we are closer to having to go to IPv6?

Sure – but hey, what's not to like about IPv6? Unless you're the developer of a piece of network software, it all just plain works.

Applications accessing file shares by name - can still access file shares over IPv6, without a line of change. Only if you're dealing specifically with the IP layer and IP addresses will you see a problem. It doesn't take a lot to turn an app from IPv4-only to IPv6-capable, and users will hardly notice a difference, if you expose names, rather than IP addresses. [It took me two hours to convert the underlying engine of WFTPD and WFTPD Pro to use IPv6. The user interface took/is taking me longer, because I'm not so good at UI, and haven't had the focused time I need. But it’s coming.]

You have to reconfigure your routers a little - they at least need to either act as DHCPv6 sources, or handle DHCPv6 traffic to/through a redirector. Ask your router manufacturer what they recommend. And, since you won’t be using a NAT as an accidental firewall, you’ll want to make sure your routers have real firewall functionality.

Technology leaders should be asking to beta test our ISPs' IPv6 support, and if your ISP isn't at least beta testing IPv6 support, get them to catch up, or move to one that does. Good gracious, even laggard Comcast is testing IPv6 for its customers!

Some things to look forward to with IPv6:

• Multicast supported natively - maybe Internet radio stations will pick up on this and make their live feeds take up less global bandwidth.
• IPsec supported natively - no excuse any more.
• Because IP addresses are longer and impossible to remember, names will become more prevalent. That’s a good thing, because it discourages you thinking of numeric IP addresses as secure, static or necessary to know.
• Every machine now becomes a "first class citizen" on the Internet. This means FTP works, H.264 and other protocols that require transmission of IP address in the protocol. [That makes IPsec easier and more efficient, too]
• No more NATs. [Except for the pesky IPv4 <-> IPv6 translation layers] No more kludges to deal with NATs.
• There are currently a number of services that are either only available on IPv6, or are available for free on IPv6. That will only grow as time goes on.

I’m going to give away a secret that I’ve successfully used at every interview I’ve had for a security position.

“Cross-Site Scripting” (XSS) is a remarkably poor term for the attack or vulnerability (code can be particularly vulnerable to a cross-site scripting attack) it describes. The correct term should be “HTML Injection”, because it succinctly explains the source of the problem, and should provide developers with pretty much all the information they need to recognise, avoid, prevent, and address the vulnerabilities that lead to these attacks being possible.

1. It doesn’t abbreviate well – the accepted abbreviation is XSS, because CSS was taken by “Cascading Style Sheets”
2. Nobody seems to be able to explain, definitively and without the possibility of being contradicted by someone else’s explanation, whether “Cross Site” means “intra site”, “inter site”, both or neither. Certainly there are examples of attacks which use one XSS-vulnerable page only to exploit a user.
3. As you will see from the remainder of this post, no actual script is required at all, either on the browser, or on the server. So, disabling script in the browser, or running a tool such as noscript that essentially does the same thing for you, is not a complete solution.
4. HTML Injection can include injecting ActiveX, Flash, Java, or JavaScript objects into an HTML page.

[Note that I am not suggesting that prior work into XSS protections is bad, just that it is not complete if it focuses on JavaScript / ECMAScript / whatever you want to call it.]

Failure to understand XSS has led to people assuming that they can protect themselves by simple measures – disabling JavaScript, filtering “<script>” tags, “BLOCKED SCRIPT” URLs, and so on. Such approaches have uniformly failed to work, in much the same way as other “black-list” methods of defeating attacks on security flaws – it pretty much dares the attacker to find a way to exploit the bug using something else.

Particularly galling is when I look at code whose developers had heard about XSS, had looked about for solutions, and had found a half-baked solution that made them feel better about their code, made the bug report go from “reproducible” to “cannot reproduce”, but left them open to an attacker with a little more ingenuity (or simply a more exhaustive source of sample attacks) than they had. It seems that developers often try, but are limited by the resources they find on the Intarweb – particularly blogs seem to provide poor solutions (ironic, I know, that I am complaining about this in my blog).

I know all this – give me something new.

Alright then, here’s something that appears to be new to many – a demonstration of Cross-Site Scripting without scripting.

We all know how XSS happens, right? A programmer wants to let the user put some information on the page. Let’s say he wants to warn the user that his password was entered incorrectly, or his logon session has expired. So, he needs to ask the user to enter username and password again, but probably wants to save time by putting the user’s name in place for the user, to save on typing.

Here’s what the form looks like – you’ve all seen it before:

The code for this form is simple, mostly to make the example easy. I’ll write it in Perl and Javascript – because the Perl version is exploitable everywhere, and because the Javascript version demonstrates how DOM-based XSS attacks work (and how browser strangeness can cause them to be flakey).

[Note: A DOM-based attack, for those that don’t know, is an attack that uses Javascript to modify the HTML page while it is at the browser’s site. These are difficult to detect, but generally result from the use of unsafe functionality such as the innerHTML call.]

Needles to say – don’t use these as examples of good code – these are EXAMPLES of VULNERABLE CODE. And lousy code at that.

Perl:

use CGI;
$query = new CGI;
print $query->h1("Session timed out."), $query->p("You have been logged out from the site - please enter your username and password to log back in."), $query->start_form(-action=>"happyPage.htm", -method=>"get"), "Username: <input name=\"username\" value=\"" + $query->param("username") + "\" />",$query->br(),
"Password: ", $query->input(-name=>"password",-type=>"password"),$query->br(), $query->submit(),
$query->end_form();

BLOCKED SCRIPT

<h1>Session timed out.</h1> <p >You have been logged out from the site - please enter your username and password to log back in.</p> <form id="safeForm" action="happyPage.htm" method="get"> Username: <input name="username" value="name placeholder" /><br /> Password: <input name="password" type="password" /><br /> <input type="submit" /> </form> <script type="text/javascript"> var queries; function getQueryString(key) { if (queries == undefined) { queries = {}; var s = window.location.href.replace(/[^?]+\?/, "").split("&"); for (var x in s) { var v = s[x].split("="); if (v.length == 2) queries[v[0]] = decodeURIComponent(v[1].replace(/\+/g, " ")); } } if (key in queries) return queries[key]; return ""; } var myField = document.getElementById("safeForm"); if (myField != null) { var un = getQueryString("username"); // Build the form with the username passed in. myField.innerHTML = myField.innerHTML.replace(/name placeholder/,un); } </script> Side Note – on DOM-based XSS and anchors

Now, why did I specifically include the “getQueryString” in my Javascript version above? I could have simply said “assume this function exists with ordinary behaviour”. Well, I chose that one (downloaded from, naturally, a blog advising how to do this properly) because it processes the entire href, anchor and all.

If we modify our URL by adding “#&” between the query and the variable “username”, it demonstrates one of the more frightening aspects of DOM-based attacks. Those of you who are aware of what an anchor does to a browser will already have figured it out, but here’s a quick explanation.

The “anchor” is considered to be everything after the “#” in a URL. Although it looks like it’s part of the query string, it’s not. Browsers don’t send the “#” or anything after it to the server when requesting a web page, so it’s not seen in a network trace, and it’s not seen in the server logs. This means that DOM-based attacks can hide all manner of nastiness in the anchor, and your scanners won’t pick it up at all.

Back to the scriptless XSS…

So, this page would normally get executed with a parameter, “username” which would be the username whose account we’re asking for credentials for – and certainly it works with http://localhost/XSSFile.htm?username=Fred@example.com :

The trouble is, it also works with the XSS attackers’ favourite test example, alert("XSS")%3b'>http://localhost/XSSFile?username="><script>alert("XSS")%3b</script> :

Now, I’ve seen developers who are given this demonstration that their page is subject to an XSS attack. What do they do? They block the attack. Note that this is not the same as removing or fixing the vulnerability. What these guys will do is block angle brackets, or the tag “<script>”. As a security guy, this makes me sigh with frustration, because we try to drill it into people’s heads, over and over and over again, that blacklisting just doesn’t work, and that “making the repro go away” is not equivalent to “fixing the problem demonstrated by the repro”.

The classic attacker’s response to this is to go to http://ha.ckers.org/xss.html for the XSS Cheat Sheet, and pull something interesting from there. Maybe use the list of events, say, to decide that you could set the ‘onfocus’ handler to execute your interesting code.

But no, let’s suppose by some miracle of engineering and voodoo the defender has managed to block all incoming scripts. Even so, we’re still vulnerable to XSS.

What happens if we try this link:

http://localhost/XSSFile?username="+type%3dhidden></form><form+action%3dbadpage+method%3dget><input+name=username+value%3d"Fred

[The “%3d” there is a hex value representing the “=” character so that the query-string parser doesn’t split our attack.]

OK, that’s kind of ugly – but it demonstrates that you can use an XSS vulnerability to inject any HTML – including a new <form> tag, with a different destination – “badpage” in our URL above, but it could be anywhere. And by hiding the attacked input field, we can engineer the user into thinking it’s just a display issue

With some piggery-jokery, we can get to this:

http://localhost/XSSFile?username="+style%3dborder:0px%3bwidth:0px/></form><form+method%3dget+action%3dsadpage.htm+style%3dposition:relative%3btop:-3.5ex><input+name%3dusername+value%3d"Fred@example.com"+style%3d"position:relative%3bleft:65px

Looks much better (and with more work, we could get it looking just right):

So, there you have a demonstration of scriptless cross-site scripting. XSS, or HTML Injection, as I’d prefer you think of it, can inject any tag(s) into your page – can essentially rewrite your page without your knowledge. If you want to explain the magnitude of XSS, it is simply this – an attacker has found a way to rewrite your web page, as if he was employed by you.

[Of course, if I hadn’t been trying to demonstrate that XSS is a misnomer, and prove that you can shove any old HTML into it, I would simply have used a piece of script, probably on the onmouseover event, to set the action of the form to post to my bad site. Fewer characters. Doing so is left as an exercise for the reader.]

Side discussion – why does the Javascript version work at all?

It doesn’t work in Internet Explorer, but in other browsers, it seems to work just fine. At first look, this would seem to suggest that “innerHTML” on a <form> tag is allowing the “</form">” in the XSS to escape out from the parent form. I can assure you that’s not the case, because if you could escape out, that would be a security flaw in the browsers’ implementation of innerHTML. So, what’s it doing, and how do you find out?

Quite some time ago, my wife was very sneaky. Oh, she’s sneaky again and again, but this is the piece of sneakiness that is appropriate for this post.

I logged on to woot.com one day, as I often do, and saw that there was a 30GB Zune for sale – refurbished, and quite a bit cheaper than most places had it for sale, but still more than I could plonk down without blinking.

I told my wife about it, and she told me that no, I was right, we couldn’t really afford it even at that price.

Then, months later, I found that my birthday present was a 30GB Zune – the very one from woot that she said we couldn’t afford.

Ever since then, I’ve been a strong fan of Zune and woot alike.

The other day, though, it dawned on me that I could use my Zune (now I have a Zune HD 32GB) to keep up with woot’s occasional “woot-off” events, where they proceed throughout the day to offer several deals. Unfortunately, I can’t actually buy anything from woot on the Zune.

I couldn’t figure this out for a while, and assumed that it was simply a lack of Flash support.

Sidebar: Why the Zune and iPhone Don’t Have Flash Support

It’s not immediately obvious that there’s a difference between the Zune having no Flash support, and the iPhone having no Flash support.

But there is – and it’s a little subtle.

The Zune doesn’t have Flash support because Adobe haven’t built it.

The iPod doesn’t have Flash support because Apple won’t let Adobe build it.

Back to the main story – why my Zune can’t woot!

I did a little experimenting, and it’s not that woot requires Flash.

I tried to logon directly to the account page at https://sslwww.woot.com/Member/YourAccount.aspx (peculiar that, the URL says “Your Account”, but it’s my account, not yours, that I see there. That’s why you shouldn’t use personal pronouns in folder names).

That failed with a cryptic error – “Can’t load the page you requested. OK”

No, it’s not actually OK that you can’t load the page, but thanks for telling me what the problem was.

Oh, that’s right, you didn’t, you just told me “failed”. Takes me right back to the days of “Error 4/10”.

The best I can reckon is that, since the Zune can visit other SSL sites, and other browsers have no problem with this SSL site, the Zune simply doesn’t have trust in the certificate chain.

That should be easy to fix, all I have to do on my PC, or on any number of web browsers, is to add the site’s root certificate from its certificate chain to my Trusted Root store.

Sadly, I can find no way to do this for my Zune. So, no woot.

Would this be a feature other people would want?

I think this would – for a start, it would mean that users could add web sites that were previously unavailable to them – including test web sites that they might be working on, which are supported by self-signed test certificates.

But more than that, adding a new root certificate to the trusted root certificate store on the Zune is a vital feature for another functionality that people have been begging for. Without adding a root certificate, it is often impossible to support WPA2 Enterprise wireless mode. So, the “add certificate to my Zune’s Trusted Root store” feature would be a step toward providing WPA2 Enterprise support.

How would that interface look on the Zune?

I’m not sure that the interface would have to be on the Zune itself – but perhaps the Zune could stock up failed certificate matches to pass to the Zune software, and then ask the operator of the Zune software at the next Sync, “do you want to trust these certificates to enable browsing to these sites?”

Similarly, for the WPA Enterprise mode, it could ask the Zune software user “do you want to connect to this WPA Enterprise network in future?”

The story at the Financial Times is that Google has quietly stopped allowing their internal users – developers, testers, etc – to use Windows operating systems. Allegedly this is because they can’t trust the operating system after the “Aurora” attack earlier this year, in which systems at Google (and other companies) were compromised to steal credentials, email and source code.

Others have already pointed out that this makes little sense – for various reasons:

• The attack was performed through a Trojan – that mostly means that human weaknesses were as much a part of the vulnerability as any technical issues. [I don’t see Google getting rid of people as a result]
• The attack used multiple points of entry, including PDF files and Internet Explorer bugs – with the IE flaws being reliably exploited (and a reliable exploit is needed if a Trojan is to be successful) only in IE6.
• The penetration through IE6 – the only operating system component under attack – was successful because the systems were not only running an outdated browser with outdated protection (IE6), but also because the users were running as unprotected administrators. [Again, I don’t see Google getting rid of those users, or requiring staff to not be Administrator on their own systems]

Me, I’d like to think that this is just a bogus story – all operating systems have flaws, and when you’re protecting against an attack that is targeted against your company, rather than scattershot against an operating system at random companies, the protection afforded by running a non-majority OS is pretty much wiped out. In addition, a managed installation of Windows (i.e. a domain) provides for far greater corporate control that can be used in instances of attack to tighten security settings, or to monitor more closely the configuration and activities of those systems. Other operating systems just don’t have that level of manageability. So it seems likely that this is just some bluster on the part of the “Anyone but Microsoft” crowd, than actual corporate policy.

Or it could just be that Google wants its employees to run the Google Chrome OS more, or perhaps even that Google wants to spread its bets across different platforms – all of these would be good reasons.

But the question in my title remains – if Google does stop trusting Windows, and stop using Windows, what does that mean for Windows users? It would mean that testing of Google’s sites and applications under Windows would be an afterthought, rather than a focus. Instead of Windows being present in some part at all stages of development, Windows would be another Quality Assurance step - “now that we’ve built it, does it work on Windows without too many problems?”

It’s totally up to Google as to whether they wish to make that happen – certainly, if they see the bulk of their user-base coming from systems other than Windows, it would make sense to focus on those. But if you’re a Windows user, I hope this story makes you anticipate this as a possible behaviour, and one that could leave you without access to the Google resources – apps, documents, storage, email – upon which you rely.

So, what can you do?

Plan your exit strategy. How will you migrate your data away, and what will be your alternate applications? Or will you switch operating systems to follow Google? How will you decide when the time has come to make that change?

Of course, you can extend this discussion – what is your plan, Apple users, for when you have to choose between Apple and Adobe? That one may come sooner than any Google / Windows split.

Neither.

OK, let me back up a little and explain.

There’s an argument I’ve been a part of at many places, with many people, as to which method of protection is best to guard against injection attacks.

What’s an injection attack?

Oh, no problem – an injection attack is something like “Cross-Site Scripting” (which I prefer to call “HTML Injection”) or “SQL Injection”.

Put simply, the attacker abuses your site’s or application’s need to collect data from users – instead of providing simple useful data, the attacker provides data that contains code along with some magical sequence or other, to allow it to execute as code, rather than merely be displayed as data.

As a simplistic example, imagine you’re writing a shopping list. On it, you write things like “Carrots”, “Potatoes”, “Steak”, etc. Then someone comes along and writes “P.S. Don’t forget to rob the bank”. Lists of items to buy at the grocery store don’t often contain instructions – but if you then give that list to a rather literally-minded, but stupid, assistant, there’s a chance the “P.S.” makes them drop out of “list of grocery” mode, and into “follow instruction” mode.

Sure, but who would be that dumb?

No person you’d send to the grocery store on their own, of course – most people have an in-built filter that prevents them from mindlessly doing stupid stuff.

Computers aren’t so fortunate, sadly – any such filter would have to be built in by the programmers. When it comes to the equivalent of reading the grocery list, the computer would have to be told specifically not to rob banks – or steal cars, murder the grocery clerks, steal from the register, or do anything else bad that might be on the list.

That’s clearly a non-starter – there are so many items in the “list of bad things you might be told to do”, it’s really much easier to start from the opposite end, and build a filter that accepts only those things that are known to be good. It can make life a little tricky, because your first run at the filter might not list all of the things that you might want to get at the grocery store – but it’s still the only way to be safe.

That’s a white-list, right?

Yes – unless you work at Microsoft, where the term is “allow list”. It’s coupled with the concept of “default deny”, a very basic security premise, which says that if you don’t know what to do, just don’t do anything. Refuse to process the grocery list, if you care to continue using the metaphor.

Input Validation

But let’s break out of the metaphor and go into technical details. What we have just described is “input validation”. Make sure that all the input that’s given you matches a restricted set of possible good inputs.

This is an easy win, and should be applied whenever possible, simply because it quickly rejects bad input in most cases. You’re asking for a quantity to purchase? Make sure it’s a positive integer – if anything other than a series of digits is given to you, refuse the value. For even more win, make sure there are no leading zeroes, and that it’s smaller than the largest expected order you can fulfill.

That’s great for such a limited case, and there are many more cases you can analyse to determine that they are indeed possible to limit.

But what happens if you need to accept a character that you know is bad, or that you don’t know is good?

How does that work with Input Validation?

Many places I’ve worked have seen me run into fanatical developers who heard about SQL Injection, and Input Validation, and determined that they needed to protect against it. Away with the word “drop” – that’s not allowed. Away with the single quote character – not allowed, either. Away with angle brackets and the word “script” in case we’re susceptible to Cross-Site Scripting (XSS) too. Probably thanks to this cartoon from xkcd.

[I’ve often thought it might be funny for there to be a follow-up cartoon, in which the school gets its revenge and assigns Bobby a GPA of 3.”>, knowing that every University web site will discard that as an attempted attack.]

Fine, but you go tell Mr O’Reilly that he can’t enter his name, or order any lemon drops; tell the French they can’t put anything in quotes (you did know that French quotation marks are double angle brackets, oui?); and as for the word “script”, I’m sure you can come up with your own examples. Start with this blog post, for instance, which uses the word “script” or “scripting” about a half-dozen times.

The pathological case of a complete inability to do input validation comes, ironically enough, in software designed to log security incidents. How are you supposed to report the exact string that triggers an XSS attack or SQL Injection, if you are afraid that the string itself will break the security incident reporting tool? [If you’ve read Douglas Hofstadter’s “Goedel, Escher, Bach, an Eternal Golden Braid”, you’ll recognise this as the record that breaks the player. If you’re under thirty you may well ask “what’s a record?”]

Quite a quandary. But you know there has to be something to solve it, and the key comes from what I said earlier – if you can’t distinguish between code and data, you may find yourself executing as code something that you should be processing or displaying or storing as data.

So you have to say “this thing that looks like code is really data”. And you have to say it unambiguously, because ambiguity means that you can’t tell between data and code.

Output Encoding

So, that’s where Output Encoding comes in.

Where Input Validation essentially says “I will only accept data that is obviously data”, Output Encoding says “I will only pass on data and code that can be told apart from each other”.

You already see the flaw, of course – to do this as the caller, I have to know how my recipient distinguishes between code and data. I have to have a contract of sorts – and developers often refer to a “code contract” – that tells me how I should tell my partner which pieces I’m sending him are code, and which are data.

There are several ways to do this, and they depend on the library / language / platform your code is running in, as much as they depend on the communication mechanism (protocol) with the partner.

Give us a simple example, then

A simple example is sending an email through SMTP – the Simple Mail Transfer Protocol. Assuming you’re the mail sender, once you’re connected and ready to go, you send the commands “RCPT TO <a@example.com>” and “MAIL FROM <b@example.com>” to distinguish who you’re sending mail to and from, followed by the command “DATA”.

After the DATA command, as you might expect, comes the data of the message, line by line. [This actually includes header information, but I’m ignoring that to keep this simple.]

And how does the DATA end? With a single line, containing simply a dot (full-stop, period, whatever you want to call it).

That’s great, except of course an attacker could send an email with a single dot on its own line, followed by some evil SMTP commands, and you, the mail sender, would essentially ask the mail server to execute those commands. Or someone could accidentally include a single dot on its own, and trigger some random command to execute.

Fortunately, the SMTP designers thought of that – when sending a line from an email that begins with a dot, you are required to add another dot before it. So, a dot on its own becomes two dots, two dots become three, etc, etc. And the server knows that this is a data line, and not a command to be executed.

That’s perhaps the simplest example of Output Encoding there is.

Other examples of Output Encoding would include calling HtmlEncode, or a similar function for your framework, on data which you know should not be executed as HTML. If the data is HTML-clean, the HtmlEncode function won’t touch it, but otherwise your simple call to that one function will prevent XSS attacks (aka HTML injection, remember).

Another example is when passing data to a SQL database – instead of concatenating code and data to make a string that you execute, all SQL libraries have parameterised queries, allowing you to pass data in a manner that will allow the SQL server to recognise it as data, rather than code. [There is a side issue here, in that the SQL code itself may build a command by concatenating code and data in a string – in that case, your SQL developers need a quick session with the clue-by-four.]

So, which one is better? Input Validation or Output Encoding?

Here we come back to the topic of this article.

Neither is better than the other. You have to do both.

You see, injection attacks aren’t an example of input validation failures, or an example of output encoding failures.

Injection attacks are caused by a failure to do throughput handling correctly.

Throughput handling requires that you do input validation and output encoding. Input validation is cheap, easy, understandable by all, and allows you to dismiss bad data immediately, before a server wastes its time on it. But it doesn’t catch everything, and it can’t be used in all cases. So, output encoding must be used, difficult though it may be, to ensure that the data which does seep through input validation looking like code doesn’t actually get passed to the next layer as code.

Both, then.

Yep. Good luck.

It’s Memorial Day weekend, so we’re doing a little relaxing.

What’s relaxing for me? Playing around with interesting random bits of code.

iFetch – suddenly slow

One piece of code that’s been interesting for a while, and very useful, is the iFetch program, which I use to download BBC Radio so that I can listen to it on my MP3 player on the bus. The iPlayer is nice, and all, but I can’t access it without an Internet connection, and the bus doesn’t have an Internet connection yet.

Lately, it seems like practically every show I’m fetching is coming down in WMA format. That wouldn’t generally be so bad, except that the WMA format streams at real-time using MPlayer, whereas the other formats stream as fast as the Internet can send them.

There’s a reason why I can only download WMAs now – the BBC recently made the choice to keep to only those formats that they feel they can adequately add DRM to. Which seriously limits your options as to what devices you can play it on. (Does that iPad do Flash? No. Does it do WMA? No idea.)

I haven’t yet figured out why there’s such a slowdown with WMAs in MPlayer, so if any readers have any ideas, please let me know.

Editing MP3 tags

Since I was unable to do much about the WMA slowness, I thought I’d look into what I can do about the MP3 files I’ve already collected – organising them by genre, broadcast date, etc.

So I looked into what I can do with MP3 tags.

Windows comes with the Windows Media Format SDK, which I thought I’d use. I’ve previously used it to set various values such as Title, Author, etc. Today’s game was to try and expand on that a little. One thing I wanted to look into was the use of various date fields. Date of recording, date of encoding – those seemed to be appropriate values.

The function to use is IWMHeaderInfo3::AddAttribute, but it just wouldn’t work for me. First I tried to use the “ID3/TDRC” tag. No dice. AddAttribute gives me error 0xc00d002b – that’s NS_E_INVALID_REQUEST. So I tried ID3/TDEN – again, c00d002b. That error is supposed to mean that I entered the wrong stream index – but I used stream zero, which is supposed to mean “this tag applies to the entire file”.

Perhaps the function doesn’t accept the ID3 tag names, and only accepts the WMA tag names, even though this is strictly an MP3 file that I’m working with.

No problem.

Next to try is WM/EncodingTime, which is supposed to translate to ID3/TDEN. No longer do I get NS_E_INVALID_REQUEST. No, this time I get 0xc00d0bd7 – NS_E_ATTRIBUTE_NOT_ALLOWED. Why not allowed? No idea. Perhaps the WMF SDK (WTF SDK, sometimes) thinks that the EncodingTime should only be set by the process that does the encoding? I kind of disagree with that, and clearly because I have so many files without the EncodingTime value set, it’s not the case that it gets set by the encoding tool. I tried various different settings – as a string, as a QWORD, even as binary, and didn’t really get anywhere.

Again, does anyone know how this should work?

A couple of other Zune notes

Finally, what’s new to complain about on the Zune?

Not much – I still really really like the device itself. A couple of minor issues that I am sure Microsoft could fix if they weren’t so busy getting rid of the people in charge of the Zune:

• Why does the Zune list Podcasts in a different order from the Zune software?
• Why can’t you make one match the other?
• For what I mean, try putting MP3 files on your system with these titles: “(A grand time)”, “A boatload”, “The ark”. The Zune displays them “The Ark”, “A boatload”, “(A grand time)” – the PC displays them as “(A grand time)”, “A boatload”, “The ark”.
• A few more apps wouldn’t go amiss.
• Seriously, next device – put Bluetooth in the box.
• Add some cards that read “Thank you for asking – it’s a Zune, and yes, it really is this good, and it really is from Microsoft. More details at zune.net”, so that I can hand them out.

If you’ve been reading this blog, you’ll have suspected for some time that I’ve been hankering after a Zune HD.

Now that I’ve changed jobs, and got my first pay cheque, my wife and I decided that it’s about time we each bought ourselves an item that we’ve wanted – she ordered a Kindle, which sadly took three weeks to arrive, and I ordered a Zune HD, which arrived a week after ordering it.

First, the good stuff.

As I remembered from borrowing my friend’s Zune HD last year, the thing is incredibly light. Same size display as the iPhone (or at least the pocket version, not the maxi version that doesn’t make phone calls even when AT&T is working), but considerably smaller and thinner and lighter.

The display is incredibly good-looking, bright and clear. I almost don’t dare put it up to maximum brightness for fear I will go blind. Perhaps I shouldn’t watch that kind of video anyway :)

Like pretty much every device of late, you have to convert your videos to H.264 in order to view it, and the Zune software will do this for most of your video content. Not MPG files, for some reason. However, there are several encoder tools available – I used encHD before I realised that I actually have the functionality built in to Expression Encoder.

The operation of the Zune HD is smooth and intuitive – sliding menus around on screen is simple and, as my friends who use iPhones tell me, far less of an effort than on the iPhone. “You don’t have to press so hard,” is the quote I heard from someone directly comparing the two. You can literally flick a long menu up or down to get through it more quickly. On a long list, if you want to go to the end from the top, you can slide the list down, and keep holding it down – the screen will shortly pop to the other end. Similarly, to go to the top of the list from the end, slide the list up to the top of the screen, and it’ll quickly pop to the top of the list.

In the Music list, you can make your way through your really long list by selecting one of the squares marked with a letter, at which point your view will change to that of a list of letters (and the “#” sign, for non-letter initials), which makes it easier to jump to a particular artist without scrolling.

A long-awaited feature (at least, for me) is the ability to delete an item on the device, without having to wait until you’re syncing with the app on the PC. Simply hold down your finger on the item to delete, and then touch “Delete” when the menu appears. Oh, and then you have to hit “Yes”, to indicate that you really do want to delete the item.

The new games are really cool, and my son has repeatedly worn down the batteries just playing PGR Ferrari Edition. I like the Labyrinth game, myself, which has you using the accelerometer to steer a ball through a maze avoiding holes and spikes.

It’s also handy that the Zune software accommodates syncing with two devices without getting them confused, so that I can move content across from my old player.

What features are still missing? Phone Bluetooth Vibrate, so that you can feel the little ball in Labyrinth roll into the holes.

OK, I don’t really want that, because it’d drain the batteries like crazy and make the device bigger and heavier. Do. Not. Want.

Er… that’s it. But there’s bad stuff too, right?

Sure there is.

Podcastliness

Like before, it really doesn’t take into account my use of audio.

I have a number of programmes I’ve recorded from the radio, each of which might be 30 minutes, some of which are up to three hours in length. I can’t put those in the Music folder, because if I do, there’s no way to listen to part of the show, then part of another show, and then come back to the show I first listened to, because each time you play a different item in Music, you get to start from the beginning.

So I’ve had to put these programmes into the Podcasts directory – and that requires that I set their “Genre” tag in the MP3 file to “Podcasts”. That’s not a huge issue, but there are some things you lose by doing this. First, of course, by setting the Genre tag, there’s no way to sort by Genre. Bummer.

Then there’s the insane issue that the image stored in the MP3 file is not displayed in the list, or when playing the “Podcast”. While it’s clear that images are supported in MP3 files in the Music folder, and images associated with Podcasts are supported, the two are not combined. Irritating.

There’s also no way to determine, from the PC, which podcasts you’ve listened to on the Zune, so there’s no good automated way to delete, or move, the Podcast files from the PC that have been listened to. You just have to do it manually. Not cool.

Deletystuff

While you can certainly delete items now, the support is not that wonderful.

From the Music menu, you can delete an album, but not a song, unless you go through to something other than the Albums list.

In the Pictures menu, you can delete an individual picture, but not a folder. Makes it rather difficult to remove your ‘personal’ pictures before handing the Zune off to a friend for a demo. And, let’s be honest, this thing’s so good you’re going to spend the first month of ownership handing it to people for a demo.

In Podcasts, you can delete groups of Podcasts as well as individual Podcasts.

In History, you can’t remove items from the History. Why would you want to?

Scrollocity

Scrolling from top to bottom is a little awkward – you have to very carefully find and grab the top of the page and slide it all the way to the bottom, not letting go until you get there. There’s no way to ‘throw’ the top down to the bottom and reach the bottom of the page.

Scrolling through Podcasts [remember? that’s where most of my good stuff lies?] is limited to up and down only. Unlike the Music folder, there’s no ability to scroll through the alphabet. The episodes in the Podcast list are still sorted in a manner I can’t completely fathom, but the primary sort key seems to be the date of the file on the hard drive of your synced PC.

In the Internet app (the browser), there’s no ability to scroll from the top to the bottom, nor can you quickly scroll massive distances – which you might want to do if you find yourself on a site like this blog, which doesn’t display well on the Zune.

Summing that all up. The good stuff far outweighs the bad stuff.

The Zune HD is a seriously wonderful device. Light and compact, it fulfils exactly the purposes I had in mind for it. Obviously there needs to be a phone-based version as well, but those are either here or on their way.

Look at the items in my “good stuff” list – they’re all ‘architectural’ components – basic concepts of the design of the app. Now look at the items in my “bad stuff” list. They’re all small winces that happen at the corners. “Fit and finish”, if you like, or a “simple matter of programming” to fix. I just hope that someone at Microsoft reads my blog and does something about it.

From the perspective of the Podcasts and syncing up and down, I’d be happy if Microsoft would just introduce an SDK that allows me to enumerate the files on the device, and whether or not they’ve been played.

It doesn’t have a fruit on the back.

I deliberately bought a Zune HD rather than an iAnything. If you read through my previous entries on the blog, you’ll find that Apple have repeatedly given me a poor computing experience that fails to jibe with the expectation that I’m given by the man in the black turtleneck and his dedicated followers. Yes, I’ve seen your Mac, and I’ve seen your iPhone, and it works far less the way I do than any other device I’ve used.

I have a phone that makes phone calls, connects to a Bluetooth earpiece, and receives text messages – and that’s currently all it does. I only charge it once a week. iPhone users that I know find themselves frustrated with the quality of Apple’s customer service, as well as the overall phone service they get from AT&T, and they charge their iPohne constantly. Not even nightly – whenever they are stationary for more than five minutes in a room with a power outlet. At least, that’s my impression.

I might consider getting a Windows Phone 7 when it does arrive, simply because I really like the Zune interface, but I suspect that I may simply remain more comfortable with a separate phone, and not buy the Phone 7, or zPhone, or whatever they wind up calling it (seriously – a dorky name like “Windows Phone 7”? The manufacturers are going to name theirs something cooler).

It needs apps.

OK, here’s where the iPhone / iPod wins out in some measure, except that Apple controls the apps you can put on your iPhone / iPod, and Microsoft lets you put any old crap on the Zune – apparently, it’s actually your Zune, not Microsoft’s. But there are relatively few titles out there, and most people will continue to get their apps from the Microsoft Zune Marketplace, rather than from third party shareware web sites.

Everyone says “oh, that’s cool”

Even the iPhone users I’ve shown the Zune too have noticed something about it that is cool. Most often, they say it’s light in weight, the screen is bright, you don’t have to poke the screen as hard to make it work, and it fits their hands better because it’s smaller.

That’s a takeaway Microsoft can be proud of.

But if they’d fix the little winces I note above, I’d be even happier. :)

Another story comes my way – from the Boston Globe, this time – talking about how we shouldn’t be bothering to change our passwords, because it doesn’t significantly improve the chances that you’ll stop some attacker from abusing your account.

I have to repeat something I’ve said before - you don't change passwords to prevent people from guessing them. If your passwords are that bad that they can be guessed, they will be guessed.

You change your passwords for the following reasons:

1. Because you gave your password to five people in the last year, and you can't remember who they were, or if they still work at the same company.
2. Because you need to know that you can change your password without killing half the applications you've worked on.

Number 2 is the big one for me.

For most ordinary users, number 1 is the most likely to have an impact, but in IT shops, it's not uncommon for IT pros and developers to have created a service or two bound specifically to their user account. Changing your password every 90 days allows you to remember that you created the service, and to either change its password manually, or (ideally) to move it over to a recognized and managed service account.

I've seen several - repeat, SEVERAL - incidents where a password was exposed, and the security team mandated immediate change, but the account owners refused, and fought all the way up the management chain. The reason they used for prolonging the fight was "we haven't changed the password in so many years that we have no idea what will die when we change the password".

I view password changes now as a cheap piece of business continuity, to ensure that when a disaster (password exposure) happens, you can quickly carry on with a new password, rather than having to stumble along for weeks with a password that you know has been stolen and exposed.

I think that alone justifies changing passwords on a regular basis.