Skip to content

Posts with keyword: threat models

Machine Learning Security: a new crop of technologies

Artificial Intelligence (AI), and Machine Learning (ML) specifically, are now at the stage in which we start caring about their security implications. Why now? Because that’s the point at which we usually start caring about the security considerations of new technologies we’ve started using. Looking at previous cases, such as of desktop computing, the Internet, car networks, and IoT (Internet of Things), those technologies first gained fast momentum by the urge to capitalize on their novel use-cases. They were deployed as fast as they could possibly be, by stakeholders rushing to secure their share of the emerging revenue pie. Once the systems started operating en masse, it was finally time to realize that where there is value – there is also malice, and every technology that processes an asset (valuable data that can be traded, the ability to display content to a user and grab her attention, potential for extortion money, etc.) will inevitably lure threat actors who demonstrate impressive creativity when attempting to divert or exploit those assets.

This flow of events is barely surprising, and we were not really shocked to learn that the Internet does not provide much security out of the box, that cars could be hacked remotely through their wireless interfaces, or that cheap home automation gear doesn’t bother to encrypt its traffic. This is economy, and unless there is an immediate public safety issue causing the regulator to intervene (often later than it should), we act upon security considerations only once the new technology is deployed, and the security risks are manifested in a way that they can no longer be ignored.

It happened with desktop computing in the 80’s, with the Internet in the 90’s, with car networks about a decade ago, and with mass IoT about half a decade ago. (In those approximate dates I am not referring to when the first security advocate indicated that there are threats, this usually happened right away if not before, but to when enough security awareness was built for the industry to commit resources towards mitigating some of those threats.) Finally, it’s now the turn of Machine Learning.

When we decide that a new technology “needs security” we look at the threats and see how we can address them. At this point, we usually divide into two camps:

  • Some players, such as those heavily invested in securing the new technology, and consultants keen on capitalizing on the new class of fear that the industry just brought on itself, assert that “this is something different”; everything we knew about security has to be re-learned, and all tools and methodologies that we’ve built no longer suffice. In short, the sky is falling and we’re for the rescue.

  • Older security folks will point at the similarities, concluding that it’s the same security, just with different assets, requirements, and constraints that need to be accounted for. IoT Security is the same security just with resource constrained devices, physical assets, long device lifetime, and harsh network conditions; car security is the same security with a different type of network, different latency requirements, and devastating kinetic effects in case of failure, and so forth.

I usually associate with the second camp. Each new area of security introduces a lot of engineering work, but the basic paradigms remain intact. It’s all about securing computer systems, just with different properties. Those different properties make tremendous differences, and call for different specializations, but the principles of security governance, and even the nature of the high-level objectives, are largely reusable.

With Machine Learning the situation is different. This is a new flavor of security that calls for a new crop of technologies and startups that deploy a different mindset towards solving a new set of security challenges; including challenges that are not at focus in other domains. The remainder of this post will delve into why ML Security is different (unlike the previous examples), and what our next steps could look like when investing in mitigation technologies.

Continue reading "Machine Learning Security: a new crop of technologies"

Product Security Governance: Why and How

The term “security governance” is not widely used in the product security context. When web-searching for a decent definition, among the first results is a definition by Gartner that addresses cyber security rather than product security. Other sources I looked at also focus on IT and cyber security.

But product security governance does exist in practice, and where it doesn’t – it often should. Companies that develop products that have security considerations do engage in some sort of product security activities: code reviews, pen-tests, etc.; just the “governance” part is often missing.

Product security is science; treat it as such.

This post describes what I think “security governance” means in the context of product security. It presents a simple definition, a discussion on why it is an insanely important part of product security, and a short list of what “security governance” should consist of in practice.

Continue reading "Product Security Governance: Why and How"

SDL and Agile

One of the challenges that agile development methodologies brought with them is some level of perceived incompatibility with security governance methodologies and SDLs. No matter how you used to integrate security assurance activities with the rest of your engineering efforts, it is likely that Agile messed it up. It almost feels as if agile engineering methodologies had as a primary design goal the disruption of security processes.

But we often want Agile, and we want security too, so the gap has to be bridged. To this end, we need to first understand where the source of the conflict really is, and this also requires understanding where it is not. Understanding the non-issues is important, because there are some elements of agile engineering that are sometimes considered to be contradicting security interests where they really are not; and we would like to focus our efforts where it matters.

We will start by highlighting a few minor issues that are easy to overcome, and then discuss the more fundamental change that may in some cases be required to marry security governance with Agile.

Continue reading "SDL and Agile"

Your Bitcoin wallet will never be your bank account

Don’t get me wrong; Bitcoin and crypto currencies are a big deal, at least technology-wise. Bitcoin and blockchains taught us a lot on what can be done with security protocols, and at a lower level, it even taught us that computation inefficiency is not always a bad word, but something that can yield benefits, if that inefficiency is properly orchestrated and exploited. It was also the most prevalent demonstration of scarcity being artificially created by technology alone. As I wrote before, blockchains will probably have some novel use-cases one day, and Bitcoin, aside of being a mechanism for transferring money, also provides a target of speculation, which in itself can be (and is) monetized.

What I truly do not understand are the advocates who see Bitcoin wallets as the near-future replacement for bank accounts, and Bitcoin replacing banks (and other financial institutions) in the near future. I understand the motivation, as those are dreams easy to fall for, but for crypto-currency wallets to replace financial institutions much more is needed, and for the sake of this discussion I will not even delve into the many technical difficulties.

Continue reading "Your Bitcoin wallet will never be your bank account"

The effect of cloud services on our intimacy with IT

Years ago, we did not trust cloud service providers, or we trusted them only when we had no choice. Then, consumers started using web-mail and other such services, and finally companies also moved into replacing their own IT with cloud applications. By now, we trust our service providers sufficiently, for the most part. We model our risks, we consider the benefits, and we usually decide that it’s worth it. But often enough, our trust in service providers still does not cause us the necessary warm and fuzzy feeling that is required for us to hand off all our data to the cloud and live a truly digital life. As it seems, thinking you are secure is one thing, and feeling you are sufficiently secure, even with your most critical data, is something else.

What do we do for now? – Use the cloud, but not for everything…

Continue reading "The effect of cloud services on our intimacy with IT"

Useful threat modelling

Do you know what all security documents have in common? — they all were at some time called “threat model”… A joke indeed, and not the funniest one, but here to make a point. There is no one approach to threat modelling, and not even a single definition of what a threat model really is. So what is it? It is most often considered to be a document that introduces the security needs of a system, using any one of dozens of possible approaches. Whatever the modelling approach is, the threat model really has just one strong requirement: it needs to be useful for whatever purpose it is made to serve. Let us try to describe what we often try to get from a threat model, and how to achieve it.

Continue reading "Useful threat modelling"

The Inevitable Collapse of the Certificate Model

Many had high expectations from the SSL/TLS certificate model. At least on paper it sounded promising and worthwhile. Keys are used to protect traffic; for this to be effective, keys shall be bound to business entities; for the binding to be trustworthy by the public, binding will be signed by Certification Authorities (CAs), which the public will recognize as authoritative. Once the trusted CA signs the binding between a business entity (represented by a domain name) and a key — every user can tell he is communicating securely with the correct entity.

In practice, it got all messed up. It is difficult to form authorization hierarchies on the global Internet, this is one thing. However, the model failed also due to the economics behind it.

Continue reading "The Inevitable Collapse of the Certificate Model"