Forgive the long blog entry. I need to talk a bit about the future of our technology for our partners and our customers. A lot of this is skinny that so far has been part of a skunk works project here. Those that are technically inclined and curious about current thinking in malware fighting, however, may find this subject of some interest.
It all started over a good dinner
On a chilly and blustery evening last January, Joe Wells, Eric Sites (our VP of R&D) and I sat outside overlooking the water at the Island Way Grill, a favorite local hangout. We were trying to recruit Joe from his position as Chief Scientist at Fortinet and the subject was along the lines of a re-invention of the anti-malware model.
In antivirus circles, Joe is a well known figure. The founder of the Wildlist, he’s spent his life writing antivirus engines, getting antivirus patents and working for Symantec, IBM Thomas Watson Labs and Trend (and in his spare time, doing a complete translation of the Bible into the Sahidic dialect of the Coptic language as well as writing science fiction).
The antispyware model: Broken
We have felt for some time that the traditional antispyware model has been fundamentally broken. Antispyware programs had started out originally as niche products, marketed by the likes of mavericks such as Patrick Kolla (SpyBot), Nicolas Stark (LavaSoft) and Bob Bales and Roger Thompson (PestPatrol), and they all relied upon a brute force method of removal.
This method revolved around analyzing the files, registry keys, processes and the like associated with a malware program and putting these values into a database along with a boatload of MD5 hashes (unique signatures generated for files). Then, this database was bolted on to a system scanner. Basically, your classic antispyware product was a giant database attached to a scanning engine.
In other words, antispyware products are basically big fat databases attached to big fat system cleaners.
Why did WebRoot and PC Tools do so well with their tools? Both came out of the system cleaning tools business (respectively, Window Washer and Registry Mechanic). These types of tools pound through a system, looking for files names, directories, registery keys and processes. WebRoot’s SpySweeper, based on the same Delphi code that was used in the company’s Window Washer, excelled at this brute force method of cleaning.
This model worked fine in the early days, and you could typically handle some pretty bad stuff with even SpyBot or Adaware. However, things got rough for the simple reason that spyware authors got really smart because the economics were so strong. The spyware programs got increasingly difficult to remove, such as the practice of using “resusucitators” — programs that would notice when you killed a file, and then recreate it (classic Direct Revenue tactic).
It got so bad that Merijin Bellekom, who had created CWShredder to kill CoolWebSearch, simply threw up his hands in frustration. As he said “I simply do not have the tools to remove the latest variants, they are too aggressive or complicated to allow automated removal by CWShredder.”
We had the typical example of a user trying to remove threats, and needing to use multiple antispyware programs, run in safe mode, beat on the machine, cry, pray, ask for help on forums, run HijackThis a few hundred times, and then maybe get the use of the PC back. Even Steve Ballmer, CEO of Microsoft, went through this hell. And I certainly did on several occasions as I helped others with their destroyed systems.
The model was (and is) flawed. While the major antispyware products have improved dramatically, they simply cannot deal effectively with all the different kinds of today’s threats. You have the problem of depth (how much work is required to remove an infestation) and breadth (the sheer number of infestations that may be found in the wild).
The antivirus model: A surgeon’s touch
Now, while spyware was evolving, antivirus vendors were playing catchup. Antivirus engines had been dealing with nasty stuff for years, and were quite capable of removing all kinds of evil malware like worms and trojans. However, antivirus engines are designed primarily for deftly removing a piece of a file from a file or removing a few files. Consider the Melissa virus, one of history’s most infamous nasties — it was a Visual Basic macro virus. Removal required removing one registry key and removing some VB code from Word’s default “Normal.dot” file.
A surgical approach, compared to antispyware’s demolition-team type of approach.
Contrast this surgical touch with one adware infestation that Ben Edelman documented a while back: 730 registry keys, 1,194 registry values, 461 files, and 43 file folders in one infestation! It was simply an epic amount of crap dumped on a machine. You didn’t need a surgeon. You needed a demolition team!
And the fact is that most of the AV companies simply took a long time to catch up. Why antivirus companies were so late in the game is a matter of speculation, but I believe it came down to the following reasons:
- A bewildering new type of malware that required system cleaning tools as opposed to surgical strikes. AV engines are designed for file-infecting viruses or removals of a few files — not the hundreds of thousands of unique threat types you find in spyware installations.
- Burdened by their own past experience. The AV guys tended to look at threats or targets through their past experience — in other words, they were looking for threats that looked like those they had encountered before. And, by and large, the newer commercial threats — especially adware — did not look like the threats the AV guys were used to dealing with. As a result, some may have been naive about installation practices (especially run-of-the-mill deception and social engineering, as opposed to the classic viruses), and thus weren’t as aggressive in targeting adware programs. They were (and often still are) much too forgiving of unsavory business practices. Finally, they tended to target files and processes, not the complete suite of items (including registry keys) that needed to be removed.
- Worries about legal problems: Antivirus companies were faced with an even more bewildering problem: They were under the threat of legal attack from listing adware and spyware, something they had never really dealt with before.
The legal problem is interesting when you add in the geographical dynamics of the business. Now, this is all my speculation, but the major antivirus companies are in the US (McAfee and Symantec, and arguably Trend). They may be used to the US legal system (meaning, you can be sued for forgetting to supply toilet paper in the bathroom), but they are large companies, so are always nervous about legal problems.
The rest of the antivirus business is largely in Europe, and these companies are simply shocked by the US legal system. So you had an interesting intersection: Large companies not wanting to get sued, the smaller companies with a strong consumer voice being European and simply not interested in getting tied down with US legal issues (even some antispyware companies may have fallen for this legal fear — PC Tools delisted a number of threats like Hotbar and new.net based on legal threats it received).
And the real problem with AV products: Bloat
It’s a known problem that many antivirus products have become bloated and inefficient . The reason has a lot to do with the fact that the major antivirus companies need to support a broad range of viruses that may not even run on today’s platforms, because of useless certifications, support for older platforms, etc. But it’s part of why your AV product may take such a big hit on your system resources.
And with a user base that’s leveling off (even declining for some), the game now is recurring revenue. It’s all about subscriptions: Get the user in and get them on a subscription plan, even if it means billing on a “negative option”. Why invest in a market which isn’t growing in huge leaps and bounds, when you can milk the subscription revenue? It’s a cruel statement, but there’s enough truth in it by simple observation. Now Microsoft has raised concerns in enough AV companies to get them moving, but a lot of what we see is the same-old, same-old. More memory-hogging suites and more bloat. It’s a broken model, because no one ever decided to really fix it.
(By the way, I’m not maligning a whole industry here. There are a number of truly standout firms in the AV world that are doing a really good job. My comments are more related to the “usual suspects”.)
Today’s user has a problem: Security has become a menace to performance. It’s also gotten more confusing, blinding users with a blizzard of scary popups (although great improvements, as in Symantec’s handling of incoming threats, have been made in this area).
What we’ve been working on
So what’s our answer to all of this? Wipe the slate clean. Rethink the ideas behind desktop security. Create a new method that’s more efficient and more powerful.
A number of parts have had to come into play to make this happen. I had to hire Joe Wells and a number of other rocket scientists and invest a significant amount of the company’s financial and human resources. I also acquired technology, such as the Kerio firewall, which brought with it a number of innovative technologies such as Host Intrusion Prevention System (HIPS) and a Snort-based Intrusion Detection System (IDS). I’m also in the process of making an investment in some bleeding-edge rootkit technology.
Meanwhile, I’ve had to just be patient and let the team do their work, something not easy for me.
CounterSpy 2.0
CounterSpy 2.0 (currently in beta) is our answer to the problems of dealing with tough blended threats, and incorporates a number of new technologies, such as VIPRE and our FirstScan technology, to deal with the really tough threats. The premise behind CounterSpy 2.0 is:
- So-called “real-time” antispyware protection is not effective. If its not working at the kernel level, it’s not worth the time of day.
- Today’s antispyware technology must work at or below where the malware is executing.
- Equally important are detection, removal and the database definitions.
We believe this new product is a big evolution in antispyware detection and remediation.
There are a number of new features in CounterSpy from the previous version, such as the fact that it runs as a service, has a small CPU and memory footprint, has a new scanning engine, and uses incremental database updates. But I’m sure our marketing people will do a much better job of pulling all of those new features together when we officially launch the product (which will be at the RSA conference in early February).
VIPRE
One of the things we had to do was develop an entire antivirus technology from scratch, and we call it VIPRE. We don’t believe that going out and bolting on an antivirus engine is a good idea from a performance standpoint. The result of piling engine upon engine is ultimately crap, and users see right through it.
VIPRE is a completely new antivirus technology, which incorporates all the classical antivirus techniques (such as removing file-infecting viruses) as well as a number of new techniques. VIPRE is especially powerful in its heuristics capability, something you may have seen if you submit malware samples to VirusTotal. VIPRE is still not done and yet it’s catching an enormous amount of viruses based on its heuristics alone.
(A few notes about VIPRE for the technically-inclined: Since 99% of all malware is compressed (packed), you need to uncompress it in order to find the original entry point (that place where the malware executes) to analyze it. However, there are a large number of different compression methods and variations used. Many antivirus companies create a static unpacker for each different piece of malware, which means they have to hand-roll an unpacking algorithm for each different piece of malware — a time-consuming process. So one of the things we did with VIPRE was develop a “generic unpacker” to dynamically unpack any piece of malware.
But what happens when you actually unpack the malware? You have to analyze it in real-time, so we then had to build an extremely fast emulator which unpacks the malware, executes a few bytes, compares it to a signature and flags it if it’s malware. And while we were at it, we built a full debugging environment for our engineers to run malware in a secure environment and rapidly create new signatures. Furthermore, while much of the AV world may still be using regex expressions in their signatures, we’ve created a new model which improves considerably on the current state-of-the-art.)
VIPRE is also platform-agnostic, able to support Linux, Mac OS, Windows, and any other platform we decide on.
This was a lot of work.
But now VIPRE is basically done. What needs to happen is to get certified by the major certification bodies and to continue adding more viruses in order to roll it up into a full antvirus product. However, a major part of the VIPRE technology is actually shipping in CounterSpy 2.0, solely for the purpose of making CounterSpy 2.0 a more powerful antispyware product. We’ve taken the VIPRE “juice” and put it into CounterSpy, and I think you’ll really notice the difference when you’re dealing with spyware.
VIPRE is a brand new antivirus engine and incorporates the latest thinking in antimalware research. It’s burning-hot fast and extremely efficient.
Kernel-level active protection
Another key thing we had to do was develop a set of kernel-level drivers, designed to run from the start on 32 bit and 64 bit systems. This Active Protection sits at the kernel and sees all, stopping the bad stuff before it has a chance to execute on your system.
Our Active Protection is part of CounterSpy 2.0 but will, of course, be used for our antivirus product in the future.
FirstScan
One nifty feature of CounterSpy 2.0 is its FirstScan technology, which scans certain locations of the drive and removes malware prior to Windows launching. This is done directly to the drive, bypassing Windows APIs, right about the time that chkdsk would run. While other products may attempt removal prior to Windows loading, none that I know of actually scan and remove (correction: there’s one other I now know about).
The purpose is simple: To get the spyware before it has a chance to execute.
The end goal
In the end, you have an anti-malware model that is a hybrid technology, melding the “system cleaning” properties of an antispyware product, along with the efficiency of a powerful antivirus engine. This will first manifest itself in CounterSpy 2.0, which will have major parts of our VIPRE technology in it. Then a full antivirus and antispyware product will follow a few months later. And ultimately, this will all be integrated into an offering incorporating firewall, IDS, HIPs, and all the rest to make a very powerful, yet efficient anti-malware system.
Alex Eckelberry