So the talk surrounding the Krebs on Security post, The Long Tail of ColdFusion Fail, continues. Some have taken issue with that he seems to be singling out ColdFusion. Personally, I like the fact that he is reporting on breaches. It helps bring to light issues with installs that are out there, so others can learn from the problems. The ones he has highlighted are severe since they involve credit card processing.
The underlying problem is that hackers have identified ColdFusion as an easy target and are going after it. The only way to fix that is to make it more difficult to attack and compromise, but that requires involvement from everyone that touches a ColdFusion installation.
Now I have no involvement with eLightBulbs.com and these are my impressions based upon the information that is publicly available from the following sources:
- The Long Tail of ColdFusion Fail
- Paul McLellan's comment on the post
- Wil Genovese's post IIS Vulnerability Steals Payment Information
eLightBulbs.com had a third party firm perform annual PCI compliance penetration testing and apparently this firm did not find any ColdFusion 8 vulnerabilities with the installation. I find that a bit hard to believe. They should have had at least one finding stating that the CFIDE directory was publicly available. Another risk that should have been noted was that ColdFusion 8.0.x had reached end of core support on July 31, 2012 (Extended support for ColdFusion 8.0.x ends on July 31, 2014).
There are many tools out there that will identify ColdFusion issues. HackMyCF, by far, is the best ColdFusion specific tool to find configuration issues. Nessus currently has 40 plugins that deal with ColdFusion. Either tool most likely would have found issues. The specific issues found would depend upon the configuration and patches applied.
Without knowing the exact scope of the testing, tools used, and state of the ColdFusion install, it is hard to say how good of a test this firm actually performed. If I was using them, I'd really question them on how they do things because as it looks, they were just taking $6000 a year from eLightBulbs.com.
While it has been stated that the hosting provider that eLightBulbs.com used wasn't contractually obligated to patch and secure the VPS they were using, one would hope that the hosting provider would have been a bit more hands-on regarding security. Given that the eLightBulbs.com hackers were able to inject a malicious IIS DLL, they had complete control of the VPS guest OS. Depending on virtualization software, it might have been possible for the hackers to break out of the guest OS that ran the VPS, take control of the host OS, and pivot to attack other servers the hosting provider runs.
It is really in the interest of the hosting provider to help customers maintain a secure environment.
eLightBulbs.com is a story where they thought they were doing everything right and it went terribly bad. They were handling credit cards and doing annual PCI compliance testing as they were required to. They had a service level agreement (SLA) with the hosting provider and thought the hosting provider was providing support to patch and secure their VPS through a managed service contract.
Unfortunately, they were too hands off regarding security and seems like they did not completely understand the scope of services or assumed some services were included in the various agreements they had with the security firm and hosting provider.
The other area where eLightBulbs.com was at fault was not staying on a supported version of ColdFusion. While just upgrading would have put them on a supported version, the underlying problems of properly securing the environment probably would have still existed.
The positive from this is that eLightBulbs.com has been very forth coming with details of their experience and the steps they have taken to remedy their security issues.
Some will probably not like what I have to say in regards to Adobe, but Adobe has not done enough to help the situation. Unlike the first three sections, I can speak here from direct experience.
Adobe has not learned from the mistakes Microsoft made with early versions of IIS, to install the product with defaults that make it secure from the start. Do not rely on the sysadmins to come back around to secure it. This is precisely the problem with the published lockdown guides. There is an expectation that whoever is installing ColdFusion knows the lockdown guides exist and follows them properly. It also does not help that they have been published months after the release of the product. That at least will be rectified with ColdFusion 11 (hopefully).
There were improvements regarding security in ColdFusion 10, but as the past year has shown they are not enough. Adobe touted that they put ColdFusion 10 through Veracode, AppScan, and fuzzing (slide 8) to find security issues. Given the ColdFusion 10 vulnerabilities found so far, the code review process for security issues should be improved.
Another security feature in ColdFusion 10 that has been pushed is Secure Profile. While it does improve the security settings within ColdFusion, it is an OPT-IN option in the installer (appears this will be fixed in ColdFusion 11, Bug #3529334) but does not protect the CFIDE directory as well as if one follows the lockdown guide. At NCDevCon last year I showed how ColdFusion 10 with Update 9 applied and Secure Profile turned on but not locked down as per the lockdown guide could be exploited using the Sub-Zero v2 script.
Patching ColdFusion 8.0.x and 9.0.x is an absolute pain, which is how Unofficial Updater 2 (UU2) came about. While UU2 is not perfect, it does make it a hell of a lot easier for older versions.
Patching has gotten better in ColdFusion 10 with the HotFix Installer but there are still issues. Almost all of the issues regarding patching are about the quality of the patches issued. The track record has not been good.
As I showed in my previous post, How patching ColdFusion 8.0.x made you more vulnerable in some cases (or fun with CVE-2013-0632 from APSB13-03), Adobe caused CVE-2013-0632 with a security hotfix and was not dealt with for nearly 5 years until it became widely exploited. Adobe had to pull Update 3 for ColdFusion 10 completely. They have had to re-issue other updates due to errors. In the case of APSB13-10 [Update 9] they broke functionality that was not related (Bug #3540876) and people had to wait over a month until APSB13-13 [Update 10] to get it resolved. These patch quality issues directly impact the confidence people have in applying them and leaves more installations vulnerable because of the slower adoption rate of the patches.
The communication of security hotfixes and updates for ColdFusion could also be improved. ColdFusion 10 will generate a notification regarding updates, but only if it is configured to do so. What is needed is a notification service that is more like the Adobe Security Notification Service which notifies for all updates to ColdFusion not just security ones. Not everyone reads blogs or follows @ColdFusion on Twitter. Adobe should be contacting those with support agreements directly via email and telephone when any update to ColdFusion occurs, otherwise what are they really paying for?
Adobe needs to engage more with its customers regarding ColdFusion security. They need to ask their customers if they are having any problems and fix them or direct them to companies and people that can. [Marketing Plug: AboutWeb has experience in securing ColdFusion and the entire web stack. We have done this for numerous clients and if you need help, please contact us.]
Also while I don’t expect them to comment on all the ColdFusion breaches that occur, silence does not help.
While nothing in the eLightBulbs.com story directly talked about code developers wrote being exploited, developers have a role in security as well.
Unfortunately, too many ColdFusion applications are written poorly in regards to security. Part of that is due to the materials that are the de-facto standard for teaching ColdFusion, The ColdFusion Web Application Construction Kit (CFWACK). I respect all the authors of the series, but it is time for a complete re-write showing good, modern, and secure ColdFusion coding practices. The community itself tried to tackle this problem with Learn ColdFusion in a Week, for which I contributed the chapter on security.
The reality is developers need to know how to write secure code. If you are a developer and have no idea what SQL injection is or why cross-site scripting (XSS) can be extremely dangerous, among others, you need to improve your skills. There are many resource out there, but the first one I’d point you to is OWASP with the 2013 Top Ten and various cheat sheets.
The bottom line is that it takes everyone involved with a web application to ensure the security of it. A security breach can happen to anyone and does not matter the underlying technology used.
How patching ColdFusion 8.0.x made you more vulnerable in some cases (or fun with CVE-2013-0632 from APSB13-03)
On March 17th, there was yet another story from Krebs on Security, The Long Tail of ColdFusion Fail, which sparked some interesting comments. It also provided some additional insight to a post that Wil Genovese made on March 6th, IIS Vulnerability Steals Payment Information since he commented on Krebs that he dealt with eLightBulbs.com.
Before I knew the two were related, I made the comment that it is really a failure of the sysadmin to properly configure and lockdown ColdFusion based upon the published lockdown guides for ColdFusion 9 or ColdFusion 10.
Now eLightBulbs.com was running ColdFusion 8.0.x (yes, Adobe there are people that DO NOT upgrade that fast) and there was no lockdown guide published, right? Well not exactly. In October 2007, Adobe published the ColdFusion 8 developer security guidelines. In this guide, on page 30 it directly states that access to CFIDE/adminapi, CFIDE/administrator, CFIDE/componentutils, and CFIDE/wizards should be restricted.
Almost all the recent stories of ColdFusion compromises involve APSB13-03 which had a patch released in January 2013 for ColdFusion 9.0.x and 10. If you were running ColdFusion 8.0.x, there was no patch for you since it had reached end-of-life (core) support, but Adobe again offered guidance to secure the CFIDE directory.
This vulnerability is actually pretty easy to exploit and is also completely preventable without a patch if CFIDE is properly secured. But I started wondering when the vulnerability was introduced. A clean install of ColdFusion 8 or 8.0.1 is not vulnerable and will generate the following error:
This has been one of the faster turn around times to get an updated Unofficial Updater 2 out. One of the items that stuck out was that one of the acknowledgments was to Alex Holden who co-discovered the Adobe password and software breach.
In the news of the latest patch, there is contention of whether the hole reported by Alex Holden is actively being exploited, from the second to last paragraph.
Holden maintains this flaw was being used by attackers prior to today. "Hold Security identified an attack attempt against a ColdFusion version 8 resource by the same hackers behind breaches like LexisNexis, Adobe, and others," Holden said. Unaware of the possible effectiveness of this attack, Hold Security reached out to Adobe. While Adobe did not find the precise attack effective against any of supported CF versions, they did identify a critical flaw in the same resource which led to the patch issued today.
The first thing that struck me was that ColdFusion 8 was specifically mentioned. Now ColdFusion 8 core support ended July 31, 2012 which means Adobe will not issue a patch to close this hole on ColdFusion 8 (the last patch was APSB12-21 on September 11, 2012). The only effective measure is to make sure you properly lockdown ColdFusion 8 or upgrade to a supported version (and properly secure it). The other thing is Adobe's statement about being "effective against any supported CF version" and since ColdFusion 8 is no longer supported it is probably valid.
Regardless of if it is being exploited, the best defense is to be running a supported version of ColdFusion, staying current on patches, and properly securing ColdFusion using the published lockdown guides for ColdFusion 9 or ColdFusion 10.
Also tomorrow (11/14) at 2pm ET, I will be giving a free webinar on the security enhancements that have been made in ColdFusion 10. To register, please visit http://events.carahsoft.com/event-detail/2919/aboutweb/
Well, I kind of missed blogging the last update to Unofficial Updater 2 back in May while I was at cf.Objective(). The latest update APSB13-19 dropped while I was on vacation at the beach, but still got it done two days after it was released by Adobe.
For ColdFusion 9.0.x the latest security update only applies if you are running as a standalone install or as a JRun Multi-Server. If you are running ColdFusion 9.0.x on top of any other J2EE server like JBoss, Weblogic, or others you don't need to apply the fix.
So there has been yet another 0-day found that can exploit ColdFusion by not having directories within CFIDE properly secured as noted in APSA13-03 from Adobe. If you haven't properly secured CFIDE that is public facing, it is only a matter of time until it gets hacked. The previous two that were found in January and April of this year should have been motivation enough.
For those that are still running ColdFusion 8, my best advice to secure your ColdFusion install is to use the ColdFusion 9 Lockdown Guide. I have used it to secure ColdFusion 8 for several different clients. The only section in the lockdown guide that doesn't apply to ColdFusion 8 is "Removing WSRP servlet mapping" since it was introduced in ColdFusion 9.
If you are attending cf.Objective() this year, there are several more security related talks. I am talking on Web Hacking Tools on Thursday (1st day) at 2:35pm. Among the demonstrations will be how ridiculously easy it is to get access to ColdFusion Administrator if you have it accessible and not recently patched. I also highly recommend seeing both of Pete Freitag's sessions, Writing Secure CFML and Locking Down CF Servers.
Lastly, Adobe has announced a security patch to resolve APSA13-03 will be released on May 14th. Unofficial Updater 2 should be updated on May 15th while I'm heading to cf.Objective().