Coinigy Security Update, Post Mortem Review
NOTICE: WE HAVE BEEN MADE AWARE OF PHISHING ATTACKS USING COMPROMISED E-MAILS. AS OF THIS WRITING, ALL USERS MUST FOLLOW THROUGH WITH A MANDATORY PASSWORD RESET UPON LOGGING IN. DO NOT USE COMMON PASSWORDS ACROSS SITES ENABLE 2FA IMMEDIATELY
NOTICE: TRADING HAS BEEN DISABLED FOR ALL ACCOUNTS AS OF THIS WRITING AS AN EXTRA SAFEGUARD AGAINST THE ATTACKS. THIS ALSO MEANS USERS WILL NOT RECEIVE TRADE NOTIFICATIONS UNTIL TRADING IS RE-ENABLED.
Please continue to follow this blog and our Twitter account (https://twitter.com/coinigy) for updates.
At approximately 9:10 pm Sunday, October 16, 2016, we received a mysterious password reset request from Stripe, shortly thereafter Bitpay, Twitter, Freshdesk and our Mailchimp accounts.
We immediately noticed something was afoot, and were able to regain access, but in that time it was determined that an outside attacker had gained access to these accounts and run a full export of user data.
In the case of Stripe, that includes: e-mail address, name, billing address, and last 4 digits.
In the case of Freshdesk, that data includes: e-mail address and ticket history.
Mailchimp: newsletter e-mail addresses
With regards to Bitpay, there is no apparent export option however the attacker may have been able to screenshot past invoices.
Unfortunately, the accounts in question were compromised due to a failure to implement two-factor authentication on our end, which would have prevented this attacker from gaining access. In hindsight, we understand how short-sighted this was, and how the compromise of just one e-mail account can lead to such a cascading failure.
Shortly thereafter, we were contacted by a figure named “Victor” on our public slack. Victor immediately made it known that he was responsible for the hack, and demanded a large payment to disclose the details, and it was implied that this data would be released publicly if we did not pay. We had internal discussions and it was determined that paying this attacker would not have provided any reassurance that the data was not made public, so we politely declined. It was also decided that we owe it to our users and the community to immediately disclose the methods used and the nature of the data that had been breached. In the midst of everything, we published an announcement of the hack here: early Monday morning.
Victor then went on to disclose that he had gained access to our private gitlab which contains both released and unreleased source code that powers the platform. This code contained old, outdated db credentials, and our backend servers are fully locked down with firewall and ssh access, so the attacker was not able to gain access to any sensitive user data. That being said, having access to this source code may grant this attacker (and whomever the code is shared with) an inside look at the platform, which may disclose more potential attack vectors. We are in the process of doing a full audit of all source code and third-party systems, and are reaching out to a third party audit service.
Victor then explained that in order to execute this hack, he (or they) had gained unauthorized access to one of our developer’s passwords that had unfortunately been shared between github (the initial point of attack) and his Coinigy e-mail address. He then used this e-mail address to gain access to our private gitlab. Our gitlab had been locked down with ip-restricted access prior, however that restriction was recently lifted temporarily while we onboarded a new developer. This would prove to be a very costly mistake, as the hacker had lied in wait until these restrictions were lifted, and gained access by resetting this developer’s password, logging in, and being granted access to development repos.
We’ve determined that the attacker used a vps located in the Netherlands as a bounce to login to the third party services, and our gitlab. This same IP range was used throughout. We are continuing to investigate through all available channels.
After extensive forensic analysis, we have definitively determined that this attacker did not gain access to sensitive user data, such as API keys, passwords, balances, account history or IP addresses.
We are taking extensive mitigation strategies moving forward, and reshaping our entire security policy and mechanisms, refactoring code, and have updated all third party API keys. We are putting in place a company-wide policy to enforce 2FA on all accounts, both internal and external.
Several Twitter users had expressed learning more about the steps taken to keep user API keys secure. I’ll preface this with the fact that your exchange API keys have built-in security features by default, courtesy of the exchanges themselves, namely the ability to grant specific permissions, and restrict access to specific IP addresses. We always stress the importance of never allowing “withdrawal” permission on your API keys as an added level of security, and we suggest that you restrict access on your keys to solely Coinigy’s IPs (which are made available upon support request).
The question was posed on Twitter by the user @flibbr, “Are they encrypted client side without coinigy servers ever knowing how to unlock the encrypted data for the api keys ?”
One of the main benefits of Coinigy is the ability for the platform to monitor users’ balances while offline, place stop limit orders, and trigger trade alerts. This frankly would not be possible were the keys encrypted on the client side, as the server must possess the key in order to make the API request. I’ll note that while on the surface, locally-encrypted keys may appear to be more secure, however it still opens the user up to compromise if an attacker were able to gain control of the local PC or encryption passkey. Local encrypted key storage is a feature we’re considering for a future version of the platform for those users who do not wish to have offline reporting or trading. As always, adding API keys to Coinigy is entirely optional.
Private keys are encrypted with middleware, and they are never returned to the client, under any circumstance. To take it a step further, technically the keys are encrypted on the client using TLS, since only secure connections are allowed to the servers all the way through. Yes, keys are decrypted while placing API calls to the exchange (and only sent via secure SSL connection), as it’s a required piece of the puzzle. Due to the nature of this transaction, we always recommend the above steps are taken to secure your keys, such as disallowing “withdrawal” access, only giving the permissions you require, and restricting keys by IP.
We have been open and honest with regards to our API key security from day one, and have gone so far as to put forth extra measures to help protect users’ keys such as SSO connections (currently active on BitMEX and C-Cex). We have reached out to, and we invite more exchanges to adopt this technology as it removes the need for users to copy and paste their keys entirely.
I would like to personally apologize to our loyal customers for any hardships that you may endure due to the breach of this information. We’ve built this platform from the ground up, as a team of two (and now grown to a team of 7), and while we’ve faced many hurdles over the past several years, this is by far the greatest mountain we have encountered. This may sound cliché but the one thing that gets us up in the morning is interacting with our users and providing a service that people can rely upon. There’s nothing more satisfying than closing a support ticket, knowing we successfully helped a user fix their issue completely, or gave them access to tools that helped them increase their profitability as traders. I want to thank all of those in the community who have reached out and offered your support. We hope to continue to receive your support as we move forward.
Please don’t hesitate to reach out about anything,
-William