Phishing Test. PHOTO: Cybercrime Magazine.

Why 20 Percent Of GitLab’s Employees Failed The Phishing Test

We are all tackling the same threats

Barbara Babati, Marketing Manager at Hoxhunt

Helsinki, Finland – Jun. 4, 2020

We love the way GitLab communicated its recent phishing test, in which 20 percent of its employees failed because they handed over their credentials. This sort of test shows how people are actually vulnerable to phishing even when they have received some training.

The full transparency of this test tells other companies that they, too, need to reevaluate their results and mentality regarding security awareness training. They need to reconsider how and how often they should train their employees on email threats.

The test showed that 20 percent of GitLab’s employees failed the test. That means 1 in 5 employees fell for the test emails — and this is bad news. For a data breach to happen, it’s enough for one employee to fail, and the aftermath can be quite disastrous.

We analyzed the GitLab phishing test: Why did so many employees fail, what do the numbers tell us, and why is this quick in-house test great? We also offer a few recommendations on how companies with currently high fail rates can prepare their employees against attacks.

Four reasons employees failed the phishing test

1. The attack was well-planned.

The attack looked very legitimate. You really need frequent practice to become suspicious about it.

To find out what the email looked like, visit the page where GitLab explains the test in detail.

To achieve a legitimate-looking test, GitLab did the following:

  • GitLab purchased the domain name ‘’
  • They used G Suite to deliver the phishing emails.
  • They made sure everything looked legitimate by also using SSL certificates so that the automated phishing site detection wouldn’t spot the test right away — removing an important technical protection.
  • Using SSL could make quite a few employees less suspicious.
  • The email mimicked GitLab emails and asked employees to click on a link to accept an update.

Typically, attacks that succeed are extremely well-planned. According to experts, developing a sophisticated and targeted phishing attack can take up to 100 hours of work. When someone is working that much to deliver a flawless threat to your employees’ inboxes, you can expect that it will be easy to fall for the bait.

2. Phishing training should be more frequent.

According to the handbook of Security Awareness Training on GitLab’s website, GitLab delivers phishing tests to employees at least once per quarter.

In order to make a real impact on the learning and behavior of employees, phishing training must be more frequent. People need to see different attack types, especially since attackers act fast and develop new vectors frequently.

3. Employees should be taught what to do.

Using frequent phishing training can help employees know what they are supposed to do once they encounter a real threat.

Going through training materials does not develop the right skills and behavior — people won’t necessarily be able to recognize difficult clues. They should also be taught that if they doubt an email, they should think critically and report it. Behavior change is the only way to reduce the failure rate from 20 percent.

While it’s great to have training materials and guides, nothing can replace actual practical and engaging training.

4. Reporting should be simplified.

When users encounter something strange, they need to send an email to the security team that monitors a ZenDesk queue at GitLab. This is common practice in many companies.

While it’s not the most complicated method, reporting could be simplified further, such as clicking a button or installing a plugin within the email client to send the report and details instantly for analysis.

What do the results tell us?

First, the sample size was rather small; GitLab used random sampling and sent the phishing test to 50 employees.

Thirty-four percent (17 employees) clicked. Clicking the link is a problem because there could be a malware download behind it.

The test aimed to try whether team members would give away their credentials. Ten out of the 17 that clicked (69 percent) exposed their login details. In real life, giving away credentials can be a big problem, especially if a company fails to implement two-factor authentication (2FA). Even if there is a 2FA implemented, attackers could bypass that, as we wrote earlier.

Only 6 employees (12 percent) reported the email to the Security Team.

The results tell us that GitLab has a long way to go in educating their employees before the fail rate will be reduced from 20 percent to an ideal 2 percent.

The numbers also reveal that only 23 employees acted on the email, so it’s possible that the email failed to engage the rest of the sample population, or they just simply ignored it and didn’t think that it was important to report.

In the case of a real phishing attack, it’s important for everyone to know that they need to report it so the Security Team can respond better to possible threats.

Why was this exercise great?

The phishing email was developed excellently, and it could have tricked even the people who had always thought that they would not fall victim to phishing.

While it was a fairly difficult attack, there were some good clues available to spot the threat. The clues were in the email address, which referenced an older computer model than what people use nowadays at the company and in the fact that there was no secondary communication method so that employees could check in if they had concerns. Additionally, the header of the message in the email also included clues, such as the keyword ‘phish’ as well as references to the illegitimate domain

The exercise also proved that the company has a lot of work to do regarding security awareness and phishing training to reduce the risks posed by such issues.

Our recommendations for decreasing the fail rate

Based on our experiences, our main recommendation for companies that want to decrease their failure rates would be to do more practical phishing training. Frequent training does not only educate people about the dangers of phishing emails, but it also teaches them that, upon encountering a phishing attempt, the right action is to bring it to the attention of the Security Team.

Minimizing human error is only possible with great, personalized training. People learn at different paces, they have different skills, and different exercises are relevant for them. Also, failing the phishing test once in a while will teach them not to make the same mistake twice.

We are all tackling the same threats

Once again, we appreciate the transparency and the great work of GitLab so much; it would be great to read more stories like this as we can all learn from these. As we are all tackling the same threats from social engineers, it’s important to raise awareness about the vulnerability we have in the workplace.

Hoxhunt Archives

Barbara Babati is Marketing Manager at Hoxhunt

Sponsored by Hoxhunt

Our mission at Hoxhunt is to enable everyone to protect themselves from cybercrime. We want you to be able to protect yourself, your family and your company.

To this date, changing employee behavior to a secure one has been incredibly hard. Organizations have tried pushing information to their employees in classrooms and in e-learning solutions. They’ve tested the results of these awareness campaigns with phishing tools and penetration tests, giving extra training only when an employee fails. While some of these methods are great for other purposes — like e-learning is for regulatory compliance. The actual results in changing employee behavior to a more cyber-secure point out otherwise, the traditional methods to patch the human component do not work.

That is why we built Hoxhunt. We want to turn employees from a company’s weakest link into the strongest asset against cyber attacks. Our gamified platform trains employees against phishing attacks in a fun and engaging way.