I have dabbled a bit in security testing. It’s something that, despite all the breaches in the news, I’ve generally struggled to get buy in from management to spend time on in the past. What suggestions would you have to help convince management that security testing does matter?
I’ve got some experience in security testing, and want to help people learn about it too. I’ve tried using the OWASP Juice Shop, and found I really struggle with it. I have the Pwning OWASP Juice Shop book, but even then I found it too daunting. Is there an online tutorial or something that would help me learn how to practice and help others practice with the Juice Shop?
It would be nice to understand what strategies you have seen to bring security into the inception / design / requirements gathering phase and how testers could contribute to this effort.
Discussing the pros/cons of external consultants for security testing and what should be considered as such.
I would also like to hear about best practices, patterns and anti-patterns on how organizations embrace security testing (is there a separate group of specialists, do specialists coach development teams, when is the testing done, etc).
There are different companies already that I’ve seen requesting for OWASP Top 10 Security Audits. Is it a regulatory thing? And if yes, who can perform this type of security audits? do you have to be certified?
How would you approach such a request? (to perform an OWASP Top 10 Security Audit)
Hi. i have been working as qa since 6 years now. Can you please tell me where to start learning security testing from scratch. Books, tutorials, courses etc anything would help. Also, could you please tell us that after learning about security testing which is the best certification i can take?
It’s reasonably typical to have a pretty widespread, varied architecture in terms of back/front end, a bunch of services, functions as services, billion json files… When it comes to security testing, it can often feel like it’s overwhelming to know where to start. Is there any method or thinking tool you could recommend for looking at your system and deciding where to start looking for vulnerabilities?
I would love to start adding Top 10 testing into projects, but the OWASP wiki style confuses me. Do you have any other resources that really stair-step you into adding (and learning) security testing? Such as, what’s the first one test you would add to a project to move toward security testing it?
I’m excited to spend the next hour trying to answer as many of your questions as possible. I’m just going to start from the top and see how far I get. I saw some really great questions so let us get started!
So I’ll be adding both my own answer and some from my colleague Tomas who is the Batman to my Robin. (Love being a side kick!)
What I’ve found helpful previously is to try and tell an engaging and compelling story illustrating why security is important and why you need to test it. In one situation, it might be sufficient to point out that your competitors are doing it and that your company should do it too. In another, you could point to the financial and bad-will risks to your company if your user database were to be hacked. Maybe the risk for THIS particular system itself might not be that significant, but by letting some of your team members do some focused pen testing on it, they can gain valuable experience and skillsets for the big development initiative next year.
Honestly, I haven’t had much issues on the management side so I struggle a bit with answering this. My issues have been with convincing testers and devs that
They can do it
They should do it
Maybe it’s because I’ve been in a management position myself? What I have done is basically putting together a business case about cost vs. benefits. That’s it. Money talks!
It started with me realizing we spent LOAAAADS of money on (maybe) yearly audits and they just kept finding the same stuff. That seemed like money down the drain to me so I started reading the reports more in detail and then found how to test it myself. That’s not proper white or black hat testing though! But the tings a normal audit finds is not hard to test.
Also: Try to find allies. I found them in a lead architect and our IT security responsible.
Hope that helps! Let me know otherwise and I’ll try and help you write a manager friendly report. It’s my super power
It can definitely be challenging to really dig into the Juice Shop challenges! While the book gives some interesting background material and useful hints on the individual challenges, it doesn’t focus as much on the larger picture, e.g. cultivating a security testing mindset. What I think might be helpful to that end is to pair up and do a few rounds of exploratory testing sessions around the shop without even thinking about the challenges themselves. Take some time to explore some of the many powerful tools to assist with auditing. Just by browsing the Juice Shop through Burp or Zap can give plenty of interesting information and openings for further investigation.
Lena: Well… I would suggest attending our workshop at TestBash Germany but it’s sold out so… Find someone to have us do the workshop again? Afterwards we’ll make sure to put the material online and maybe even do some video to support it. We (as in Tomas) has put a lot of effort into making it accessible to people new to the area and we spend more time discussing the solutions than flexing our tech muscles.
Apart from that I agree with Tomas. Find someone to pair with and play around with it. Try to spend more time on why the different challenges are done the way they are than on scoring. They each have somethin to teach.
Or find a meetup! There are security related meetups all over the world and they are awesome.
Also I see a lot of talks and workshops on the topic on conferences around the world so I’m super excited to see where that takes is.
I know a lot of people have done similar things so there probably are loads of tutorials out there, I just know them. Sorry! Let’s explore that more afterwards and I’ll see what we can find.
But also: Don’t use Juice Shop if that is not up your alley! There are many many other ways of learning.
Bug Magnet has some good standard input tests that you can use to start a discussion.
Try out some tools.
Install SonarQube and discuss what that finds.
Try writing evil personas & user stories and discuss the implications of those.
Try out the security tab in Chrome dev tools
I also personally LOVE the OWASP Testing guide. I find that to be super super helpful in learning about what to test and why: https://www.owasp.org/index.php/OWASP_Testing_Guide_v4_Table_of_Contents
For each of the top 10 it has categories and for each categori it explains why it’s a risk, how to test it, how hard it is to test and how big the risk it. (and more)
One useful strategy to include security in the requirements gathering process is to include the attacker as a persona interacting with the system, in addition to the usual customers, administrators and other ”white-hat” users of the system. The attacker should also appear in (mis-)user stories and have verifiable requirements of their own. For instance, you might express the requirement that when an attacker attempts to login with the username ”administrator” and a random password, the system should log the failed attempt. You can then refine and add to these requirements with each iteration of your development. This approach weaves security into the application, just like any other feature or aspect and also allows the development team to assume ownership of the security of their solution. I think that in order to sustainably deliver reasonably secure solutions, the development teams need to be in charge of the security as well. I can recommend The OWASP Top 10 Proactive Controls project for more useful guidance when it comes to security ”shifting left”.
Thank you so much for this question! This is where I love to start and where I think we have the biggest impact.
There are so many ways we can increase understanding of security and risk. Evil personas & Evil user stories are fantastic. Make them colourful and fun if needed to lower the sense of complexity. Try doing some sort of over the top “What’s the worst thing that could happen if…” workshop and keep it whimsical if people are scared.
OWASP (again) has great material but if you google there is SO much out there https://www.owasp.org/index.php/Agile_Software_Development:_Don’t_Forget_EVIL_User_Stories
So security starts with requirements, just as you say. The basics is just having security as a requirements. To make it easy one could start with testing for OWASP top 10 (or even pick 2-3…) My IT Security responsible-ally uses the NIST framework which can be a bit intimidating but on the other hand you get a sense of urgency when it’s formal and standardized. They also have good documentation: https://www.nist.gov/cyberframework/online-learning
BUT my biggest suggestion is to make sure EVERYONE (that is… EVERYONE) understands the basics!
So I think my list would be:
Get security into requirements. Use evil personas/user stories or just put some owasp req. into every single story.
Get security into the testing. As a manager, I put it as a part of DoD and our testing strategy as a must.
Get security into the pipeline. Install SonarQube, Whitesource, Zap and/or other tools into the pipelina and make sure we at least check that the code follows standards, that we don’t use insecure libraries/tools, that we use updated versions of our tools.
We’ve just implemented Zap in to our CI. Do you have any experience doing this? Do you use Zap or could you recommend something else? We are trying to do as much as we can to gain more confidence in the software we are developing.
Lena: I honestly don’t know if any part of the world has made it a legal requirement but some areas definitely have regulations. There are a number of standards and depending on the business I would say it’s more or less formalized. I would assume there are parts where you need to be certified and since I’m not I would not accept such an assignment. I’m more into getting security into the normal development process in every business.
So if we assume it’s a company that wants a OWASP TOP 10 audit but they don’t require a certification I would use the OWASP Testing guide and create a checklist from that. There probably are tons out there already but I would tailor one to my needs.
I promise, you can learn to test most of them without being a super certified expert penetration tester!
I would also make sure Static and Dynamic security testing tools was mandatory in my dev. pipeline. I like SonarQube and Whitesource but there are loads out there. ZAP is interesting and something I will learn on a deeper level in the upcoming year.
Not sure that answered youe questions but contact me if I can help out further!
Lena: OMG this is such a big question I don’t know where to start.
Ok first of all: I’m not a penetration testing expert. I’m not a white hat hacker or a black hat hacker. I am a normal tester who thinks everyone should know the basics of security. With that said: It’s a huge area and testing networks or API-request are totally different. Do you want to have another tool in your belt of are you looking to take on a full-time role as penetration tester? So I would:
Decide on a goal. What part of security testing do you want to learn and to what level?
Prioritize 1-3 parts of that.
Get your manager in a room and create a plan. It will take money and time. You need his/her support in this.
Do it the agile way, small steps and continous evaluation and experimentation. Maybe start with learning ONE single thing and get comfortable with that
Resources depend on point 1. The certification I know about are not something I would recommend unless you need it to get a certain promotion, job or salary. I have heard other people like them but the I have really bad experiences with horrible teachers and money-hungry teaching companies so I can’t promote them.
I would probably prefer attending conferences, reading blogs, books, doing pairing before formal training.
So. First of all: don’t do a big bang. Find one small thing and start with that. Do agile experimentation and evaluation.
Apart from that I would look into:
Evil user Stories.
Find blogs, mailing lists and forums to learn from.
BUT ok OMG I’m gonna try and answer this in as short time as possible…
My main focus would be to start with understanding WHY things are a problem.
Why is Bobby Drop Tables-jokes funny? What do they mean?
What happens in an injection?
Why do we need to use httpS?
Why do we need to use safe encryption?
What is a strong password and why is it so hard to get people to do safe passwords?
Why are “helpful” error messages a risk and how do we find a balance between UX and security?
One at a time! When you understand the why the how become so much easier!
Resources to try:
Again: The fantastic testing guide! I really think it is amazing!
Lena: I’m running out of time so sorry for the short answer.
Understand API request/response models. Like really really understand them.
What are the different responses, authorization models, how can they be exploited?
Especially error messages… I love error messages! They can give you soooo much information…
Amber has a fantastic course on Test Automation U that I can’t praise enough!
Play around with Postman!
Restful Booker is a good resource for testing APIs