Hacker Time

All right, we're live. This is Hacker Time and I'm Evan Johnson. I'm from the Cloudflare security team and with me today is, I call him the break man, but Justin Collins, President Beef himself, thank you for coming on the show and I really appreciate it.

Do you mind, like I called you the break man, do you mind giving us an intro to yourself and like who you are and how you earned that nickname?

Sure, thanks for having me first of all.

Glad to come on and as I've said before I owe you because you came and gave a talk for me.

Oh that's right that's the whole reason you're on, that's the whole reason you jumped on short notice.

I also like to help.

Yeah, so my name is Justin Collins. I think if people know me it's because of a tool I wrote called break man static analysis security tool for Ruby on Rails and when I go to conferences I've started, well when we used to go to conferences, I would just write break man guy on my name tag because people are familiar with break man the tool but haven't don't really associate it with a person a lot of the time.

I've been on security teams at yp .com, twitter.com, surveymonkey.com and currently at gusto.com.

Noticing a trend. Yeah, so a lot of SaaS companies with a brief detour at a very large company called Synopsys who actually bought break man and so I did it, I don't know, a tour of duty very briefly off of the dot-coms into into the land of dinosaurs and back out.

Yeah, well that's that's kind of the point that I really think is interesting where there's really two sides to what I've noticed.

There's two sides to the AppSec industry and there's this whole consulting world of Synopsys and Rapid7 and NCC and all of these dinosaurs and then then there's like the SaaS companies and there are these internal AppSec teams and the jobs are completely different.

It's a completely different world both of them.

I'm curious like what you thought when you went to the other side having been at a bunch of dot-coms and on AppSec teams really good AppSec teams at Twitter like all of these places and going to the other side.

Yeah so I think it's definitely been an interesting trajectory.

You know Twitter of course huge engineering team really you know at the forefront of like shipping things fast sort of DevOps if you will you know continuous deployment that kind of thing and then going to SurveyMonkey.

SurveyMonkey was a little bit slower pace they weren't public yet whereas Twitter had gone public so as you know once you go public there's like this cadence that happens where you're like oh my gosh like every quarter we have to deliver on what we said we were gonna do so it kind of pushes the company a little bit and then going to Synopsys where you know I was on the Covarity team and they were going fast because they'd switched to quarterly releases and to be honest it was a very relaxing in some aspects relaxing.

For one thing I wasn't on a security team so the stress level was like yeah yeah don't worry about security.

Not your problem. And you know my role there was not to go into it too much but not significant enough to have any stress so yeah and yeah seeing the way they they sold the product the overhead that they have for just like getting in with these big companies and selling them multi-year you know multi -million dollar deals but also the machinery of the company requiring that is a part that's kind of mind-blowing right they couldn't do small deals you know from what I understand my perspective etc they couldn't do small deals so for me it's like look there's a whole part of the market that you're just ignoring because your product is too expensive to develop and sell so like there's a whole set of people that will never benefit from your product.

Yeah I mean this gets me on a huge tangent of like just SaaS products and and security in general like one trend I've noticed is that you go to nine out of ten security websites of a security product and in order to try it you have to talk to an enterprise sales representative and fill out a form with your cell phone number and they're gonna call you four times a day for the next three years and and that's just to do a seven-day trial or something where they quote you an insane price off the bat and and I really think that there are so many that this is because the products are first of all the products aren't very good because they're built by security people and then the other thing is that the overhead like they've tailored their their companies in a way where the overhead based on the parties that they're throwing at all of these expensive formerly known as conferences all of these expensive gatherings and all of these like sales cycles that are very expensive they just end up with a marketing machine and not really a and this is on the SaaS side I think that it's different on the consulting side but similar where there's a lot of overhead and it is a different business model where you're selling hours and not a not a SaaS product yeah I agree you know something that I was was pretty shocking to me is finding out that you know the reason they want you to fill out those forms and they won't tell you a price is because they're trying to figure out how much they can charge you you know it's not like you where you go to a store and you're like yeah how much does this this thing cost and there's a price tag you know yeah that's the cost and I pay it or maybe in some cultures you haggle like a little bit on the price but there's still like a price tag right that you start from instead they're like okay you know hey this is Cloudflare like they're huge I'm gonna tailor my pricing to what I think they can pay or like hey this is a company I've never heard of probably not gonna be able to make as much from them I'm not gonna give this much attention but I'm also not gonna quote them as as high of a price yep and it's a competition to see you know because they do have a price right there's a price list they have internally and it's just a competition for them as salespeople to say like how much more than this price can I sell this for yep right or they can say they can go the other direction right and say like well it's gonna be more important that we have a business relationship with this company so I'm gonna work with them and give them a deal so that we can put their logo on their web on our website and you know tell everybody that they're a customer and that will get us more customers and we can charge those customers more and it's not really so much about how can we help these companies be more secure right that's that's really not like the focus of the sales machine right yeah it's a the I have a joke where it's just like all security products start at an initial quote of 250 ,000 when you're talking to different vendors on the like I want this tool the static analysis tool I want this penetration test like it doesn't matter what it is the going rate for anything in security starts at 250,000 and then like from there you ask for a lower price and it either comes down a little or sometimes it comes down to 1 10th of the original cost something like there's no rhyme or reason for what happens next it's all everything's made up and the dollar sign actually the number after the dollar sign actually doesn't matter yeah but it starts at 250 K that's fascinating how and I appreciate that I'm not the only one feeling the insanity of this part of the industry what how did you do the pricing for Brakeman for Brakeman so for Brakeman Pro first of all we we launched it well for people's reference Brakeman is an open -source project or was an open-source project and so people were used to using it for free and I would say the majority of users were developers not security people so you're starting from okay here's a free thing and they're like hey we have something we want you to pay for and we priced it if I recall correctly this the the base price was like $5,000 and I can't remember if that was like per user per year or something like that and we were really targeting security professionals was the thinking and it was way too high because people were like wait there's a free thing and then you want me to pay $5 ,000 a year minimum what is up with that so we ended up dropping the price to $500 per for that for the desktop we had two different things there was a desktop product and then there was the like command-line product that you might throw in CI or wherever so for the desktop product it was $500 per seat per year and then we ended up doing tier pricing to make that a little bit easier and then for the like I want to install the Ruby gym it was 2000 I think $2,000 a year just a flat rate like just $2,000 a year you can have this and then it was like we kind of stacked things right oh you want support like it's a little bit more if you want to talk to us on the telephone it's a little bit more because like that would mean like you'd call me right and just that's worth it I don't know if people thought that but there definitely were a couple customers where I'm like you know they were in sort of like the government space right either government agencies or government contractors and they really did expect like I'm gonna pick up the phone and I'm gonna talk to somebody and they're gonna solve my support problems fortunately not not too much of that and this pricing was on the website so you could go on the website put in your credit card buy a license or however many licenses and there was no enforcement just so everyone knows like there was no phone home nothing like that there was you know and I think I think this is where maybe coming from this like actually being a security person you have like a little bit different mindset where you're like whoa I don't know if I want to build a tool that like phones home right because then all those like paranoid security people are gonna be like why is my brake man Pro desktop like app like pinging back somewhere right to do a license check or whatever so really it was on our system and I really didn't care you know the the first of all if you want if you start a product or a company of any kind the fact that people start giving you money it's just like whoa like someone gave me money for something that I made so every time we got a customer it was exciting like well look someone someone else gave us money this is amazing the secondly it was really about getting customers and much less about making money because you know typical sort of startup mindset was hey we'll get customers first and then we'll figure out like how to actually make money of course our we had like almost no cost so that wasn't like a big deal but that was kind of the mindset that I had anyway and the other part was like yeah transparent pricing you know sometimes people would ask us for discounts for the most part didn't really give out any discounts because it was cheap and occasionally you would get we'd get resellers now resellers is like a whole nother thing where they want to be the middleman they want you to give them a discount and then they bump the prices up for the people they sell to and I was like no I'm not giving you a discount and also our prices are public so if you charge your customer more like they can see it right like they can see the list prices on the website so I mean it's up to you if you want to charge them more but like literally here's our prices it's on the website right here yeah we're the resellers were they like third-party security companies like doing doing like pen tests and no no they were like software resellers oh that's bizarre right like you go through like you know some some companies like it's easier you set up a reseller relationship right you you say okay instead of going and buying software from Cloudflare and pro like whatever I buy it from one company that sits in the middle that way I have one contract one billing statement like one salesperson that I talked to and then the people in the middle you know they go out and they say okay break man pro like I have someone who wants to buy your stuff give me a quote give me some pricing and then I'll give you money you give them licenses it kind of works like that yeah the whole industry is a little crazy when it comes to pricing and I appreciate the the honor system of having software that doesn't enforce licensing or anything like that like they could probably since I'm guessing it was written in Ruby they could probably open it up and change an if statement or two and absolutely that's awesome but one thing you mentioned is that you kind of let out here is you used to work at Twitter and Twitter is all in the news the and so I don't want to get into any details you're uncomfortable sharing but I'm guessing that you you like have a lot of context that other people don't have when you read this it's your few your few years removed at this point where probably a lot's changed but but kind of the engineering culture the the types of internal controls there were when you were there and one thing that I found interesting is there's been a lot of companies that have been public published having this kind of an internal dashboard that has way too many permissions and everything I think every company builds things that help them help their customers and so and what was interesting to me about the Twitter issue was that we have this internal dashboard that that hackers ended up getting access to but the as an app sec person I would totally be in the room when they're building that that internal dashboard and and I would have trouble pushing back on on something like we're gonna have a dashboard and you're gonna be able to mess with anybody's account and change the email address like like existed in this dashboard and in this hack because there's no OS top 10 vulnerability for that it's not an XSS it's not a code fix it's kind of like a business problem around you are you have this thing open to everybody at the company in Twitter's case or some subset of employees and and it is it really apps X job to be doing that I'm curious what you thought about this whole thing because you've been on a lot of apps like teams and I'm sure you've pushed back on something similar well so I think you have to start from where Twitter used to be so there's a very interesting document which is the FTC injunction against Twitter 2008 -2009 when some high-profile accounts were taken over Barack Obama then candidate Obama Britney Spears some news anchors and if you read that document which I know it's a legal document people are scared of legal documents but it's very straightforward especially if you're a security person and then they lay out what Twitter did wrong very clearly and there's some really like just to recap the way you would log in to Twitter as a Twitter employee was the same way everyone else logged into Twitter go twitter.com login if you happen to be a Twitter employee suddenly you were an admin for all of Twitter like that in every employee had the same access yes and it was the front door login and on top of that there weren't like password requirements yeah it's it's exciting so that of course is not the case now one if statement I'm just imagining that code block and there's one if statement protecting the entire website yep yep of course even by the time I was at Twitter that was not the case anymore but my reaction with to people's reactions on the Internet's I was a little bit surprised that people surprised because every SAS has this you can't run a SAS business without having an administrative dashboard the way I like if you're let's say not in this world the way to think about it is like if you go to a store sorry to bring up stores again I know we can't go to stores now but wouldn't you talk about doors the the front of the store is beautiful right everything's lined up everything is clean you know everything's orderly it's there to entice you get you spend money it's there to make you feel comfortable right but if you go in the back of the store it's terrible everything's a mess it's not clean there's no comfort you know no one spends time on the back of the store and that's what the admin dashboards are they're the back of the store who is going to be the person who's like hey I think we should spend time making the thing that no one sees but us beautiful and secure and easy to use like no it's a mess it's an afterthought really so you have to come from the perspective of like that's the starting point right and the starting point is look we just need to get things done for our customers right they have a problem we have to be able to fix it then you kind of back your way into like hey maybe not everyone at the company needs to do everything on the admin side and then you start layering on permissions and some security but I really think it's in general for most companies it's going to be an afterthought until they get to a certain size on the you know a second point here is think about who actually needs access it's going to be your support people who's probably one of the lowest paid people at your company probably the support people so you have like this the people who need the most access are the people who are probably the easiest to social engineer and the because they actually talk to people all day long that's like their job so they're used to talking to customers they're used to getting you know requests to change things oh I messed up my email I forgot my password I need to change my phone number they're used to getting those requests and they're probably the easiest maybe not the easiest I don't want to like paint them in a bad way but if you think about who you need to bribe in a company right or who you need to maybe we'll stick with bribing it's gonna be whoever is paid the least right like so it you have like a very vulnerable population and they're the ones who really really do need access yeah I mean just thinking about customer support their job is to make people's problems go away and like and they're in this mindset all day of helping people and this person wants me to do X I'm gonna do X and if that's click a link and like investigate their problem I'm not gonna say sorry I don't click links I'm gonna help because they click links all day they're gonna help they're gonna help the customer whoever's writing in whether it's good or bad like hey I have a problem with my account or hey let me why don't you click this so I take over your your laptop and and so customer support is definitely like the front door of a lot of this because they're helping people because they're front line they're on the front lines with of everybody's problem and that's their job like you said yeah and if you have someone who's angry or pretending to be angry right like you know why can't you just fix it and like you know I'm gonna like whatever like I'm gonna cancel my subscription or you know whatever it is my contract because you know you're not fixing this thing yeah there's a lot of pressure there a lot of emotional pressure so it does make sense but I'm hoping that you know what's happened with Twitter first of all people should recognize that yeah every every SaaS company has these kinds of dashboards they're not malicious or evil sometimes they are but like the intent is like we need to help our customers right so that's the first thing the second thing is I'm hoping that this is an opportunity for application security teams product security teams to go back and say like hey look at what happened to Twitter let's do a review of what our admin panels look like what our permission structures look like could this happen to us and would we even know if it happened would we have the right logs all of those kinds of things and then how do we put protections around it you know I think you know we talked about support people I think the other people who tend to get ways you mean permissions are the engineers themselves right like oh they're an engineer they need access to everything yep often not the case and as an engineer myself I don't want that responsibility actually coming back to Twitter that reminded me that I didn't have access to the admin panel for a long time and I'm like good great good thing I don't have access to that right of course eventually I needed it for something but yeah I really hope this is sometimes as a security professional you have to take things from the news and use them to your advantage right yeah hey look look what happened to Twitter oh we have an admin panel just like that what if that happened to us you know hey this would be a good time for us to revisit and implement some controls around this and you know depending on what's already in place right yeah we have a RBAC system in place in our internal dashboard there's always more we can do but we've definitely used this to to get some changes out there and and not let a good crisis go to waste and I'm curious what controls like if you're at a new company and look it's a 20 person company and they're saying we're gonna build an admin dashboard like as an app sec person I'm always thinking about good design where what are some design decisions that we can make early on that will that'll help us later later down the road be in a much better place what do you have any ideas of like the hills you die on for good design principles of your admin dashboard well I'm sorry you put it that way because I probably would not like I you know at some point you have to say like my job is to give you guidance on stuff and if you're you know the most I can do is tell you I think that's a really bad idea but I think what you mentioned is a good starting point right like if you can have fine -grained permissions built in from the beginning instead of something you have to retrofit even if it's like hey we're 20 people everyone in the company has access to everything right now if it's much easier to have that stuff from the beginning rather than try to retrofit it I mean that's the case with a lot of engineering but I would say having fine -grained permissions even if they're all on but at the beginning being able to go back and say like hey like you know we're not 20 people anymore now we're a hundred people and there are people in the company that just don't need this they don't need that they don't need to see this information or that information being able to go through and like make those changes very easily as opposed to like oh now we have to do a bunch of engineering work to implement this system like that's something that's much harder to get let's say a manager to sign off on like oh yeah you can spend a month implementing these controls that on this sprawling code base that no one put any effort into I'm a I yeah you're totally right I I've had to push back before on on like this exact thing of somebody wanting to build something and do absolutely none of the future work and of kind of setting the stage for a good situation down the road and and it's really important to at least get some framework out there of what good in the future looks like early on I'm a big fan of like little things of small design decisions that change a lot of behavior so like something like like re -prompting for 2FA on on any state -changing actions this is something that is makes it so if somebody's stolen your cookie or something then you're you're not very likely for them to to be able to change email addresses or something and also of things like that and but there's no products or anything out there doing that right now so it's it's hard everything is custom any of these things that you want to do is it's mostly custom all right well we're actually running out of time and I want to say thank you for coming on our show Justin I'd love to have you back anytime you'd like to come on or any time anytime you've got something to share or anything it was great having you I appreciated the discussion on pricing and everything thank you for the invite I'm happy to be here nice and I also appreciate the insights on Twitter and I've got to go check out this FTC report all right well thank you so much and I'll talk to you soon all right thanks Evan see ya