💻 Optimizing Developer Workflow with Sourcegraph

Hi, everybody. I'm just gonna present my screen. All right. Welcome, everybody. I will be talking about optimizing your developer workflow with Sourcegraph. Let me introduce myself before I continue. So I'm Prosper Otemuyiwa, co-founder of 4Loop Africa, and also a service called Eden Life that works in Lagos. 4Loop Africa is the largest developer company in Africa at the moment. It's currently in about eight countries. And Eden Life is a service that is based in Lagos that allows you to order food, laundry, and cleaning, and just have your home services taken care of. And then I work currently as a developer advocate at Sourcegraph. And then I'll be talking a lot about Sourcegraph on this call. So we talk about the software developers mantra a lot, build, ship, scale. If you're a developer at any level, you must have heard this several times, right? But as much as we say build, ship, scale all the time, there are a lot of processes that are involved in building, shipping, and scaling our applications, right? And building, shipping, and scaling great software is really, really hard. Like very hard. It involves a lot of processes, right? And that's one of the pains that we experience as a developer. But one of the best companions we can have as a software developer is to ensure that we have great developer tools that can make you move fast and also at scale, right? You can't do it on your own. Even if you add more engineers to a project, which a lot of people believe is the right decision, most times, if you don't have the best developer tools to help you scale or to help you move fast, you're still going to be back to ground zero. So I'm just going to be walking through all the developer workflow optimization skills that you can use with Sourcegraph, right? I have tips for you today, about four tips for how to optimize your workflow with Sourcegraph. So number one, optimizing with code search, right? We read more code than we ever write code in our career, right? And one of the things that will enable you to move fast as a developer or move fast when you join a new team is the ability to search for any type of code that you want. You know, we've heard rumors and we've heard in several places that if you join Google as a new developer, within the first week you have access to all the code base. And that is only possible through the help of a code search tool, right? Now, if you're outside of Google or Facebook, what kind of code search tool can you use to level up like this? Sourcegraph, right? Sourcegraph is a code search tool that allows you to search all of your code, public and private. And as Samuel Johnson said, knowledge is of two kinds. We know a subject ourselves, or we know where we can find information upon it. And one of the key metrics I want to talk about is that Sourcegraph users report an average of 30 minutes a day saved by using code search, right? So, when you double or triple that number, or when you have 500 engineers searching code repositories multiple times a day, throughout each day, that's a huge time saving, okay? So, if you're with me right now, I would like you to, you know, take your laptop, put up your phone, and go to sourcegraph.com, right? I want you to follow me as we go through in this, you know, presentation. Sourcegraph.com. Sourcegraph.com is literally a code search engine. It's a search engine where you can paste in any type of code that you have, or any type of code that you're looking for, and it's going to bring out results for you, right? And we are going to go through all the different types of code search patterns that Sourcegraph allows you to operate on. So, let's start with the first one, literal search, right? With literal search, you can literally put anything you want, okay? Here, in this example, you can see I put in a suspense component, and I used a keyword called lang. Lang allows you to narrow your search, right? If you want to search for anything, JavaScript, Python, PHP, assembly, GraphQL, you can narrow the search. And what Sourcegraph is just going to do right now, this snapshot is a snapshot of Sourcegraph.com, right? With Sourcegraph.com, if you don't have an account, and you're pasting your code, it searches through over 2.1 million open source repositories for code, basically. It searches through GitHub and GitLab. So, putting the suspense component, for example, and narrowing it down to JavaScript, when you click on search, this is the kind of results you get, right? It searches through over 2 million repositories for all the code base that uses the suspense components. So, you can scroll through what it is, the result is returning for you, because these results right now are sorted by the GitHub stats, like the most stat repository. So, you can see the most important ones, or the ones that people are looking at, right? And you can see how people are using this same suspense component that you're looking for. So, you can see how all the different implementations, or the different ways that suspense component has been used in code base all over GitHub and GitLab, right? So, this is a very good example. Another example, I gave this during the talk at React Advanced London, right? So, look at this repo keyword. You are searching through the React.js organization, right? And we are searching for this API called unstable batch updates, right? You've probably read about it in an article somewhere, or a friend has mentioned it. Or you just want to know more about API. Like, what does unstable batch updates look like? And who are the people using it? Which packages are dependent on it? How are they using it in the world? When you put it in source graph like this, and you press enter, it returns all the results for you. And you can scroll down and go through each of these code repositories and see how this API is being used. Now, we move on to the next one, regular expression search. This is one of my favorites, right? Because it reminds me of how we use regular expression in some of our day jobs. Many times we don't use regular expression, but when the time comes, we go to search, or we go to regular expression.com and start looking for how to use it. In source graph, if you already know how to use regular expression, you can harness the power of regular expression in searching for code. So what does this example do here, right? We have the lang Java, and we have imports on the MISC base 64 encoder. If you're a Java developer, or if you've ever used Java before, you will know that this is a package, right? It's one of the Java packages that we have. When you're migrating from Java 8 to 11, this is one of those packages that was duplicated. So what I'm trying to do right now is, hey, go through my code base, or go through a set of repos, and with this regular expression, I want you to bring out all the Java code that still makes use of this package that has been duplicated, right? So it's going to return all the results for me, and then I can use this result to start refactoring my code base. Isn't that wonderful? So if we have 10, 15, 20, 100 repositories, all in our organization, it's going to search through everything and bring out all the results of every piece of code in all of these repositories that has this package that's still in use, so we can know how to start refactoring our code base. Now there is a toggle ID here, right? You have to toggle this regular expression point here for you to be able to do your search with regular expression. So we go to the next one, structural search. Structural search is very fantastic, and I'll show you why. So you look at this button by the top right here in my slide, you can see I toggle this on. This allows you to do a structural search, and this is what structural search does, right? With structural search, you can literally search for blocks of code. So for example, you can do a try-catch search, right? Or you can take a function. Let's look at this example. I'm saying look into the source graph organization repository, point to the source graph repository itself, search within the 3.25 branch or the 3.25 tag, right? Again, within this branch, I want you to limit it to just go files. Now this is the method. There is a line of code called imports, right? Imports, dash, dash, dash, errors, dash, dash, dash, right? This dash, dash, dash is a placeholder that says, hey, there is something or there are some arguments, there's an argument that comes before errors. I have no idea what those arguments are, but I want you to return all the code, all the lines of code, all the codebase files that has an import errors line with errors in the middle. So the three dots start for a placeholder that says, hey, there's an argument that comes before this, but I have no idea, right? And there are some arguments that come after the errors, but I just want you to return all the lines of code that has errors somewhere as the second argument, right? And that's exactly what source graph is going to do. It's going to search within this particular branch, and it's going to return all the results, all the go files that has errors as a second argument, right? And it's so powerful because you can do this, you know, you can express yourself with this structural set as much as you want, right? And before the end of this talk, I'm going to introduce you to, you know, the source graph sheets that we have that can allow you to understand all the different syntax that you need to be able to set your code very well. So this is the second example, right? This is the block of code I put. If not paths .length and dot, dot, dot within a callee brace dot, dot, dot, right? This is literally a block of code, right? I am not, I'm interested in the codebase that returns this expression, but I don't know what is inside this callee brace. I'm just like dot, dot, dot, just return the codebase that has this as the beginning of the expression and anything within the callee brace. Narrows to JavaScript and search within the Google open source repository. And voila, we have our results, right? So here is returning every file that is within any repository within the Google organization that has this expression. This is the power of structural search and source graph makes that a walk in the park. So you should try it. Now the number two flow or number two tip that we're going to talk about is efficient code browsing, right? Is one thing to search is another thing for you to be able to browse through the files, jump to definitions, fight core references, you know, just jump around easily. And we're going to go through that with source graph. So a very good example of that is when you're reviewing pull requests, right? I can't count so many times where you're reviewing a huge pull request and you have different function calls. You really just want to click on it like you're doing your ID, right? So that it can take you to a particular file or it can take you to a particular definition or it can take you to a particular symbol. With source graph, if you look at my screenshots here, I have a source graph icon show up here. That is because I installed a source graph extension. Source graph has a Chrome extension, not just a Chrome, Chrome, Firefox. And I think there are extensions that we have that allows you that once you add this to your browser, once you're on GitHub, it adds code intelligence to your PRs basically, right? So see what I'm doing here. So this is a PR that has been reviewed. When I hover on it, it shows me the definition. It shows me precise code intelligence, right? On hover. So when I click on go to definition, it takes me also to the definition of this file. And when I click on find references, it allows me to find the references, all the references of that particular method call. So when I click on references, it takes me into the source graph instance. And then I can see all the different references of the new histogram vec method call that has been happening here. Then also look at it here, right? I clicked on go to definition and it took me straight to the definition. This is awesome. I review PRs a lot every day. And I'm sure a lot of you on this call to review PRs or you go through your PR or maybe you go to other people's PR, right? You want to be able to jump to different definitions easily. You also want to be able to find references. You also want to be able to know more about the code at that point. Source graph allows you to do that efficiently. And about this, I said something. He said, you don't have to know everything, but you should learn how and where to find the thing you need. I want to know. I love this code so much, right? And as you grow your career as a developer from junior to mid to senior, there are some things that you'll be an expert at. There's some things you will know, right? But what makes you senior, what makes you a developer that everybody wants to work with is being able to know how to learn and how to learn fast. And source graph allows you to do that. Okay? So the number three optimization we'll be talking about is developer workflow optimization with batch changes. This is an amazing feature. And I'll tell you why, right? A very good example I can give here is you're using a shared design system, for example, right? And this shared design system, you've called it in several repositories, right? Or there's a general change you want to have. For example, maybe you are using this ES5 way of writing code in JavaScript, and you want to move to ES6. Now, this is scattered all over your code base, right? How do you change code everywhere without having to submit pull requests, clone all these different repos, submit pull requests a hundred times. Imagine you having hundreds of repos in a land organization, and you need to change code everywhere. With source graph, you can do that with batch changes. With one single declarative YAML file, you can literally put an instruction for you to go ahead and change code in a hundred repositories. Isn't that amazing? So I'm going to show you now. So on sourcegraph .com, or if you have the self hosted version of source graph, or the hosted version of source graph where source graph maintains it for you, you can declare a YAML file. With a YAML file, look at this example I have here. The purpose of this batch change is to convert JavaScript functions to arrow functions, right? So you can declare it, put the repositories you want it to affect, put the steps, and then go ahead and put the title, the body, the branches, right? There is a guide on this, so you don't need to worry yourself about cramming this, because I'm going to make the slides available after the talk, and you can reach out to me on how to do this. So you have this. Now, when you go ahead and create this batch change, this is what happens. It goes ahead and look at all the repositories within the organization, right? And then it goes ahead to make that change that you have specified within the YAML file. Isn't that amazing? So if you have 50 repositories, for example, or you have 100 repositories, that singular change you've had within the YAML file, it goes ahead to open different pull requests, and then you can go ahead and now click Apply. It's going to show you, like, hey, these are the repositories we've run through. This is what it's going to look like. Do you want me to continue? So you will just go ahead and click Apply, and then it goes ahead to apply these changes to all the hundreds of repositories you have within your organization. This saves you hours of time, saves your organization, saves your team. So you can get started with batch changes in less than 10 minutes. I have a quick start guide here that you can click on, and then also, again, I'm going to make this slide available after the talk, so you can go ahead and just click on it. Number four, another way to optimize your workflow is via the optimized vulnerability discovery that we have at SoftGraph, right? With SoftGraph, you can easily find secrets, tokens, keys across your entire organization's codebase with a specific type of optimized code search, and I'm going to show you how. This is a very good example, right? I'm searching in the world and say, hey, RepoHash file is a keyword that you can use. So you can say RepoHash file artisan. If you're a Laravel developer, if you have this PHP, you will know that the artisan file mostly, if not only, exists in Laravel projects, right? So you can say RepoHash file csconfig, for example, or any specific file that you have in that framework or within that project or within those repositories. And then with regular expression, this regular expression, for example, looks for any occurrence that starts with key, secret, or token and matches the length to 32 keywords, right? And if you look at all the secrets and tokens we have, mostly they are either 32 or 64. So with this, you can try it yourself. I just took this piece of code, I put it within SoftGraph, and I clicked on the regular expression toggle here. And this is what it brought out to me. I hope these keys are no longer in existence or they are not working right now, because I hope the developers that put them have gone ahead to rotate it. But if you have that in your codebase, you should use SoftGraph to get rid of them as soon as possible. So I'll show you other ways of doing this, right? So let's look at this, for example. So I want you to look at this slide. Here, you have the repo, right? The repo is, you know, it can be one repo, or it can be an organizational repo. Then you have the pattern type regex, you've toggled that button. Now, this is an example, right? You can search for strings that are longer than 32 characters, or maybe bases before you encode it. Or you can search for private keys, like this. This looks at some of the private keys you see when you're sending up on GitHub and trying to search, you know, sync your servers with GitHub, you know, once your servers to recognize that you're the one trying to SSH into that machine, right? Or you can use this third example that I have here. I have so many examples here. So the second slide is going to show you some more examples, right? So this is another example of regular expression. So it looks for anything that starts with credential, secret, private, token, sensitive, password. Or you can just use this that we have. Don't worry, I have this available in the slide and I'm going to make them available. So you don't have to memorize them. Another way also is just to find exact vulnerable dependencies in your code. So this is an example, and we have this happen even in the JavaScript ecosystem. So you have maybe a package that a lot of repositories, you know, a lot of projects out there depend on, right? So you can take the name of that package and also specify the exact version and dump it within source graph. So here I'm saying, look for Laravel Sanctum, that is a version 2.11 in all of the Laravel projects, or PHP projects to be specific, because PHP projects are the ones that contain the composite.json file, right? So it's going to look into any project that has this file. It's going to look into this file and look for this particular package. So for the JavaScript ecosystem now, I assume this composite.json is going to be a package.json file. You're putting the name of your package or the name of the node module, and then you specify the exact version. And then of course you toggle on the regular expression and click the search button. So it's going to bring out all the repositories, all the repositories that have this exact matched version within the composite.json file, or within the package.json file. Then you can go ahead and do whatever you want to do, whether change or upgrade to the version that has been recommended by the authors of the package or by the ecosystem. All right. So integrating Sourcegraph to your team's workflow. How do you make Sourcegraph a part of your DevTools stack? Very easy, right? Number one, what's the best way to run a trial of Sourcegraph? I included a link here, but you can just go to docs.sourcegraph.com slash adopt slash trial. And then there is a documentation on how to run a proof of concept of Sourcegraph, right? You can run it on your docker machine, or you can set it up within your team if you have the time, or if you don't want to set it up, you can just reach out to us and then we'll do a trial with you. Number two, how can I best communicate the value of code search to developers who have not used it before? There are a lot of people that use Sourcegraph internally. Uber, Lyft, Yelp, they use Sourcegraph, right? Some of my favorite companies use Sourcegraph because we have huge, big code base that they deal with on a regular basis, and it helps them to I included the link here, so when I share this slide, you can click on it and see how all these companies are using Sourcegraph. Then the final one, you can just sign up on Sourcegraph Cloud right now, right? If you're on Sourcegraph.com, or if you have open Sourcegraph.com right now, you can go to Sourcegraph.com, register, sign up with your GitHub account or your BizBucket account, and then you're going to have access. Then you can go ahead and just add your repositories. You can add your private repositories, or you can even add your public repositories that might have not been synced by Sourcegraph, right? But you can add your private repositories. And then we have team support, the early access coming in next month, right? From December 9th, if I'm specific, if I'm right, we are going to be supporting doing early access support for teams. So right now, we do not have the support for teams on Sourcegraph Cloud. You would have to run the Docker install and try it yourself, or reach out to the team for them to set it up for you. Or you can, if you are looking for team support right now, you can just wait till next month, and we have team support on Sourcegraph.com, which is Sourcegraph Cloud available to everyone. So a recap, just before a recap, right? I said this before the beginning of my talk, hiring more engineers is good, but leveraging better developer tools is great. Make it a priority to bring the best developer tools to your organization. From experience, your team, your organization, your CEO, your CTO, we thank you. If you can mix the developer experience of developers within your team by bringing in the best developer tools, and one of the best ways to do that is to add Sourcegraph to your team. Recap, one, you can search over 2.1 million repositories, open source, and public repositories across GitHub and GitLab on Sourcegraph.com. You can also add private code and search private code across several repositories. Also, you can have access to precise code intelligence, like I've shown earlier in my slides. Then you can also have access to batch changes, right? Batch changes right now is not available on Sourcegraph.com because it's a paid feature, but if you come in as an enterprise client, you can have access to this. Now you can try it out. If you go to dogs.sourcegraph.com, look for batch changes, you can see how it works. Code monitoring. Code monitoring is just saying, hey, there's a part of the code base I want us to monitor to see how it changes over time, right? Sourcegraph allows you to do that. Code insights, too, also gives you visualization, right, about how different things are changing your code over time. That allows you to understand the most important parts of your code or the part of your code base where developers spend their time and interact with, right? It's still in beta, but Sourcegraph gives you the ability to do that, and by sometime next year, we're going to take it out of beta and make it available to everybody. So final recap, right? If you're trying to look for how to convince your teammates or your lead to use Sourcegraph, just let them know that right now, it allows your developers to move fast. You can try it yourself, right? With the code intelligence feature, with the search that I've just talked about, it allows you to discover your code very easily. It also allows you to onboard new developers to a code base easily. You don't have to jump on multiple calls to be explaining the same thing over and over to new developers. With Sourcegraph, you have access to your entire code base at once. So they can search for anything. They can search for a string, they can search for a class, they can search for a method, they can search for API documentation. Sourcegraph just allows you to do all of that. Also allows you to refactor your code base very easily. So do you want to learn more about Sourcegraph? I'm sure you wanted to learn more about Sourcegraph after this talk. We have a platform called learn.sourcegraph.com, right? If you go there, you can see multiple tutorials about search, how to add open source code to Sourcegraph, how to add private code. You can see the Sourcegraph cheat sheets here, where you have access to all the different types of keywords that you can combine to get the best information out of the code base that you have. And then Sourcegraph resources. Again, as I mentioned, we have learn.sourcegraph.com, we have docs.sourcegraph.com. Also, we write heavily on DevTool. So if you go to the DevTool slash Sourcegraph, you're going to see all the articles that we have written about how to use Sourcegraph in so many ways. We also have the blog, right? We write a lot about developers using Sourcegraph, companies, engineering leaders, and we also have a DevTool time podcast. I won't call it a podcast, like a bi-weekly YouTube session that we have, where people join in, where we interview developers about their workspace and how they use different tools. So if you go to YouTube and search for Sourcegraph, or search for DevTool time, you'll see all the different episodes we've recorded over time. And then we also have a podcast where we talk to different engineering leaders about how they optimize their teams and the organizations, and what the tools they use in their code base, and how they also use Sourcegraph. And I think right now, I have come to the end of my talk. I hope this talk has planted the seed of code search, code browsing, batch changes, optimizing the way you look for code, optimizing the way you learn within your organization, also just as you're developing things outside of your organization, right? If you're looking for how to use the latest React API, if you're looking for how to use Vue, if there's a particular API you need to integrate, Twitter, or any of those tools, you can just copy, paste into Sourcegraph, and it can tell you how others have been using these developer technologies, and it can allow you to ramp up easily on what you're learning. So I'm very active on Twitter as a Unico developer. I'm also very active on GitHub. Feel free to tweet at me. Feel free to come into my DMs. Feel free to email me at, you know, prosperousourcegraph.com as well, and I'll be willing to answer your questions. We also have a Sourcegraph Slack community, right? So if you go to about.sourcegraph .com slash community, you can join our Sourcegraph, where we are actively waiting for you to answer your questions about code search and how to use Sourcegraph. So thank you very much, and I hope you enjoyed this talk. Feel free to give me feedback as well. Bye for now. Carousel is one of the leading C2C e -commerce marketplaces in the Southeast Asia region. The marketplace mainly consists of preloved items, and we have categories such as electronics, fashion, property verticals, autos. More than a quarter of the whole population of Singapore is on Carousel. Our end-user traffic is close to 10,000 to 15,000 queries per second. Carousel started working with Cloudflare to address their immediate DNS needs, but quickly expanded their usage in order to continue serving web content quickly as their user base grew. We started off as a solution for our DNS and SSL termination requirements, and then we sort of explored the product further, and our usage grew after that. We started exploring the ability to cache our assets at Cloudflare, which we found quite good. In the past three years, we have moved all our caching requirements to Cloudflare. Carousel also adopted Cloudflare security features like the web application firewall after noticing vulnerabilities in their existing firewall. As someone who would run a public property on the Internet, there is a constant threat of adversaries targeting you for DDoS attacks for various purposes. The firewall which we had existed at that time was, there were a few lacunas, and we wanted to have a solution which was much more comprehensive in nature. We explored the web application firewall offered by Cloudflare, and it ticked all our boxes. It had the basic OWSP security related rules as well as various other specialized rules that Cloudflare themselves added, and we also had the ability to add custom rules. So all of these features essentially made it a perfect fit for our need. As a thriving e-commerce platform, Carousel understands that its Internet security and performance must work hand-in -hand. The company cannot allow its security posture to impact site speed and availability, especially when flash sales attract sudden influxes of valuable traffic. One of the biggest benefits that we get out of Cloudflare is that the security is nearly free of cost when it comes to performance. So user experience doesn't degrade with all these checks that are put in place before a particular request is served from our origin. So that is one of the benefits that we definitely see. In the past three and a half years of using Cloudflare, they have solved our problems around the various areas of handling traffic, in caching, in security, reliability. Cloudflare is very key for the experience that we offer to our users. With customers like Carousel and over 25 million other Internet properties that trust Cloudflare with their security and performance, we're making the Internet secure, fast, and reliable for everyone. Cloudflare. Helping build a better Internet.