Yesterday, Today on the Cloudflare Community

Welcome, everyone. Welcome to Yesterday Today on the Cloudflare community. I'm your host, Tim Cloonan. Join us every Friday for a new edition of Yesterday Today. Each week we start with This Community Day and the Community Traffic Report at 10 a.m. Pacific, followed by a review of top community issues from last week, the ever informative Using the Community Tip, occasional interviews with Cloudflare MVPs or support engineers, and every week we conclude with In Class with Cloudflare, where we learn a few things in the community. Before we get started, I want to mention that our episode next week will be on Thursday. We'll return to our normal day and time after our summer hiatus. We'll be back on Friday, July 10. Turning to This Community Day, the most popular area for discussion in the community last week was DNS and network, with questions about editing DNS records leading the category. Other popular areas for discussion were Getting Started with 502 errors and name server issues leading the category. Overall, community traffic was seasonally flat versus the previous week, with posts up slightly and new topics holding steady over the prior week. The number of new customers joining the community last week was flat week over week, but remains up dramatically for the year. Last week, the most popular areas for discussion on the community was up dramatically over the year. Last week, the top three searches on the community were for 1016, 523, and questions about orange and gray clouding DNS records. Visitors to the community were looking for assistance on topics with a lot of self-help resources that are available. Those resources are available if you know where to look for them, how to search for them, and when and where and how you reach out for assistance. So today, we're going to talk about how to find self-help resources like tips, tutorials, learning center, as well as the help center, and what tools to use to be able to troubleshoot them and how to interpret the results of those tools. Before we dig into the issues that we're going to be talking about today, I'm going to be referencing a number of sites, posts, tips, and tutorials throughout the show. Go to the link that's shown on the screen to follow along or to reference it later. We'll work through the issues based on the popularity of the searches, starting with the 1016 and the 523 error, and then we'll go in and we'll talk about some of the other additional resources that are available to help you. Welcome to the Cloudflare community. The Cloudflare community is open for all Cloudflare customers to use. Membership is free, and folks come here and ask questions as well as trying to solve problems as well as trying to improve performance. There are two search options within the community. There's a quick search that's located in the top right, and under the quick search, you actually have an options menu that takes you to an advanced search. But before we get to searching for the issue, before you determine what to search for, you actually have to figure out how to ask the correct question to elicit the response that you want. We talked about this on last week's episode with mixed content. If you know what it is, you probably already know how to fix it. But because mixed content is an unknown term or an unfamiliar term, and it's boot to boot, it's indicated differently with different browsers. Visitors to the community think about and search for mixed content in a whole bunch of different ways. That usually confounds the search results, and it ultimately confounds the visitor as well because they don't get the results they need to fix the problem. Luckily, that's not the case with these two errors. We have a specific error code, and we can search based on that error code. So I'm going to go through. I'm going to do a quick search and find the error. So if I click switch, select quick search, it actually even gets easier because it tells me that I don't have to wait for reply. I can search for the community tip and the error and get a result. So I'm going to take the tool tip up on that advice. I'm going to search for community tip and 1016 error and see what results we get. The search results found a couple of different options. It found the community tip, which is exactly what I was looking for with ideas on how to solve the problem. It also found something that I'm going to remember for later, which is a reference to all of the published tips. And that's a great catalog to keep handy if you're wondering what resources are available. But today, we're going to spend some time looking at this community tip and following the troubleshooting techniques to be able to fix the problems that we're having. If we go into that community tip, the 530 1016 origin DNS error, I can see that it's some sort of a problem with either my A record or my CNAME record. Now, I'm not, this is a just a very, very simple demo site that we're talking about today. So I don't have any of these fancy performance features like the load balancer. So I know it's probably not going to be a load balancer issue. It really is going to be either my DNS record or what looks like my CNAME record. So I'm going to start to dig into those to try to understand exactly if that record is working or not. This is actually kind of an interesting issue that it's a DNS error because that explains a lot and makes me understand a lot of why DNS record editing was also a very, very top category for search last week on the community. We saw the search for the two different hours, as well as a bunch of discussion around how do I edit and change my DNS records. It could be that all of that traffic is related and trying to answer the same type of questions. So what we're going to do is we're going to take a look at how do we figure out how to test that A record. Now, if you're in logged into your dash, you can see the A record in the DNS app of your Cloudflare dashboard. We're going to look at that later, just a little bit later on in the episode. It ought to have an orange cloud next to it that indicates that it's proxied through Cloudflare and the A record should have a value that points to the IP address of your origin server. So those are a couple of things that we want to be able to make sure that we're checking for, but here's a way that you can check it outside of the actual Cloudflare dashboard. I'm going to go into a terminal window. If you watched our earlier episode, we looked at broken SSL. In that episode, we curled into a site with the curling into the broken SSL site, and we're going to call up that command again. So what we'll do is we'll go ahead and we'll curl into and actually we see now, we see the NGINX indicating that this site is probably proxied through Cloudflare. We also have another example that we can take a look at where we're not going to get the NGINX response. We have a site that's proxied through Cloudflare, and we're going to not proxied through Cloudflare. What we're going to do now is we're going to dig in a little bit deeper with using these two sites as our example, and we're going to actually dig into the A record to see if we can find out if the site has a value that's resolving to Cloudflare. So I have entered these commands earlier. So let's go in and go through and dig into the first site. And here I get an IP address that I don't necessarily recognize, but what I want to do is I'm going to compare that against the other site to see if I can find the correct IP address. Now this is kind of interesting as I have two different sites that I've just issued this dig command against the A record. The command came back and it gave me a result. One of them gave me this IP address of 104. The other one gave me this IP address of 185. 185 doesn't sound familiar. The 104 does sound familiar. So what I want to do is I want to find out a little bit more, and I want to verify that this is who owns this IP address so I can see if it's actually a site that's resolving through Cloudflare. Now I can search pretty easily and I can find a whole list of Cloudflare IP addresses and a page that references that. You can find it on the community. You can find it through a simple Google search. But I don't really want to go out and do a search. Right now what I want to do is I just want to be able to understand if indeed this is a Cloudflare IP. So taking the IP, I'm able to actually issue a whois against that IP to understand who owns that. And yep, that's being run through Cloudflare. So now that tells me a lot because now I'm able to understand that I have an A record that's resolving and it's actually working. Inside of the dash what that's going to look like is it's going to have an orange cloud next to it. It's going to have the IP address of my origin server and it's going to be and it'd be right proxied through Cloudflare. Well, that's actually really, really helpful because now I know that I have an A record that's working. So now the question becomes, is there an issue with my CNAME record? Which was again, if you remember going back to the tip, that was the other issue that I could potentially have, that I had a CNAME record that wasn't resolving. So again, I'm going to stay in the terminal window and what I want to do is I'm going to look for the CNAME record. But first, to see what a good response is and what a bad response is, I'm going to look for a CNAME record that I know doesn't exist. So again, I'm going to dig into the site. I know that the CNAME record called Nope doesn't exist. So I'm going to dig against that and see if I can get a response. I don't get any sort of response. Now it might be difficult to understand and interpret the results that you're seeing here. A good way to understand if it's actually working or not working is to issue the command against something that you do know is going to return a response. And for example, let's go against the www CNAME record. So let's bring back that command. And here, I know that I don't have a CNAME record called Nope, but I'm pretty sure I have one called www and I'm pretty certain that that is being proxied through Cloudflare. And indeed, it comes back with the IP addresses. Again, I can verify those IP addresses online, or I can verify them in a Whois in the terminal window, and it's going to show a valid Cloudflare IP. Now that's troubleshooting the 1016 error, making sure that my records are actually working. But let's go back into that tip and make certain that we've addressed all of the questions that we have here. So I don't have a load balancer. I'm not worried about checking that. My A record is actually resolving and is coming through Cloudflare. And my CNAME record is valid and can be resolved as well. I'm running the Linux system. If you're running something else, you may want to use other commands and the commands may be subtly different, but the same body of commands are going to be available to you. There's also a whole series of online sites that you can go to. So rather than going through a terminal window, if you're unfamiliar with how to use the terminal, or you don't want to use the terminal, there's online sites that can perform a lot of these tests for you with the same kinds of results. And there are tutorials to understand how to edit the DMS tab and edit the DMS records. What we're going to do is we're actually going to look at that in a little bit more detail as we go through. So let's dig into that 523 error. So the 523 error, what we're going to do is we'll actually go through, we search for community tip in 523 using the suggestion that we saw on the tool tip, and it returns a couple of results. Again, we're seeing this catalog of all published tips, which I really encourage you to remember, because that's a very helpful one. And we see the 523, the origins unreachable. If I look at the origin unreachable error, this says that it's also an issue with my DNS settings. So now this explains a whole lot. We had a bunch of questions about 523, we had a bunch of questions about 1016s, and we had a whole lot of inquiries around how do you edit the DNS records. What this tells me is that a lot of the community activity last week was focused on these issues, and it wasn't actually this range of things that we were seeing people ask about. It was really focused in on some very, very specific things. So let's go dig through this tip and see how we resolve it. The quick fix ideas that are here are what's worked for most customers most of the time. So what we've done is we've actually gone through, taken all of the troubleshooting resources, boiled them down to figure out how are most folks able to solve the problem when they raise this question. And so these are ideas that are taken directly from the community, directly from our customer support team, directly from suggestions from other customers. So in this instance, let's go through and look. It says that I have always online is triggered and it's triggering some sort of a page. So we can go into the site and we can verify that. I know that I'm not running always online, so what I'm going to do is take a look at the next idea and the quick fix ideas. So I need to understand the IP address for my A record and make sure that that's correct in the Cloudflare dashboard. So now again, we're talking about the A record that we were looking at with the dig from the terminal command. But to do this, we need to look at it a little bit differently. We need to look at it from the Cloudflare dashboard DNS tab to verify the value of the A record's correct, not just that there is a value. Now, in order to be able to do this, I need to know the correct value of my IP address with my host. Or at least in this case, I need to be able to identify a bogus IP address. So when I log into the Cloudflare dashboard and I navigate to the DNS tab, I'm presented with a number of different records. I can see some things from looking at these records. Now, I don't know the correct IP address for my host right off the bat. I have to contact them and find that out. But I do know two things. One, I know that the first IP address is a Cloudflare IP address because it looks really similar to the one that we were looking at in the terminal window. I also know that this is probably a valid IP, but probably not my valid IP. That'd be a fantastic one to have, but it's not mine. And it looks like an example IP that was inadvertently entered. So those are probably the root of my problem, although they are being proxied. So in order to do this, I'm going to have to be able to understand and know my IP address, like I say, at least be able to spot these bogus ones right off the bat. If I go back into the tip and I keep on scanning the ideas in terms of how we're going to troubleshoot this, I know that I'm not running a virtual private server. So it's probably not an issue with my VPS. It may be a routing issue that's happening between the Cloudflare systems and the web server. And so there may be some information in that RAID or the Cloudflare ID that you have that you want to be able to take to Cloudflare support. One of the ways that you can check this is by running a trace route. Now, the issue with a trace route is that you really want to run it from an affected machine. Me running a trace against a specific system isn't going to tell you a whole lot if I'm not having problems reaching your site. But what I want to do is I actually do run that trace. So how you run the trace varies depending upon what kind of system you're on. Now, I am on the Linux system. If you're running a Windows system, there are different commands that you would use and different ways of getting to the command. Like I say, I'm running Linux. So I want to run a trace route like this to be able to find out where this connection is being broken. Again, what I would do is go back to my terminal window and I've run the trace before. So I'm going to call up and I'm going to bring up the trace. So now I'm tracing down to this site. Now, I know this connection is good and all of the results are going to be valid. So I'm not actually going to see the break. If you were running this against the site where your customer was running it against your site because they're having an issue, they would actually see that the packets weren't actually handled properly and that the trace didn't work. Knowing where it's breaking helps us to determine where the locus of the problem is because that's an incredibly helpful tool to use. So the trace is going to be there in order to help us figure more things out about how we diagnose and solve the problem. Now, in this instance, we've already solved it or we really understand the issue is that I've got this 104 IP address, which is a Cloudflare IP. That's going to result in an error 1000 DNS points to a prohibited IP. So if you ever see that error message, you know right off the bat that you put in the Cloudflare IP versus your origin IP as the correct IP address. So what I'll do now is I need to contact my hosting provider. I'm going to ask them about the IP address of my origin server, and I'm going to actually edit this record to enter that value. Now, to edit that record, what I would do is I would click on the IP address, and here I have the opportunity that I can enter in the new IP. So we'll use that example IP just as a placeholder reminder to me that I need to go back and change that at some point. Now, I know that's not a valid IP for me, so I'm going to turn the proxy off. Click an orange cloud to turn it to gray. Click a gray cloud to turn it to orange. This question comes up quite frequently, and knowing to click on the clouds is simply the easiest way to be able to change the status. What we'll do is we'll go ahead and save that IP address. So now I've edited that IP, and it says that I've already entered that IP address, so we don't have to worry about it. So what we'll do is we've entered that name. So what we'll do is we're going to go through, and we won't actually even worry about saving the IP, but that's how you actually go through and edit the IP address, and it's also how you go through and change from proxied to not proxied. Those questions come up pretty frequently and actually addresses the third most popular search area that we had last week, which was how do I edit my DNS records once I know what I should be editing them to. So we've run the trace command, and we've been able to determine that we have a break. We've also been able to determine how we can edit our records to fix the problem that we're having, and we were editing and changing our records from proxy to unproxied. What we've learned today is we've learned how to address a couple of the main issues, and there are a whole bunch of other resources that are available on the Cloudflare community that we want to be able to talk through today. So very, very specifically, one of the questions came up was talking about community tip, and the question that came in was specifically around what tips are available and how frequently do the tips come out. So we have actually a catalog of the community tips that's come up a couple of times. Let's take a look at that. So here's the catalog of all of the published community tips. This is updated every time a new tip becomes available, and they're broken down by the type of tip. So if you're going through and you're troubleshooting a specific error, you have tips that are associated with that error message, whereas if you're going through and you're trying to figure out how you can optimize your performance on Cloudflare and tweak the performance on Cloudflare, we have a whole series of best practices that address how you can do certain specific actions within the community. So the tips are always a great resource and a really, really helpful resource for finding details. Other resources that are really complementary, the Cloudflare community tutorials. The community tutorials address a lot of the common functions that folks hit when they come in, and for example, in this issue, how do I edit my DNS records and how do I add my DNS records is actually covered in the tutorials. And the tutorials will go through based upon where you are, and we'll talk about how you move to the solved state of being able to do the edit or add the additional records that you need. Tutorials are really helpful. Tips are very, very helpful. The other thing to check is that to understand that perhaps it's not you, it might be us. And the way you do that is through Cloudflare status. Now, if you click to the Cloudflare community, you can get to the status through the homepage. We also record out the Cloudflare status in the status category where we're providing instant updates from the Cloudflare status site. Or you can go directly to the Cloudflare status site, Cloudflare status site itself at Cloudflarestatus.com, and you can understand if there are issues. So a lot of times we'll see issues where folks are posting, I edited my DNS record and the update didn't happen. It took too long to propagate. And what we'll sometimes find is that there was an issue with DNS records being slow to propagate, or perhaps the data isn't being routed through the data centers that you would think. And we'll go through and check Cloudflare status and find that some of the data centers may have been offline at that time that you were asking about. So it's always a good idea, particularly if the issue that you're facing seems odd or confusing, to give a look at Cloudflare status. You can also go through and you can subscribe to the RSS feed for the status so that you're getting the updates sent to you. Or from within the community, you can watch the Cloudflare status category and have the updates sent to you as well. That's a good way of staying proactively informed about the status of the system so that you're not chasing errors that really aren't errors with your specific site. The other resources that I wanted to mention today are the Cloudflare Learning Center, as well as the Cloudflare Help Center. The two sites have different focuses, and the Help Center is what it sounds like. It's solving your problems, filing tickets with Cloudflare support, reading a bunch of deep dive articles on how you use specific features within Cloudflare. The Learning Center is completely different. The Learning Center is really focused about how do we impart basic information and basic knowledge. It's not necessarily about Cloudflare, but is about the tools that Cloudflare offers, and understanding how those tools work, why they're necessary, and understanding a little bit of the background behind the types of things that you're doing with your online property. The Learning Center is really targeted at folks that are coming up to speed and becoming more and more familiar. If you're an expert or very well-versed, you're a network engineer that runs a lot of different sites, the Learning Center is actually still a really valuable place to reference for great articles because it has a lot of information that forms some of those prerequisites that you need to know before you move on to understand the next thing. I'd encourage you to look at the Learning Center. It's actually quite a fun site and an informative site. The other piece that you want to be aware of is the Cloudflare Help Center. A question often comes up when we're in the community, which is, how do I get help from Cloudflare? The Cloudflare Help Center is here. Going through and searching the Cloudflare support knowledge base that's on the Help Center actually gives you information about specific functions within Cloudflare, and helps you to understand how you're going to be able to use those better, and is a really necessary site to keep bookmarked. This is a place that you're going to want to go back to pretty frequently. The last piece that we want to take a little bit of a look at today is we do want to take a look at expert tips. Expert tips on the community are posts that have been made by other Cloudflare customers that get the idea right. These are really, really important as a spot to look at, because you can use the expert tip tag, and you can actually search for specific issues. I was having issues with my DNS records. I wonder if there are any really great posts about DNS records. I'm going to search for expert tip in DNS, and I actually find some information about the DNS records. I'm going to search for some great posts about that specific issue. There's some DNS cache poisoning, attaching to specific sites. Do I change my Cloudflare DNS IP to the one given by my hosting company? I haven't read that post, but I'm going to say the answer is probably yes. You can see that there are a whole lot of ways that you can use expert tips to be able to inform how you're actually getting information from the Cloudflare community. All right, so we're running up to the end of our time. We have no more questions that have come in. I'd like to thank you for joining us for Yesterday Today on the Cloudflare community this week. I'm your host of Yesterday Today, Tim Clunan. I'm the Cloudflare community manager. Join us next Thursday for another edition of Yesterday Today on the Cloudflare community. We'll be on at 10 a .m. Pacific. We look forward to seeing you next week, and until then, we're going to see you on the Cloudflare community. So we have seen malicious foreign actors attempt to subvert democracy. What we saw was a sophisticated attack on our electoral system. The Athenian Project is our little contribution as a company to say, how can we help ensure that the political process has integrity, that people can trust it, and that people can rely on it? It's like a small family or community here, and I think elections around the nation is the same way. We're not a big agency. We don't have thousands of employees. We have tens of employees. We have less than a hundred here in North Carolina. So what's on my mind when I get up and go to work every morning is, what's next? What did we not think of, and what are the bad actors thinking of? The Athenian Project, we use that to protect our voter information systems. and allow it to be securely accessed by the citizens of Rhode Island. It's extremely important to protect that and to be able to keep it available. There are many bad actors out there that are trying to bring that down, and others trying to penetrate our perimeter defenses from the Internet to access our voter registration and or tabulation data. So it's very important to have an elections website that is safe, secure, and foremost accurate. The Athenian Project, for anyone who is trying to run an election anywhere in the United States, is provided by us for free. We think of it as a community service. I stay optimistic by reminding myself there's a light at the end of the tunnel. It's not a train. Having this protection gives us some peace of mind that we know if for some reason we were to come under attack, we wouldn't have to scramble or worry about trying to keep our sight up that Cloudflare has our back.