Yesterday, Today on the Cloudflare Community

Welcome to Cloudflare TV.

Welcome to yesterday today on the Cloudflare community. I'm your host Tim Cloonan.

Join us every week for a new edition of yesterday today. Each week we started with a review of the popular topics on the community with this community day and the community traffic report at 10am Pacific, followed by a review of top issues from last week, the informative using the community tip, occasional interviews with MVPs or support engineers, and every week we conclude with in class with Cloudflare where we learn a few things about the community.

If you have questions, send them to the address shown at the on Cloudflare TV would be happy to address them during the episode today.

Turning it to the community. The most popular area for discussion in the community last week was performance with questions about routing leading the category.

Other popular areas for discussion was security with questions about SSL certificates leading the category.

Security and performance were also the most popular categories three weeks ago, although their positions were reversed.

Overall traffic remains seasonally flat versus last week with with new topics and posts holding steady over the prior week, the number of new customers joining the community was flat week over week remains up dramatically for the year.

Last week, the top three searches on the community for 524 hours and X domain hours and the security are no cipher overlap.

Last week, visitors to the community asked the exact same questions in the exact same categories that we saw three weeks ago when security and performance for the top two categories.

Three weeks ago, we opted not to devote time to the 524 error as it was just not that interesting of an error to talk about for 30 minutes.

In fact, we actually skipped over the DNS error about NX domain and the security error about cipher overlap as well.

Today, those errors aren't any more interesting than they were three weeks ago.

There's a lot of resources available.

If you have questions about those errors stopped by the community to troubleshoot the issue and be happy to help.

Today, we're actually going to talk about something that benefits a lot of people is an incredibly powerful feature of Cloudflare and while potentially confusing is ultimately really, really easy to use.

You just need to understand some of the basics. We're talking about page rules.

In fact, if you visit the community, you'll see that folks have been talking about page rules already this morning, asking a couple of questions that we're going to be answering in today's episode.

You can define a page rule to trigger one or more actions whenever a certain URL pattern is matched when people visit your site.

The page rule app is available on all plans on the Cloudflare dashboard.

The number of page rules you have varies by plan type. Page rules, and this is the most important thing to understand as you get into them, require that there's an orange clouded DNS record for your rules to work.

We've looked at orange clouding record in the previous two episodes, so you need to have that orange clouded record to make certain that your page rule is going to work.

Before we dig in to the whole page rule discussion, we're going to be referencing a number of posts, links, and tutorials throughout the presentation.

Follow the short link that you see on the screen to be able to look at those assets, follow along during today's presentation, or look at them later on.

Next, let me welcome back to the program today, Dom.

Dom is one of our community MVPs and is coming back today to talk to us about page rules and answer some of your questions.

Hey, Dom. Welcome back to Cloudflare TV.

How are you? Thanks, Tim. It's great to be back. Fantastic.

Hey, do you want to share? We see a lot of questions about page rules, and in fact, you just pointed out we had two questions that came up just this morning.

They were talking to some of the issues that we've been discussing as we prepare for today's episode.

Do you want to share a bit of the questions that you see on the community about page rules and how you approach them?

Yeah, so the first thing is to understand the basics of what a page rule is and what it does, and it gives you the ability to control Cloudflare settings on specific URLs or host names, and you can find your page rules in the dashboard under the page rules app.

And you can see the option to create a new page rule and also a list of any existing ones and change the order of them here.

The next thing that is most commonly asked about is about wildcards and what they do and how to use them.

And wildcards are really good to use in page rules because they allow you to match more than one page at a time.

So you can create a page rule that just matches one page of your website, but it's generally best if you've got a section of your website that you want to match, that if you use wildcards, then you can tell Cloudflare a specific section that you want the page rule to apply to without having to specify each URL.

So if I just share this, then as I said, you can find page rules in your dashboard, going to the page rules tab, and you'll see a list of the page rules you have and the create page rule option.

If we start looking at wildcards, this is a fairly standard page rule that will just match one specific page.

So in this case, I'm putting in the URL match box.

So tell Cloudflare which URLs I want this page rule to match.

So in this case, I just want it to match forward slash page in this case.

And once you select the settings that you want down here, you can add several settings usually.

And once you select the settings you want, this will apply them to just the page that you specified on both HTTP and HTTPS.

Unless you specify the protocol in here, then the rule will match both.

But a lot of the time, you may want settings to apply to a whole section of your site.

And this is where you can use an asterisk as a wildcard.

So in this example, you can see that I have got an asterisk in my URL here, and I've got the path as forward slash A asterisk Z, which means that it will match any path, any subdirectory beginning with A and ending with Z.

So the wildcard just says match anything between these two.

So if it starts with an A and ends with a Z, then it's going to be matched.

So it would match A, B, C, X, Y, Z, for example, in the path. Then you can use wildcards more generally like this, which will match example site1.cf as the root domain and any subdirectory.

So it would match the forward slash foo, forward slash foo, forward slash bar.

It would match any directories below the root domain as well as the root itself.

And then if we add another asterisk at the start, we can then match the root domain and any subdomains with subdirectories as well.

So it would match both example site2 .cf, but it would also match subdomain .examplesite2.cf, and it would match any paths on those as well.

And then there's a fractional change between that one and the next one, which is we just added a dot between the asterisk and the domain name.

And what that does is because we've then added the dot, that is saying that the page rule will apply to any subdomains, but not the root domain itself.

So it wouldn't apply to example site2.cf, but it would apply to any subdomain of that site.

And you can use asterisks to match a huge number of things.

Like in this example, this would match any PNG files on the website.

So by putting the asterisk followed by the file extension, it will then match any files that end with .png.

So that is really a summary of using wildcards in page rules, which is one of the most important things to get your head around before you start using them.

That's pretty cool. So the next thing that is really common use of page rules is to use the forwarding URL setting, which creates a redirect.

And we'll talk about the principle of redirect and then some specific common use cases.

You can normally perform redirections on your web server, but it's generally easier and it requires less technical knowledge to configure them in page rules.

And it's faster as well because the visitor doesn't need to connect to your origin server to get the redirect.

So when redirecting, one of the most important things to remember, as Tim mentioned earlier, is the need for a proxy DNS record where you want the redirect to work.

Although the redirect may mean that the request will never hit the server specified in your DNS record, you must still have the record there and proxied for the page rule and therefore the redirect to work as you expect.

So for redirections, you want to pick the setting forwarding URL, and then it will give you the option as to whether you want a 301 permanent redirect or a 302 temporary.

And search engines are generally the main reason why you would want to choose one over the other.

The search engine will replace the old page with a new one if you use a 301, but it might not with a 302.

So it's often useful to use a 302 temporary to test a redirect.

And then once you're happy that it's working as expected, you can then switch it to a 301, which is permanent.

And so you get this additional option when you select forwarding URL, which is down here, and this is the destination URL.

So in this example, we've just entered one URL to redirect from and one to redirect to.

So here we're redirecting example site 3.cf slash foo to example site 3.cf slash bar.

So this is just a simple we want to redirect this page to a different page.

But you might want to redirect traffic from a set of pages from an old domain to a new one or from one section of your site to another.

A common use case for this is if you have, for example, you used to have a blog on your site that was on yourdomain.com forward slash blog forward slash the post name.

But you're moving that to a subdomain of blog.yourdomain.

So you don't want to have to create a page rule for every single blog post that you want to redirect.

So this is where wildcards come in again, we mentioned earlier, and you can use wildcards in forwarding URL page rules.

So in this example, we are forwarding everything on example site 3.cf to the root domain and the any path after that as well.

And the extra thing that you can do with forwarding URL is this dollar symbol and a one here.

And what that does is it means that whatever the path up here is where you've got the asterisk, it will then put that path down here where you put the dollar one.

So in this example, we're just redirecting the root domain to the www subdomain.

This is if your site is using www and you just want to redirect visitors there.

And it's useful to be able to keep the path.

So if they go to example site 3.cf forward slash page, then you don't want it to just be redirected to the main site.

You want them to go to the page they wanted just on the www subdomain.

And so that's what this dollar one here does. And if you have multiple asterisks in the forwarding URL, you can use $1, $2, $3 to match each one in the order they are.

It doesn't just have to be used on a path. Let's say that you had multiple different pages in a site directory on your domain, but you're moving into subdomains.

So if you had example site forward slash sites forward slash the site name, but instead of having slash site slash site name, you want the site name to be its own subdomain.

And so you want to redirect anyone that visits the old URL to the new one.

And so what this does is that you can see we've got two asterisks in here.

So it's going to match both the root domain example site 3 and it would match subdomains as well.

And we've got a second asterisk here to say we want to match anything within the site directory.

And so in this case, we want to forward it to a subdomain.

So we put $2 here instead of $1 because it's the second wildcard in our URL match.

So therefore we use the two in the forwarding URL as to where we want to send that to.

So if we went to forward slash sites forward slash site 1, it's going to redirect us to site 1 .examplesites3.cf in this case.

Wow, that is really, really impressive.

It seems like if I learn how to use wildcards, I get a whole lot of flexibility with the page rules.

I mean, even if I just have a free plan with three rules, if I can plug in wildcards, then all of a sudden I can manage a lot of the corner cases without having to write one -off corner case rules.

I think that's going to become increasingly important as people's sites grow and they become more complex.

It would seem like being able to match patterns as opposed to discrete URLs is going to be a heck of a lot more efficient in terms of maintaining the page rules.

This also starts to then beg into the question of performance and cache control, because I know that that comes up a lot with page rules as well.

Do you see confusion around those issues with page rules?

Yeah, we get a lot of questions on the community about page rules and caching.

So by default, Cloudflare is going to cache static files such as JPEGs, CSS files, PDFs, but it's not going to cache HTML or other dynamic content on your pages that is likely to change regularly.

But you can use page rules to customize caching behavior and you can use wildcards, like we discussed earlier, to match sections of your site or specific file extensions.

And you can choose the setting cache level, like I have here.

And then there are a few different options that you've got.

If you don't want Cloudflare to cache anything, then you can bypass the cache altogether if it matches the URL in the page rule.

You've got different settings to do with query strings.

If you've got query strings on your URLs and you want the caching to ignore the query string or get rid of the query string, then you've got those options.

Or the option that we see a lot of questions about is the cache everything.

And so for this page rule, the example site 4.cf, we've used the wildcard again here.

So we're covering the entire site.

And by telling Cloudflare to cache everything, then what we're telling Cloudflare to do is to cache the HTML, to cache the entire website, which can be good.

It should speed up your site if Cloudflare caches the whole lot because requests don't have to go as far as your server.

But you should be aware that telling Cloudflare to cache everything does mean that you could end up showing stale content or showing users an admin area they shouldn't see.

And so one of the questions we get is about why content's not updating.

And it's often because people have configured a cache everything rule, and they're either not sure of how they've configured it or they've misconfigured it in some way.

That means that things they don't want to be cached are being cached.

So let's say that your site has got a news page.

And if you cache that news page, then new content that you post there is not going to be reflected because Cloudflare is still caching the old version.

And so you just need to be aware of what you're setting cache everything to.

And so in this example, we've got two page rules on this demo site.

We've got a cache everything that's the rule I just showed you that is going to cache everything on the entire site.

And then we've got a rule here to bypass the cache for anything in the admin directory.

So this is a way around that. You can set cache everything on your site, but then you can set another rule to bypass the cache on specific areas like the news section or the admin section.

You just need to be aware of the order of your rules here.

But no, cache everything is something that can be used really nicely with page rules as long as you make sure that you're only caching things that you want to cache.

That's pretty cool. So, hey, Tom, a couple of questions came in on the email to Cloudflare TV.

You can kind of maybe talk to them as you'd like to.

But one of the questions that came up is something that we do see on forums.

I hadn't seen it for a couple of days. Using wildcards to change global settings like SSL, specifically the question was, can page rules override always use HTTPS?

And then the other question that came in is checking if things are actually cached.

So we've got a couple of different directions.

I don't know if you want to talk to the cache question first or if you want to talk about redirecting for security.

Yeah, I'll go for the first one on overriding the settings.

And those kind of questions do come up pretty frequently. And it does depend on the setting as to what you can override with a page rule and what you can't.

So in most cases, settings can be enabled or disabled globally. And then page rules can be used to the opposite on specific areas of the site.

And the page rule setting would override the global setting.

So say, for example, the automatic HTTPS rewrites feature that helps with mixed content.

I could have that disabled globally, but enable it on a specific area of my site in a page rule.

Or I could do the opposite and have it enabled globally, but disable it in a specific area.

But with the specific question about the always use HTTPS setting.

So globally, you have a toggle option to turn that setting on or off. But in page rules, it's a statement.

So it's a statement that is on all the time. So if you add the setting, always use HTTPS, then that's going to be on.

You don't have an option to disable it if you've got it enabled globally.

So if you need always use HTTPS with different settings on different areas of your website, you'd need to disable it globally and then enable it where needed with a page rule.

And so you can use like a couple of page rules to enable it in certain places and disable it in certain places.

But you need to make sure that you've got the rules in the right order again for them to function.

And to make sure that with always use HTTPS specifically, that you disable it globally and then enable it with a page rule and not the other way around.

And with the question about caching and how to check if something's cached or not.

The first thing is if your site is active on Cloudflare and you haven't got development mode enabled, because that will disable caching.

You can fairly easily see if a file is cached on the Cloudflare edge.

And you can do this by looking at the response headers. And this can be done either in your browser developer tools on the network tab or use your curl command.

So I'll just show you how to do it on the command line. So this is a terminal that I've got.

And I'm going to use a curl command. And I'm going to use dash I because I just want to see the headers.

And so if I send a curl request to this demo zone that I've got, then you will see this header that is CF cache status of miss.

And what that means is that Cloudflare was instructed to cache this file, but it didn't happen to be in the cache at the time that I requested it.

And so if I send a second request, you can see that it now shows CF cache status as hit.

So the first time it was a miss because it didn't happen to be in Cloudflare's cache.

But once I requested it, Cloudflare then held that in the cache.

So when I requested the same URL a second time, it then shows as a hit. And so those are the two most common, that you either get a hit if it's served in the cache, you get a miss if it should be cached, but it's not currently in the cache.

And the other one is if you've got a CF cache status of dynamic. And that means that Cloudflare was instructed not to cache that file.

So if you keep requesting it, it's still not going to be.

And that might be because it's not in Cloudflare's list of files that it caches by default, or it might be that you've specified that you don't want that to be cached.

And so for this example, if we look at the page rule configuration that I've got here, the bypass one, which was the command I sent second, which gave the result of dynamic, I have set the cache level to bypass on this because PNGs would by default be cached by Cloudflare.

But for whatever reason, I don't want this file to be cached.

Maybe I change it frequently.

So I've set the cache level to bypass. And so when you look at the headers, you can see a few different things, hit, miss, and dynamic are the most common.

There's a great article in the Cloudflare Support Center at support .Cloudflare.com that lists all the different answers that you might get, all the different header values that you might see there and what each one of those mean.

That is really, really cool.

You had commented, and I was just confused for a second as I was watching the screen and listening to you.

You had commented about rules having an order.

And I've seen folks that seem confused by this on the community.

This has come up a couple of times. Is that a cosmetic thing or rules ordered alphabetically?

What does it matter? I mean, can't I just write them as I think of them?

So the priority of page rules is really important. And it's not just a cosmetic thing for you to see which rules you put at the top because only one page rule will activate on a page.

So if you have more than one rule matching the same URL, the higher one up the list will run.

This generally means that you need to place more specific rules above less specific ones.

So in this example that I've got here, I've got three page rules here.

I've got one that is affecting everything in the admin directory.

I've got one that is affecting the entire site. And then I've got one that is affecting a specific page within the admin directory.

So right now, if I send a request to forward slash admin forward slash page, even though I've got a specific page rule for this, it wouldn't get that far.

So this page rule is never going to be activated because it's also matched by this page rule here, which is to match everything in the admin directory.

And so these rules are in the wrong order because this rule down here is never going to be activated.

And so, as I said, you need to put the more specific rules above the less specific ones.

So in this case, I need to reorder this to be most specific first. So in this example, I have a specific page within the admin directory as my first rule now.

Then I have the entire admin directory as my second rule. And then I have the entire site as my third rule.

So by having them in this order, it means that if I hit slash admin slash page, the correct rule, the most specific one for that page is going to be activated.

But if I have them in the wrong order, I could end up with the wrong rule activating.

So the order is really important to make sure that the right rule is going to fire and to make sure that you have the more specific ones before the more general ones.

Wow, that is actually really, really cool, Dom.

So I go from specific to general. So I have kind of this inverted pyramid sort of an approach.

What other things should I consider as folks approach page rule?

I noticed you're working within free plans there, so you're doing a lot with three rules.

What are the three things that folks should consider when they think about page rules?

Yeah, I mean, page rule is just so versatile because even with the three rules you get on the free plan, you can still do a lot with them.

And for me, the first three things, the top one would be checking that you have the DNS records proxied.

Page rules won't activate if a request is bypassing Cloudflare or it's not proxied.

So it's important to make sure you have your DNS records orange clouded for page rules to work, especially when you're configuring redirects.

The next thing to check would be the priority of the rules. As I mentioned earlier, only one page rule will activate on a request, so it's really important to get them in the right order, but the more specific rules before the more general ones.

And the final thing I would say is if you're using the cache everything setting in your page rule, is to make sure that you're not caching anything that you don't want to like admin or user pages.

And caching everything can really speed up your website, but it should be used with caution and make sure that you're only caching things that you want to cache.

So if I only have three rules, should I try to make the rules more efficient?

Should I buy more rules?

What's the approach I should take? Efficiency is always great, but it's not always possible.

So do it where possible and work within the limits of your plan if you can.

So if you've got three rules, then you can work with wildcards to match multiple URLs, and there's lots of great stuff you can do to minimize the number of page rules that you need.

And efficiency is great, but you should recognize the limits and buy more if you need to, because you can't do everything in three page rules.

And if you need more than that, then you are going to need to buy more page rules if you can't manage to simplify them enough to get them down to the amount you've got on your plan.

Or if you're really proficient, then try using a Cloudflare worker script to do the job instead of page rules, and you pay for workers and you can configure redirection.

You can do so much with workers, but redirects and page rules, settings, that kind of thing.

You can configure redirects in workers. There's template galleries for workers that you can go and check out.

If it's your first time using workers, you can check out the gallery for redirection templates.

If you're a bit more proficient, then try doing it in a worker script, which if you've got a limited number of page rules, that is likely to save you.

Wow, that is actually really, really cool.

I hadn't thought about using a worker for it. This is kind of changing the way I'm thinking about page rules.

I believe I was probably intimidated by the wildcards.

In fact, I'm pretty certain I was. But if I use the wildcard as a tool, then I can perform redirects, I can set security levels, I can control caching, or maybe I just need to get a bit more adventurous with workers, and that sounds like it might be potentially a topic for a future episode.

Yeah, sure. Thank you for joining us, Dom, and thank you for joining us in Yesterday Today on the Cloudflare community.

I'm your host of Yesterday Today, Tim Clunen, and a Cloudflare community manager.

Join us next week. We'll return for our normal day and time. We'll see you next Friday on July 10th for another edition of Yesterday Today on the Cloudflare community.

Until then, we'll see you on the community. Thank you. You You