One of Drupal's greatest features, and often mysterious underworking to many site builders and developers, are its caching layers especially render caching. We have all been there before: we have no idea why it's not quite working, so we press "Cache rebuild" (or more infamously, "Flush all caches" of the pre-Drupal 8 eras), and like magic, things are displaying properly. Whenever we have to resort to clearing the cache to fix a page, that usually means a render cache item was not properly invalidated when it was supposed to have been. This is caused by missing cacheable metadata. Cacheable metadata is the information Drupal uses to understand when a cache item should be invalidated by its tags or maximum age. It's also used to provide variations of the cache based on context.

Drupal's rendering pipeline is based on a render tree that contains multiple subtrees. Each subtree has its cacheable metadata calculated to cache the output of the subtrees into the render cache. Some other systems call this fragment caching. This process is very important for performance but also very hard to debug. With a debugging tool, it can be possible to understand render cache item hits or missed or if the subtree is considered uncacheable.

There are modules like Render Visualization that help. But in Drupal 9.5.0, Drupal has a built-in rendering debug setting to expose this information.

A quick cacheable metadata summary

Before moving forward, I would like to give you a quick summary of cacheable metadata.

• Cache tags are a way to label cache items and provide a mechanism for invalidating cache items in bulk when something changes.
• Cache contexts allow variations of cache items to be created based on context values – such as the current language or user permissions.
• Cache max-age defines the time to live for a cache item

Then there is the act of "bubbling." Sometimes when performing rendering, additional cacheable metadata is added by other components. The capture of the cacheable metadata is called "bubbling."

Some content may be considered uncacheable due to the content needing to be more dynamic. By default, this happens if there is a requirement on the user or user session cache context (unique caching per user) or if the max-age is zero (no time-to-live.)

What does the new renderer debugging provide?

The debug mode for the renderer does not alter the page's appearance. Instead, HTML comments are added to the page that annotates the following:

• When the renderer has started.
• The cache tags, cache contexts, and cache max age associated with the rendered output.
• The cache keys used for looking up a cache item.
• The cacheable metadata before combining bubbled metadata; this is the known cacheable metadata before rendering began.
• The end of rendering and time taken.

The output shows the pre-bubbling cacheable metadata

This does not disable the render cache bin; it is for outputting information about the render cache process. This is important because disabling the render cache would always result in a cache miss and not allow debugging cache hit rates or finding sections of the page that are considered uncacheable.

Let's walk through an example. Drupal has a demo installation called Umami. We will walk through the cacheable metadata for its Umami Home Banner block that links to a featured recipe and references a media image.

The following output was created after clearing the Drupal site's cache and visiting the homepage, so that each render was a cache miss. First, it shows when a render begins and if there was a cache hit or miss.

<!-- START RENDERER -->
<!-- CACHE-HIT: No -->

Next it shows the cache tags for the cache item. This part is very important, as each cache tag associated with the cache item represents dependencies of the cached render output. Any time these dependencies are modified, their cache tags as invalidated which will invalidate the cached render output. The header is a block content entity with a few text fields and an entity reference to a media entity for an image. The cache tags contain information from the block configuration, block content entity, media entity, the referenced file, display configurations, and the image style configuration for formatting the image. If someone were to edit the image style used then this cache item would be invalidated so that the new changes would be displayed.

<!-- CACHE TAGS:
   * block_view
   * config:block.block.umami_banner_home
   * block_content_view
   * block_content:3
   * config:core.entity_view_display.block_content.banner_block.default
   * media_view
   * media:18
   * config:image.style.scale_crop_7_3_large
   * file:35
   * config:core.entity_view_display.media.image.scale_crop_7_3_large
-->

Then it shows the cache contexts for the cache item. Drupal always adds the languages:language_interface, theme, and user.permissions cache contexts to all render cache items. This varies render cache based on the current language, current theme, and user roles. The block content entity has translation enabled, so the languages:language_content and languages:language_url are added to provided appropriate cache variations based on the entity's translations.

<!-- CACHE CONTEXTS:
   * languages:language_interface
   * theme
   * user.permissions
   * languages:language_content
   * route.name.is_layout_builder_ui
   * languages:language_url
-->

Then it shows the cache keys for the render cache item. These keys are used to construct a cache identifier, such as entity_view:block:umami_banner_home. This information is useful for reviewing cache items in the cache bin backend itself, like the database. These are not displayed on a cache item hit.

<!-- CACHE KEYS:
   * entity_view
   * block
   * umami_banner_home
-->

The cache max-age is displayed for reviewing the render cache item's time to live. The value of -1 means permanent. Using a max-age of 0 means uncacheable.

<!-- CACHE MAX-AGE: -1 -->

Next the pre-bubbling cacheable metadata is shown. This is the cacheable metadata when rendering begins. I find this to be one of the most important parts. It allows inspecting original cacheable metadata and then comparing what was bubbled during rendering. These are not displayed on a cache item hit.

<!-- PRE-BUBBLING CACHE TAGS:
   * block_view
   * config:block.block.umami_banner_home
-->
<!-- PRE-BUBBLING CACHE CONTEXTS:
   * languages:language_interface
   * theme
   * user.permissions
-->
<!-- PRE-BUBBLING CACHE KEYS:
   * entity_view
   * block
   * umami_banner_home
-->
<!-- PRE-BUBBLING CACHE MAX-AGE: -1 -->

Finally, when rendering is completed, the total render time is output. This is not displayed on a cache item hit since no rendering took place.

<!-- RENDERING TIME: 0.004593849 -->

The output still requires a certain level of knowledge to Drupal's inner workings, but I believe it helps demystify the process and expose the rendering process and render cache. I highly recommend reading the following article by Lee Rowlands of PreviousNext which explains how they solved a render caching issue with blocks by fixing missing bubbled cacheable metadata. It may be from 2016, but is still completely relevant. In their instance they had a similar page setup as the Umami demo, but in the footer of the site. If someone edited the block content entity, the footer wouldn't update! The way they solved it was by reviewing the debug headers in X-Drupal-Cache-Tags for things that were missing. That's like looking for a needle in a haystack, as the header contains all of the cache tags that were associated with the page.

I know browsing HTML comments may not seem the greatest, but it does beat sifting through some output like the following. It's the X-Drupal-Cache-Tags value from the Umami demo home page.

block_content:1 block_content:2 block_content:3 block_content_view block_view config:block.block.umami_account_menu config:block.block.umami_banner_home config:block.block.umami_banner_recipes config:block.block.umami_branding config:block.block.umami_breadcrumbs config:block.block.umami_content config:block.block.umami_disclaimer config:block.block.umami_footer config:block.block.umami_footer_promo config:block.block.umami_help config:block.block.umami_languageswitcher config:block.block.umami_local_tasks config:block.block.umami_main_menu config:block.block.umami_messages config:block.block.umami_page_title config:block.block.umami_search config:block.block.umami_views_block__articles_aside_block_1 config:block.block.umami_views_block__promoted_items_block_1 config:block.block.umami_views_block__recipe_collections_block config:block_list config:configurable_language_list config:core.entity_view_display.block_content.banner_block.default config:core.entity_view_display.block_content.disclaimer_block.default config:core.entity_view_display.block_content.footer_promo_block.default config:core.entity_view_display.media.image.medium_8_7 config:core.entity_view_display.media.image.responsive_3x2 config:core.entity_view_display.media.image.scale_crop_7_3_large config:core.entity_view_display.media.image.square config:core.entity_view_display.node.article.card_common config:core.entity_view_display.node.recipe.card_common config:core.entity_view_display.node.recipe.card_common_alt 
config:filter.format.basic_html config:filter.format.full_html config:image.style.large_3_2_2x config:image.style.large_3_2_768x512 config:image.style.medium_3_2_2x config:image.style.medium_3_2_600x400 config:image.style.medium_8_7 config:image.style.scale_crop_7_3_large config:image.style.square_large config:image.style.square_medium config:image.style.square_small config:responsive_image.styles.3_2_image config:responsive_image.styles.square config:system.menu.account config:system.menu.footer config:system.menu.main config:system.site config:user.role.anonymous config:views.view.frontpage config:views.view.promoted_items config:views.view.recipe_collections file:35 file:37 http_response local_task 
media:1 media:17 media:18 media:19 media:2 media:21 media:3 media:4 media:9 media_view 
node:1 node:10 node:18 node:2 node:3 node:4 node:9 node_list node_view 
rendered 
taxonomy_term:1 taxonomy_term:10 taxonomy_term:11 taxonomy_term:12 taxonomy_term:13 taxonomy_term:14 taxonomy_term:15 taxonomy_term:16 taxonomy_term:2 taxonomy_term:3 taxonomy_term:4 taxonomy_term:5 taxonomy_term:6 taxonomy_term:7 taxonomy_term:8 taxonomy_term:9 taxonomy_term_list user:4

But, they did find that the block content entity's cacheable metadata was missing. From the Umami header example, it would look like the following:

<!-- CACHE TAGS:
   * block_view
   * config:block.block.umami_banner_home
   * config:core.entity_view_display.block_content.banner_block.default
   * media_view
   * media:18
   * config:image.style.scale_crop_7_3_large
   * file:35
   * config:core.entity_view_display.media.image.scale_crop_7_3_large
-->

The following tags would be missing

   * block_content_view
   * block_content:3

The block_content:3 cache tag is what gets invalidated whenever the block content entity is saved. The trick is to then figure out why it might be missing. For cases like this, where it should always "just work," it is best to review the theme templates (as they did.) The template was rendering individual components and not the entire entity output. The cacheable metadata is associated on the root element, so it was never captured.

Turning on renderer debug

To enable the renderer debug, you need to customize your Drupal site's service container parameters. To do this, edit the web/sites/development.services.yml in your Drupal codebase. We need to add renderer.config under parameters. Unfortunately, array based parameters are not merged with other defaults. That means the following will not work and will break the renderer.

parameters:
  http.response.debug_cacheability_headers: true
  renderer.config:
      debug: true

We have to copy the entire renderer.config parameter from web/sites/default/services.yml. Here is an example of what your development.services.yml should look like, code comments omitted. 

parameters:
  http.response.debug_cacheability_headers: true
  renderer.config:
      required_cache_contexts: ['languages:language_interface', 'theme', 'user.permissions']
      auto_placeholder_conditions:
        max-age: 0
        contexts: ['session', 'user']
        tags: []
      debug: true
services:
  cache.backend.null:
    class: Drupal\Core\Cache\NullBackendFactory

Next we need to activate this file. Copy the web/sites/example.settings.local.php file to web/sites/default/settings.local.php. Edit your web/sites/default/settings.php file. The bottom of the file contains commented out code to include this file, remove the # characters so the following code is executed

if (file_exists($app_root . '/' . $site_path . '/settings.local.php')) {
   include $app_root . '/' . $site_path . '/settings.local.php';
}

Rebuild your Drupal site's cache so that the service container is rebuilt. You now have renderer debug enabled!

Want to learn more about caching and Drupal?

I'm working on a book about Drupal's caching layers! Check out Understanding Drupal: A Complete Guide to Caching Layers.