[ ← blog / drupal ](/) May 16, 2023  ·  \#drupal  ·  10 min read # Simplifying the frontend developer experience in Drupal with a click of the button Last year at DrupalCon Portland 2022, Dries announced that "[**Drupal is for ambitious site builders**](https://dri.es/drupal-is-for-ambitious-site-builders)." It refined the "[**Drupal is for ambitious digital experiences**](https://dri.es/drupal-is-for-ambitious-digital-experiences)" vision. It chooses to focus on a specific persona and improve their experience with Drupal. It didn't sit perfectly well with me, so I wrote about [how improving the Drupal developer experience empowers the Ambitious Site Builder.](https://mglaman.dev/blog/improve-drupal-developer-experience-empower-ambitious-site-builder) And with Drupal 10, there has been a huge shift in the development experience for frontend developers. Drupal 10.1.0 will bring even more improvements to the frontend developer experience. One of those will be the new **Development Settings** form to manage Twig development mode and markup caching. [Make it easier for theme builders to enable Twig debugging and disable render cache Problem/Motivation Seeing changes in your Twig templates is difficult without specific Drupal knowledge. Here is my go to configuration when theme building in Drupal: Turn on Twig debug in sites/default/services.yml: twig.config: debug: true auto_reload: null cache: true Disable page, dynamic_page, render cache so I get fresh data as I edit Twig templates: $settings['container_yamls'][] = DRUPAL_ROOT . https://www.drupal.org/node/3278493 ![Make it easier for theme builders to enable Twig debugging and disable render cache](https://www.drupal.org/sites/all/themes/bluecheese/images/og.jpg) ](https://www.drupal.org/node/3278493) ## The pain of bypassing markup caching when building Drupal One reason I love building applications in Drupal is its Cache API. There isn't anything like it in other frameworks. We have fragment caching for our markup. That then builds into dynamic page caching for partially caching responses of pages with uncacheable content. Then we have the page cache for full response caching like a reverse proxy. Amazing for production performance but a roadblock for local development. There is a way to bypass these caching layers, and you can do so *if you know* to read the [`sites/example.settings.local.php`](https://git.drupalcode.org/project/drupal/-/blob/11.x/sites/example.settings.local.php#L5-17) file in your Drupal code base. Here are the steps required to setup bypassing these caches: 1. Copy `sites/example.settings.local.php` to `sites/default/settings.local.php` 2. Remove comment to allow `render` cache bin to be bypassed ([source](https://git.drupalcode.org/project/drupal/-/blob/11.x/sites/example.settings.local.php#L57-69)) 3. Remove comment to allow `page` cache bin to be bypassed ([source](https://git.drupalcode.org/project/drupal/-/blob/11.x/sites/example.settings.local.php#L79-91)) 4. Remove comment to allow `dynamic_page_cache` cache bin to be bypassed ([source](https://git.drupalcode.org/project/drupal/-/blob/11.x/sites/example.settings.local.php#L93-100)) 5. Run `chmod 775 sites/default` to ensure it's writeable after Drupal hardened the directory's permissions 6. Edit your `sites/default/settings.php` to [allow including your new `settings.local.php` file](https://git.drupalcode.org/project/drupal/-/blob/11.x/sites/default/default.settings.php#L829-845) 7. Rebuild caches using Drush or the user interface All right! We did it! Wait, no? Yes, that's right. This was only part of the battle! This will bypass markup caching for your content within render arrays in your blocks and controllers. It does not bypass Twig's template caching. That means changes to Twig templates won't show up automatically yet! This next part is convoluted and the most painful part of the process. 1. Open [`sites/development.services.yml`](https://git.drupalcode.org/project/drupal/-/blob/11.x/sites/development.services.yml) in your editor 2. Copy the `twig.config` parameter defaults from [`sites/default.services.yml`](https://git.drupalcode.org/project/drupal/-/blob/11.x/sites/default/default.services.yml#L63-117) 3. Paste the `twig.conifg` parameter defaults into the `development.services.yml` file under `parameters` 4. Modify `debug` to be `true` to enable Twig's debug mode 5. Optionally set `cache` to be `false` to bypass template cache and only reload templates if they have been modified 6. Rebuild caches using Drush or the user interface (Note you need to copy and paste the entire `twig.config` contents because array-based parameters are not merged with other defaults when defining service parameters.) And now we are there! If you modify a Twig template in your theme the changes will automatically appear without being caught behind various layers of cache. That is, until you run `composer install` or `composer update` and Drupal's scaffolding plugin destroys your modifications to `sites/development.services.yml`. The `sites/development.services.yml` is provided by Drupal core and automatically placed as part of Drupal's scaffolding. 1. Read ho[w to exclude scaffolding files](https://www.drupal.org/docs/develop/using-composer/using-drupals-composer-scaffold#toc_6) from the documentation 2. Edit your project's `composer.json` file 3. In `extra.drupal-scaffold.file-mapping` set `[web-root]/sites/development.services.yml` to `false` 4. Run `composer update --lock` to ignore checksum warnings Many Drupal agencies have set up project templates or tooling that scaffold this setup. I am one of them. You can browse some boilerplate that I copy around: [`composer.json`](https://github.com/mglaman/whiskeydex/blob/main/composer.json#L105-L114) and [`development.services.yml`](https://github.com/mglaman/whiskeydex/blob/main/web/sites/development.services.yml). As you can see, this requires *understanding* these details as an established Drupal agency or consultant. What about all of the new developers we want to attract to Drupal? What about those new frontend developers being brought in to work on a Drupal site instead of WordPress or Next.js? They *need* to have a delightful experience that makes them want to work with Drupal again. ## The new Development Settings form in Drupal 10.1.0 One fix would have been to improve the `default development.services.yml` to have defaults for Twig and [Render debugging](https://mglaman.dev/blog/debugging-your-render-cacheable-metadata-drupal). But that still wouldn't solve the root problem. It's hard to get template modifications to "show up" when building your Drupal site. It should be easier than having to copy and edit files. And now, it will be as simple as pressing a button on the Development Settings form in Drupal 10.1.0, releasing in June! ![The Development Settings form and its checkboxes to turn on Twig development mode and bypass markup caching](/sites/default/files/styles/large/public/2023-05/drupal-development-settings-form.png?itok=msDhFMQh) I opened an issue after DrupalCon Portland that we needed to [make it easier for theme builders to enable Twig debugging and disable the render cache](https://www.drupal.org/node/3278493). We couldn't have just one or the other. If you have Twig debug mode, your markup is still being cached. If you bypass markup caching, your Twig templates are still cached. They are very much intertwined. I worked on the concept for a few days on personal time but had to pivot as I had to begin focusing on Drupal 10 upgrade tooling. Many contributors worked on this issue to provide user experience feedback, testing, and code contribution. I recommend [reading Mike Herchel's blog post](https://herchel.com/articles/disabling-twig-caching-just-got-helluva-lotta-easier-101) about this new feature as it provides more history about managing to land this feature for Drupal 10.1.0. We just made the 10.1.x code freeze. Otherwise, this feature would have to wait for 10.2.0 in December 2023! There is currently a follow up ticket. We never invalidate the Twig template cache or anny markup cache bins. That means when you re-enable Twig caching and markup caching the Drupal site shows stale content, like a time machine. I have an initial merge request opened to resolve this problem. It needs tests and review of the approach. [Latest/real markup should be used when caches are enabled Problem/Motivation [#3278493] adds a new UI for enabling development settings. However, in comment #120, @crasx pointed out a weird behavior: if you have the cache enabled, you disable it and make changes, and enable it again, you get the old cached markup. This is odd and not expected, so a bad and confusing DX experience. https://www.drupal.org/project/drupal/issues/3359931 ![Latest/real markup should be used when caches are enabled](https://www.drupal.org/sites/all/themes/bluecheese/images/og.jpg) ](https://www.drupal.org/project/drupal/issues/3359931) ## How it works Now to the fun part! I am excited about this feature because it brings a new approach to manipulating Drupal's application state. These development settings rely on customizing the service container by editing various settings files. All previous steps have been consolidated into one form and two checkboxes. ### State API versus Configuration API When we think of administrative forms in Drupal, they are associated with modifying configuration objects to control how Drupal works. The new **Development Settings** form utilizes the [State API](https://www.drupal.org/docs/8/api/state-api/overview) for storing these settings. The State API provides a "simple system for the storage of information about the system's state." Values stored in the state are not meant to be exported and are intended to be specific to an individual environment. That is exactly where development settings should be stored. If we had used configuration objects to store these settings, someone may perform a configuration export and accidentally commit these settings. Then they could be deployed and pushed to multiple environments. If you have any sort of auditing tools for your Drupal site, the settings are stored with the following keys. If they exist and their value is `true`, then they are active. - `twig_debug` - `twig_cache_disable` - `disable_rendered_output_cache_bins` ### Modifying the service container with state values For these development settings to take effect, we need to modify the service container when it is being built. To do so, we implemented a [compiler pass](https://symfony.com/doc/current/service_container/compiler_passes.html) to modify the service container when it is being compiled. The new [`DevelopmentSettingsPass`](https://git.drupalcode.org/project/drupal/-/blob/11.x/core/lib/Drupal/Core/DependencyInjection/Compiler/DevelopmentSettingsPass.php) compiler pass reads values from the state information and applies changes to the service container. It is *kind of* funky because it causes initialization of the `state` service during container compilation. But, this compiler pass runs after all services have been registered and any service modifiers have been executed. Here's an [exerpt of the code in `CoreServiceProvider`](https://git.drupalcode.org/project/drupal/-/blob/11.x/core/lib/Drupal/Core/CoreServiceProvider.php#L57-62) where the new compiler pass is registered, amongst the other existing ones: ```php // Add the compiler pass that lets service providers modify existing // service definitions. This pass must come first so that later // list-building passes are operating on the post-alter services list. $container->addCompilerPass(new ModifyServiceDefinitionsPass()); $container->addCompilerPass(new DevelopmentSettingsPass()); ``` Why put the logic in a compiler pass rather than in the `CoreServiceProvider` class? Honestly it's for encapsulation of that code. Most modules that modify the service container implement a `MyModuleServiceProvider` and register or alter services in that service provider. Compiler passes are the method provided by [Symfony's DependencyInjection component](https://symfony.com/doc/current/components/dependency_injection.html). The `ServiceModifierInterface` and `ServiceProviderInterface` concepts are additions Drupal has provided (so modules could add their own compiler passes, or other manipulations of the service container.) If this is a new concept, I recommend reading the documentation page [Altering existing services, providing dynamic services](https://www.drupal.org/docs/drupal-apis/services-and-dependency-injection/altering-existing-services-providing-dynamic-services). The compiler pass checks if Twig debug mode is enabled or if Twig cache has been disabled. It then updates the `twig.config` parameter with the proper values. It is a lot simpler than having to copy a block of YAML content from one file to another! ```php /** @var \Drupal\Core\State\StateInterface $state */ $state = $container->get('state'); $twig_debug = $state->get('twig_debug', FALSE); $twig_cache_disable = $state->get('twig_cache_disable', FALSE); if ($twig_debug || $twig_cache_disable) { $twig_config = $container->getParameter('twig.config'); $twig_config['debug'] = $twig_debug; $twig_config['cache'] = !$twig_cache_disable; $container->setParameter('twig.config', $twig_config); } ``` Next, when disabling markup caching, we have to do two things. The `cache.backend.null` service is only defined if `sites/development.services.yml` has been included. The compiler pass registers the service if it does not exist. We then iterate through the relevant cache bins and set their default backend to `cache.backend.null`. ```php if ($state->get('disable_rendered_output_cache_bins', FALSE)) { $cache_bins = ['page', 'dynamic_page_cache', 'render']; if (!$container->hasDefinition('cache.backend.null')) { $container->register('cache.backend.null', NullBackendFactory::class); } foreach ($cache_bins as $cache_bin) { if ($container->has("cache.$cache_bin")) { $container->getDefinition("cache.$cache_bin") ->clearTag('cache.bin') ->addTag('cache.bin', ['default_backend' => 'cache.backend.null']); } } } } ``` Fun fact! Before this code wasn't in a compiler pass. I refactored it into the compiler pass later to help keep the code clean and tidy for review. When I did this, I registered the `DevelopmentSettingsPass` compiler pass *last*. When I did that I hit a regression. Disabling markup caching no longer work! It runs out it was because the `DevelopmentSettingsPass` compiler pass ran after [`BackendCompilerPass`](https://git.drupalcode.org/project/drupal/-/blob/11.x/core/lib/Drupal/Core/DependencyInjection/Compiler/BackendCompilerPass.php). That compiler pass reads `default_backend` and properly injects the storage backend service. For the changes in `DevelopmentSettingsPass` to take effect it needed to run before `BackendCompilerPass`. That is why it currently runs after `ModifyServiceDefinitionsPass`. ### Applying changes by invalidating the service container Whenever you make changes to `sites/development.services.yml`, the cache bins in `sites/default/settings.local.php`, or `sites/default/services.yml` (if present) you need to rebuild Drupal's caches to have them take effect. That is because Drupal caches the service container definition and it needs to be rebuilt. However, rebuilding Drupal's caches causes it to have to *a lot* of other work besides rebuild the service container. Luckily it is possible to just invalidate the container so the container cache is purged. This provides a more precise action and keeps other caches intact – like plugin definitions. When you submit the **Development Settings** form, we invalidate the service container so that settings are applied immediately. The logic is very verbose as we try to minimize invalidating the service container unless we have to when values change. I recommend checking it out in full: [`DevelopmentSettingsForm.php#L120-147`](https://git.drupalcode.org/project/drupal/-/blob/11.x/core/modules/system/src/Form/DevelopmentSettingsForm.php#L120-147). But here is a quick highlight of how it works. Our from has `DrupalKernel` injected. The kernel is the root entrypoint of Drupal and handles requests and managing the service container. ```php public function __construct( protected StateInterface $state, protected DrupalKernelInterface $kernel ) { } public static function create(ContainerInterface $container) { $instance = new static( $container->get('state'), $container->get('kernel') ); $instance->setMessenger($container->get('messenger')); return $instance; } ``` Then in our `submitForm` method we call the kernel's [`invalidateContainer`](https://git.drupalcode.org/project/drupal/-/blob/11.x/core/lib/Drupal/Core/DrupalKernel.php#L1186) method. This deletes the cached container definition, which causes Drupal to recompile the container the next time it is requested. ```php $this->kernel->invalidateContainer(); ``` And then, magic! The page load after form submit will have Twig debug comments in its markup and bypass markup caching. ![](/themes/custom/mglaman/images/headshot.webp)Thanks for reading I write about Drupal, PHP, and the quiet infrastructure behind large sites — caching, config, and testing. New posts land every week or two. [Subscribe via RSS →](/rss.xml) [About me →](/about-me) [← Older Upgrading my site from CKEditor 4 to CKEditor 5 ](/blog/upgrading-my-site-ckeditor-4-ckeditor-5) [Newer → Dependency injection anti-patterns in Drupal ](/blog/dependency-injection-anti-patterns-drupal)