Matthias Kestenholz: Posts about Django

Preserving referential integrity with JSON fields and Django

2025-06-04T12:00:00Z

Preserving referential integrity with JSON fields and Django

Motivation

The great thing about using feincms3 and django-content-editor is that CMS plugins are Django models – if using them you immediately have access to the power of Django’s ORM and Django’s administration interface.

However, using one model per content type can be limiting on larger sites. Because of this we like using JSON plugins with schemas for more fringe use cases or for places where we have richer data but do not want to write a separate Django app for it. This works well as long as you only work with text, numbers etc. but gets a bit ugly once you start referencing Django models because you never know if those objects are still around when actually using the data stored in those JSON fields.

Django has a nice on_delete=models.PROTECT feature, but that of course only works when using real models. So, let’s bridge this gap and allow using foreign key protection with data stored in JSON fields!

Models

First, you have to start using the django-json-schema-editor and specifically its JSONField instead of the standard Django JSONField. The most important difference between those two is that the schema editor’s field wants a JSON schema. So, for the sake of an example, let’s assume that we have a model with images and a model with galleries. Note that we’re omitting many of the fields actually making the interface nice such as titles etc.

from django.db import models
from django_json_schema_editor.fields import JSONField

class Image(models.Model):
    image = models.ImageField(...)

gallery_schema = {
    "type": "object",
    "properties": {
        "caption": {"type": "string"},
        "images": {
            "type": "array",
            "format": "table",
            "minItems": 3,
            "items": {
                "type": "string",
                "format": "foreign_key",
                "options": {
                    # raw_id_fields URL:
                    "url": "/admin/myapp/image/?_popup=1&_to_field=id",
                },
            },
        },
    },
}

class Gallery(models.Model):
    data = JSONField(schema=gallery_schema)

Now, if we were to do it by hand, we’d define a through model for a ManyToManyField linking galleries to images, and adding a on_delete=models.PROTECT foreign key to this through model’s image foreign key and we would be updating this many to many table when the Gallery object changes. Since that’s somewhat boring but also tricky code I have already written it (including unit tests of course) and all that’s left to do is define the linking:

Gallery.register_data_reference(
    # The model we're referencing:
    Image,
    # The name of the ManyToManyField:
    name="images",
    # The getter which returns a list of stringified primary key values or nothing:
    getter=lambda obj: obj.data.get("images"),
)

Now, attempting to delete an image which is still used in a gallery somewhere will raise ProtectedError exceptions. That’s what we wanted to achieve.

Using a gallery instance

When you have a gallery instance you can now use the images field to fetch all images and use the order from the JSON data:

def gallery_context(gallery):
    images = {str(image.pk): image for image in gallery.images.all()}
    return {
        "caption": gallery.data["caption"],
        "images": [images[pk] for pk in gallery.data["images"]],
    }

JSONPluginBase and JSONPluginInline

I would generally do the instantiation of models slightly differently and use django-json-schema-editor’s JSONPluginBase and JSONPluginInline which offer additional niceties such as streamlined JSON models with only one backing database table (using proxy models) and supporting not just showing the primary key of referenced model instances but also their __str__ value.

The example above would have to be changed to look more like this:

from django_json_schema_editor import JSONPluginBase

class JSONPlugin(JSONPluginBase, ...):
    pass

JSONPlugin.register_data_reference(...)

Gallery = JSONPlugin.proxy("gallery", schema=gallery_schema)

However, that’s not documented yet so for now you unfortunately have to read the code and the test suite, sorry for that. It’s used heavily in production though so if you start using it it won’t suddenly start breaking in the future.

How I'm bundling frontend assets using Django and rspack these days

2025-05-26T12:00:00Z

How I’m bundling frontend assets using Django and rspack these days

I last wrote about configuring Django with bundlers in 2018: Our approach to configuring Django, Webpack and ManifestStaticFilesStorage. An update has been a long time coming. I wanted to write this down for a while already, but each time I started explaining how configuring rspack is actually nice I look at the files we’re using and switch to writing about something else. This time I managed to get through – it’s not that bad, I promise.

This is quite a long post. A project where all of this can be seen in action is Traduire, a platform for translating gettext catalogs. I announced it on the Django forum.

Our requirements

The requirements were still basically the same:

Hot module reloading during development
A process which produces hashed filenames depending on their content so that we can use far-future expiry headers to cache assets in browsers
While running Node.js in development is fine we do not want Node.js on the server (in the general case)
We still want transpiling and bundling for now

We have old projects using SASS. These days we’re only using PostCSS (especially autoprefixer and maybe postcss-nesting. Rewriting everything is out of the question, so we needed a tool which handled all that as well.

People in the frontend space seem to like tools like Vite or Next.js a lot. I have also looked at Parcel, esbuild, rsbuild and others. Either they didn’t support our old projects, were too limited in scope (e.g. no HMR), too opinionated or I hit bugs or had questions about their maintenance. I’m sure all of them are great for some people, and I don’t intend to talk badly about any of them!

In the end, the flexibility, speed and trustworthiness of rspack won me over even though I have a love-hate relationship with the Webpack/rspack configuration. We already had a reusable library of configuration snippets for webpack though and moving that library over to rspack was straightforward.

That being said, configuring rspack from scratch is no joke, that’s why tools such as rsbuild exist. If you already know Webpack well or really need the flexibility, going low level can be good.

High-level project structure

The high-level overview is:

Frontend assets live in their own folder, frontend/.
We’re using fabric and rspack, their configuration resides in the root folder of the project as does Django’s manage.py.
The frontend is transpiled and bundled directly into static/ for production and into tmp/ during development.
We use the HTML plugin of rspack to emit snippets containing <link> and <script> tags. The HTML snippet can be included as-is, without any postprocessing.
frontend/ or frontend/static is optionally added to STATICFILES_DIRS so that some of the files from the frontend can easily be referenced in {% static %} tags.

During development:

We use the dev server of rspack/node to handle 127.0.0.1:8000. This server handles requests for frontend assets and the websocket for hot module reloading and proxies everything else to the Django backend running on a different random port.

During deployment:

The assets are compiled to static/ and either rsynced to the server or added to the container separately from the standard ./manage.py collectstatic --noinput.

In production:

Separate cache busting filenames from ManifestStaticFilesStorage and rspack allow us to set far-future expiry headers on all static assets.
I’m serving static assets from the same origin as the website itself. (rspack can be configured for different requirements!)
I don’t worry anymore about duplicating assets which are both referenced from frontend code and backend code. This doesn’t affect many assets after all.
The HTML snippet is loaded once only.

Example configuration

Here’s an example configuration which works well for us. What follows is the rspack configuration itself, building on our snippet library rspack.library.js. We mostly do not change anything in here except for the list of PostCSS plugins:

rspack.config.js:

module.exports = (env, argv) => {
  const { base, devServer, assetRule, postcssRule, swcWithPreactRule } =
    require("./rspack.library.js")(argv.mode === "production")

  return {
    ...base,
    devServer: devServer({ backendPort: env.backend }),
    module: {
      rules: [
        assetRule(),
        postcssRule({
          plugins: [
            "postcss-nesting",
            "autoprefixer",
          ],
        }),
        swcWithPreactRule(),
      ],
    },
  }
}

The default entry point is main and loads frontend/main.js. The rest of the JavaScript and styles are loaded from there.

The HTML snippet loader works by adding WEBPACK_ASSETS = BASE_DIR / "static" to the Django settings and adding the following tags to the <head> of the website, most often in base.html:

{% load webpack_assets %}
{% webpack_assets 'main' %}

The corresponding template tag in webpack_assets.py follows:

from functools import cache

from django import template
from django.conf import settings
from django.utils.html import mark_safe

register = template.Library()

def webpack_assets(entry):
    path = settings.BASE_DIR / ("tmp" if settings.DEBUG else "static") / f"{entry}.html"
    return mark_safe(path.read_text())

if not settings.DEBUG:
    webpack_assets = cache(webpack_assets)
register.simple_tag(webpack_assets)

Last but not least, the fabfile contains the following task definition:

@task
def dev(ctx, host="127.0.0.1", port=8000):
    backend = random.randint(50000, 60000)
    jobs = [
        f".venv/bin/python manage.py runserver {backend}",
        f"HOST={host} PORT={port} yarn run rspack serve --mode=development --env backend={backend}",
    ]
    # Run these two jobs at the same time:
    _concurrently(ctx, jobs)

The fh-fablib repository contains the _concurrently implementation we’re using at this time.

The library which enables the nice configuration above

Of course, the whole library of snippets has to be somewhere. The fabfile automatically updates the library when we release a new version, and the library is the same in all the dozens of projects we’re working on. Here’s the current version of rspack.library.js:

const path = require("node:path")
const HtmlWebpackPlugin = require("html-webpack-plugin")
const rspack = require("@rspack/core")
const assert = require("node:assert/strict")
const semver = require("semver")

assert.ok(semver.satisfies(rspack.rspackVersion, ">=1.1.3"), "rspack outdated")

const truthy = (...list) => list.filter((el) => !!el)

module.exports = (PRODUCTION) => {
  const cwd = process.cwd()

  function swcWithPreactRule() {
    return {
      test: /\.(j|t)sx?$/,
      loader: "builtin:swc-loader",
      exclude: [/node_modules/],
      options: {
        jsc: {
          parser: {
            syntax: "ecmascript",
            jsx: true,
          },
          transform: {
            react: {
              runtime: "automatic",
              importSource: "preact",
            },
          },
          externalHelpers: true,
        },
      },
      type: "javascript/auto",
    }
  }

  function swcWithReactRule() {
    return {
      test: /\.(j|t)sx?$/,
      loader: "builtin:swc-loader",
      exclude: [/node_modules/],
      options: {
        jsc: {
          parser: {
            syntax: "ecmascript",
            jsx: true,
          },
          transform: {
            react: {
              runtime: "automatic",
              // importSource: "preact",
            },
          },
          externalHelpers: true,
        },
      },
      type: "javascript/auto",
    }
  }

  function htmlPlugin(name = "", config = {}) {
    return new HtmlWebpackPlugin({
      filename: name ? `${name}.html` : "[name].html",
      inject: false,
      templateContent: ({ htmlWebpackPlugin }) =>
        `${htmlWebpackPlugin.tags.headTags}`,
      ...config,
    })
  }

  function htmlSingleChunkPlugin(chunk = "") {
    return htmlPlugin(chunk, chunk ? { chunks: [chunk] } : {})
  }

  function postcssLoaders(plugins) {
    return [
      { loader: rspack.CssExtractRspackPlugin.loader },
      { loader: "css-loader" },
      { loader: "postcss-loader", options: { postcssOptions: { plugins } } },
    ]
  }

  function cssExtractPlugin() {
    return new rspack.CssExtractRspackPlugin({
      filename: PRODUCTION ? "[name].[contenthash].css" : "[name].css",
      chunkFilename: PRODUCTION ? "[name].[contenthash].css" : "[name].css",
    })
  }

  return {
    truthy,
    base: {
      context: path.join(cwd, "frontend"),
      entry: { main: "./main.js" },
      output: {
        clean: PRODUCTION,
        path: path.join(cwd, PRODUCTION ? "static" : "tmp"),
        publicPath: "/static/",
        filename: PRODUCTION ? "[name].[contenthash].js" : "[name].js",
        // Same as the default but prefixed with "_/[name]."
        assetModuleFilename: "_/[name].[hash][ext][query][fragment]",
      },
      plugins: truthy(cssExtractPlugin(), htmlSingleChunkPlugin()),
      target: "browserslist:defaults",
    },
    devServer(proxySettings) {
      return {
        host: "0.0.0.0",
        hot: true,
        port: Number(process.env.PORT || 4000),
        allowedHosts: "all",
        client: {
          overlay: {
            errors: true,
            warnings: false,
            runtimeErrors: true,
          },
        },
        devMiddleware: {
          headers: { "Access-Control-Allow-Origin": "*" },
          index: true,
          writeToDisk: (path) => /\.html$/.test(path),
        },
        proxy: [
          proxySettings
            ? {
                context: () => true,
                target: `http://127.0.0.1:${proxySettings.backendPort}`,
              }
            : {},
        ],
      }
    },
    assetRule() {
      return {
        test: /\.(png|webp|woff2?|svg|eot|ttf|otf|gif|jpe?g|mp3|wav)$/i,
        type: "asset",
        parser: { dataUrlCondition: { maxSize: 512 /* bytes */ } },
      }
    },
    postcssRule(cfg) {
      return {
        test: /\.css$/i,
        type: "javascript/auto",
        use: postcssLoaders(cfg?.plugins),
      }
    },
    sassRule(options = {}) {
      let { cssLoaders } = options
      if (!cssLoaders) cssLoaders = postcssLoaders(["autoprefixer"])
      return {
        test: /\.scss$/i,
        use: [
          ...cssLoaders,
          {
            loader: "sass-loader",
            options: {
              sassOptions: {
                includePaths: [path.resolve(path.join(cwd, "node_modules"))],
              },
            },
          },
        ],
        type: "javascript/auto",
      }
    },
    swcWithPreactRule,
    swcWithReactRule,
    resolvePreactAsReact() {
      return {
        resolve: {
          alias: {
            react: "preact/compat",
            "react-dom/test-utils": "preact/test-utils",
            "react-dom": "preact/compat", // Must be below test-utils
            "react/jsx-runtime": "preact/jsx-runtime",
          },
        },
      }
    },
    htmlPlugin,
    htmlSingleChunkPlugin,
    postcssLoaders,
    cssExtractPlugin,
  }
}

Closing thoughts

Several utilities from this library aren’t used in the example above, for example the sassRule or the HTML plugin utilities which are useful when you require several entry points on your website, e.g. an entry point for the public facing website and an entry point for a dashboard used by members of the staff.

Most of the code in here is freely available in our fh-fablib repo under an open source license. Anything in this blog post can also be used under the CC0 license, so feel free to steal everything. If you do, I’d be happy to hear your thoughts about this post, and please share your experiences and suggestions for improvement – if you have any!

Django, JavaScript modules and importmaps

2025-05-22T12:00:00Z

How I’m using Django, JavaScript modules and importmaps together

I have been spending a lot of time in the last few months working on django-prose-editor. First I’ve rebuilt the editor on top of Tiptap because I wanted a framework for extending the underlying ProseMirror and didn’t want to reinvent this particular wheel. While doing that work I noticed that using JavaScript modules in the browser would be really nice, but Django’s ManifestStaticFilesStorage doesn’t yet support rewriting import statement in modules out-of-the-box without opting into the experimental support accessible through subclassing the storage. A better way to use JavaScript modules with the cache busting offered by ManifestStaticFilesStorage would be importmaps.

Motivation

Developing Django applications that include JavaScript has always been challenging when it comes to properly distributing, loading, and versioning those assets. The traditional approach using Django’s forms.Media works well for simple use cases, but falls short when dealing with modern JavaScript modules.

The ability to ship reusable JavaScript utilities in third-party Django apps has been a pain point for years. Often developers resort to workarounds like bundling all JS into a single file, using jQuery-style global variables, or requiring complex build processes for consumers of their apps.

Importmaps offer a cleaner solution that works with native browser modules, supports cache busting, and doesn’t require complex bundling for simple use cases.

The history

The conversation around better JavaScript handling in Django has been ongoing for years. Thibaud Colas’ DEP draft come to mind as does the discussion about whether to improve or deprecate forms.Media.

A few packages exist which are offering solutions in this space:

django-esm provides a solution for using ES modules with Django without bundling.
django-js-asset provides helpers for delivering JavaScript modules, importmaps, JSON blobs etc. to the browser through Django’s forms.Media. The blog post Object-based assets for Django’s forms.Media explores this in more detail.
The article on Content Security Policy compliance explores better approaches to use JavaScript in the Django admin while avoiding inline JavaScript.

django-js-asset came before Django added official support for object-based media CSS and JS paths but has since been changed to take advantage of that official support. It has enabled the removal of ugly hacks. In the meantime, Django has even added official support for object-based Script tags.

My DEP draft

Building on these efforts, I’ve been thinking about submitting my own DEP draft for importmap support. It hasn’t yet come far though, and I’m still more occupied with verifying and using my existing solution, especially learning if it has limitations which would make the implemented approach unworkable for official inclusion.

The current effort

As alluded to above, I already have a working solution for using importmaps (in django-js-asset) and I’m actively using it in django-prose-editor. Here’s how it works:

importmap.update({
    "imports": {
        "django-prose-editor/editor": static_lazy("django_prose_editor/editor.js"),
    }
})

A minimal editor implementation using this:

import {
  // Tiptap extensions
  Document, Paragraph, HardBreak, Text, Bold, Italic,

  // Prose editor utilities
  Menu, createTextareaEditor, initializeEditors,
} from "django-prose-editor/editor"

const extensions = [
  Document, Paragraph, HardBreak, Text, Bold, Italic, Menu,
]

initializeEditors((textarea) => {
  createTextareaEditor(textarea, extensions)
})

The importmap looks as follows when using Django’s ManifestStaticFilesStorage which produces filenames containing the hash of the file’s contents for cache busting (edited for readability):

<script type="importmap">
{"imports": {
  "django-prose-editor/editor": "/static/django_prose_editor/editor.6e8dd4c12e2e.js"
}}
</script>

This means that when your code has import { ... } from "django-prose-editor/editor", the browser automatically loads the file from /static/django_prose_editor/editor.6e8dd4c12e2e.js. The hashed filename provides cache busting while the import statement remains clean and consistent.

Problems with the current implementation

While this approach works, there are several issues to address:

I don’t really like global variables but there doesn’t seem to be a way around it. Browsers want to use a single importmap only (even though the algorithm for merging importmaps exists in the spec!) and the importmap has to be included above all ES modules.
The fact that browsers only want a single importmap also means that when you use django-js-asset’s importmap support you cannot use a different package offering its own solution for importmaps.
The importmap may be added twice to the HTML when using a widget that works in both the admin and frontend contexts. Currently, if you want to avoid this problem or ugliness you have to determine in your Django form field if the code is requesting an admin widget or another widget, either by inspecting the callstack (very ugly) or by checking if the widget argument to the form field constructor is set to an admin-specific widget (also somewhat ugly, since widgets can be classes, instances, or not provided at all).
It would be nice if we the installation of django-prose-editor didn’t have more steps than what we have when installing any other Django widget integration. I’d like a more elegant solution, but haven’t found one yet that doesn’t introduce too much magic.

Comparison to django-esm

django-esm takes a different approach. It assumes you’re using JavaScript modules everywhere and solves the problem of exposing the correct paths to those modules to the browser. It supports both private modules from your repository and modules installed in node_modules.

However, it doesn’t fully address the scenario where a third-party Django app (a Python package) ships JavaScript modules that need to be integrated into your application.

I still use a bundler for most of my JavaScript from node_modules, so I don’t need this specific functionality yet. That will probably change in the future.

Using bundlers

If you’re still using a bundler, as I do, you want to ensure that the import isn’t actually evaluated by the bundler but left as-is. The rspack configuration I’m using at the moment is also documented in the django-prose-editor README but I’m duplicating it here for convenience:

module.exports = {
  // ...
  experiments: { outputModule: true },
  externals: {
    "django-prose-editor/editor": "module django-prose-editor/editor",
    // Or the following, I'm never sure.
    "django-prose-editor/editor": "import django-prose-editor/editor",
  },
}

This configuration marks the dependency as “external” (so it won’t be bundled) and specifies that it should be loaded as a module using a static import statement.

For browser compatibility, you can also include es-module-shims to support browsers that don’t yet handle importmaps natively (around 5% at the time of writing according to caniuse.com).

Using django-compressor or similar packages

Tools like django-compressor aren’t well-suited for modern JavaScript modules as they typically produce old-style JavaScript files rather than ES modules. They’re designed for a different era of web development and don’t integrate well with the importmap approach.

Note

The problem is that django-compressor at this time emits non-module script files. Using import statements in these files isn’t possible, instead you have to use dynamic imports.

// Instead of
import { Document, ... } from "django-prose-editor/editor"
// you need
import("django-prose-editor/editor").then(({ Document, ... }) => {
})

Both work fine. The bundle emitted by django-compressor will not contain the prose editor module itself though; including this module inside the bundle is not possible.

Conclusion

Using importmaps with Django provides a clean solution for managing JavaScript modules in Django applications, especially for third-party apps that need to ship their own JavaScript. While there are still some rough edges to smooth out, this approach works well and offers a path forward that aligns with modern web standards.

Have you tried using importmaps with Django? I’d be interested to hear about your experiences and approaches.

Weeknotes (2025 week 21)

2025-05-21T12:00:00Z

Weeknotes (2025 week 21)

I have missed two co-writing sessions and didn’t manage to post much outside of that, but let’s get things back on track.

django-prose-editor 0.12

The last weeknotes entry contains more details about the work of really connecting Tiptap extensions with server-side sanitization. 0.12 includes many improvements and bugfixes which have been made during real-world use of the prose editor in customer-facing products.

I’m not completely happy about the way we’re specifying the editor configuration and haven’t been able to settle on either extensions or config as a keyword argument. The field supports both ways, at least for now. It’s probably fine.

Releases

django-auto-admin-fieldsets 0.2: I wrote a blog post here: Customizing Django admin fieldsets without fearing forgotten fields
django-debug-toolbar 5.2: This release contains the second half of improvements from Djangonaut Space session four where I helped out as a Navigator. The toolbar properly supports code highlighting in dark mode, sanitizes request variables better, allows customizing redirects, supports projects using django-template-partials and more!
FeinCMS 25.5.1: The first FeinCMS release of 2025. We’re still maintaining the project and fixing bugs!
django-prose-editor 0.12: See above.
django-json-schema-editor 0.4.1: Fixes much too small checkboxes when used inside tables.

Customizing Django admin fieldsets without fearing forgotten fields

2025-04-14T12:00:00Z

Customizing Django admin fieldsets without fearing forgotten fields

When defining fieldsets on Django modeladmin classes I always worry that I forget updating the fieldsets later when adding or removing new model fields, and not without reason: It has already happened to me several times. Forgetting to remove fields is mostly fine because system checks will complain about it, forgetting to add fields may be real bad. A recent example was a crashing website because a required field was missing from the admin and therefore was left empty when creating new instances!

I have now published another Django package which solves this by adding support for specifying the special "__remaining__" field in a fieldsets definition. The "__remaining__" placeholder is automatically replaced by all model fields which haven’t been explicitly added already or added to exclude¹.

Here’s a short example for a modeladmin definition using django-auto-admin-fieldsets:

from django.contrib import admin
from django_auto_admin_fieldsets.admin import AutoFieldsetsModelAdmin
from app import models

@admin.register(models.MyModel)
class MyModelAdmin(AutoFieldsetsModelAdmin):
    # Define fieldsets as usual with a placeholder
    fieldsets = [
        ("Basic Information", {"fields": ["title", "slug"]}),
        ("Content", {"fields": ["__remaining__"]}),
    ]

I have used Claude Code a lot for the code and the package, and as always, I had to fix bugs and oversights. I hope it didn’t regurgitate the code of an existing package – I searched for an existing solution first but didn’t find any.

The package is available on PyPI and is developed on GitHub, at least for the time being.

Autocreated fields such as surrogate primary keys or fields which aren’t editable are also excluded automatically of course. ↩

Weeknotes (2025 week 15)

2025-04-09T12:00:00Z

Weeknotes (2025 week 15)

Djangonaut Space

We have already reached the final week of the Djangonaut Space session 4. I had a great time as a navigator and am looking forward to participate more, but for now I’m also glad that I do not have the additional responsibility at least for the close future.

We have done great work on the django-debug-toolbar in our group, more is to come.

Progress on the prose editor

I have done much work on django-prose-editor in the last few weeks and after a large list of alphas and betas I’m nearing a state which I want to release into the wild.

The integration has been completely rethought (again) and now uses JavaScript modules and importmaps. The ground work to support all of that in Django has been laid in django-js-asset.

The nice thing about using JavaScript modules and importmaps is that we now have an easy way to combine the power of modern JavaScript customization with easy cache busting using Django’s ManifestStaticFilesStorage. A longer post on this is brewing and I hope to have it ready soon-ish.

As a sneak peek, here’s the way it works:

from django_prose_editor.fields import ProseEditorField

content = ProseEditorField(
    extensions={
        "Bold": True,
        "Italic": True,
        "BulletList": True,
        "Link": True,
    },
    # sanitize=True is the default when using extensions
)

The nice thing about it is that the sanitization allowlist for nh3 only includes tags and attributes which are enabled via the extensions dict. So, you don’t have to do anything else to be safe from XSS etc.

Check out the pre-releases on PyPI or have a look at the documentation to learn more about this project!

Using Claude Code

I have been using Claude Code (without editor integrations, thank you very much) more and more. It’s a good coding companion when it comes to throwing around ideas, drafting docs and writing unit tests including integration tests.

Sometimes I’m really surprised at how good it is. Other times… less so. The tool often finds a way to get tests passing, but when the editor integration tests directly manipulate innerHTML and then Claude proclaims that interacting with the editor is now shown to work I have to chuckle a bit. And when I insist on doing what I mean and not just finding broken workarounds it doesn’t really change anything. After spinning more we’re always back where we started.

I am somewhat glad that this is where we’re at now. I’m not 100% sure if it’s progress. At least it’s surprisingly funny at times.

Releases

I haven’t written a regular weeknotes entry since the end of January, so naturally the list here is longer than usual.

feincms3-forms 0.5.1: I inadvertently bumped the Django dependency without actually wanting that; this patch release reverts that (while adding official support for new Django and Python versions).
django-mptt 0.17: Mariusz has done all the hard work for supporting newer versions of Django. I just had to press the release button. That being said, four years after marking the package as unmaintained I’m still maintaining it. At least I don’t get complaints anymore…
django-json-schema-editor 0.4: Added a dependency on the pre-release of django-prose-editor and added a test suite including integration tests so that we actually now when stuff breaks the next time!
django-debug-toolbar 5.1: See above.
feincms3-data: Added fixes to dump distinct objects. Spent more time than useful on the Django change which added a final newline to JSON-serialized data.
django-js-asset 3.1.2: Importmaps support, added a static_lazy helper which is useful to define module-scoped static URLs. The later wouldn’t work with the ManifestStaticFilesStorage because the manifest doesn’t yet exist when collectstatic runs, so the actual evaluation of static URLs has to be postponed. The lazy version solves this nicely.
feincms3-sites 0.21.1 and feincms3-language-sites 0.4.1: See the relevant TIL blogpost.

TIL: Tools exist which do not lowercase domain names when requesting websites over HTTP(S)

2025-02-13T12:00:00Z

TIL: Tools exist which do not lowercase domain names when requesting websites over HTTP(S)

About a week ago I received error mails for a surprising behavior (to me!) where some tool requested an URL from one of our websites using feincms3-language-sites (a Django library for multilingual websites) with a domain name containing uppercase characters.

I knew that the domain part of all sorts of URLs is case sensitive, but what surprised me was that our server actually got a request with such a domain name, I hadn’t really seen that before.

After researching a bit I learned that for example curl intentionally preserves the casing of domain names, but browsers generally do lowercase domains because it’s more consistent. It’s interesting that the initial error was caused by a client with a proper Safari/macOS user agent, but further research showed that the request was probably sent by something called go-social-activity-parser, whatever that is.

I fixed the bug in feincms3-language-sites and also in feincms3-sites by switching to case-insensitive matching of domain names. I have not yet added punycode or IDNA equivalence to the code because I haven’t needed it yet and because I’m not 100% sure how to do it without breaking anything. Even though I often work on websites in languages with lots of accents and umlauts we don’t use such domain names too often, so it hasn’t been a problem yet. I’ll cross that bridge when I get there.

Weeknotes (2025 week 05)

2025-01-29T12:00:00Z

Weeknotes (2025 week 05)

Djangonaut Space

In December I wrote a few paragraphs about my decision to not run for the Django Steering Council, mentioning that I want to contribute in different ways.

I have offered to contribute to Djangonaut Space to do some mentoring. I’m already a bit stressed, but that’s normal and to be expected. I’ll probably have more to share about that in the close future!

Releases

feincms-cookiecontrol 1.6: Removed the hardcoded dependency upon feincms3 and some additional code golfing. The cookie banner JavaScript is now back to <4KiB.
django-curtains 0.7: Updated the CI job list and modernized the package somewhat, no code changes necessary. It’s good to release updated versions though just to show that it’s still actively maintained.
django-prose-editor 0.10.3: Small CSS fixes and mainly updated TipTap/ProseMirror.
django-imagefield 0.22: The updated version no longer autodeletes processed images; this wasn’t really a problem before but I was a little bit fearful that images are still referenced elsewhere and this change let’s me sleep better.
feincms-oembed 2.0: Oembed support for FeinCMS 1 without actually depending upon the FeinCMS package itself. Still works.
django-content-editor 7.2: The Region type is now hashable; this may be useful, or not.
feincms3 5.3.1: I undeprecated the TemplateMixin because even though PageTypeMixin is nicer, sometimes all you need is a template selector.

Django admin tip: Adding links to related objects in change forms

2025-01-17T12:00:00Z

Any issue which came up on the Django Forum and Discord is how to add links to other objects to the Django administration interface. It’s something I’m doing often and I want to share the pattern I’m using.

It’s definitely not rocket science and there are probably better ways to do it, but this one works well for me.

Method 1: Override the change form template

In one project users can be the editor of exactly one organization. The link between organizations and users is achieved using a Editor model with a ForeignKey(Organization) and a OneToOneField(User).

I wanted to add a link to the organization page at the bottom of the user form. An easy way to achieve this is to add a template at templates/admin/auth/user/change_form.html (or something similar if you’re using a custom user model):

{% extends "admin/change_form.html" %}

{% block after_related_objects %}
{{ block.super }}

{% if original.editor %}
<fieldset class="module aligned">
<h2>Organization</h2>
<div class="form-row">
  <a href="{% url 'admin:organizations_organization_change' original.editor.organization.pk %}">{{ original.editor.organization }}</a>
</div>
</fieldset>
{% endif %}

{% endblock after_related_objects %}

The original context variable contains the object being edited. The editor attribute is the reverse accessor for the OneToOneField mentioned above.

Method 2: Add a method to the model admin class returning a HTML blob

A terrible but also nice way is to add a method to the ModelAdmin class which returns the HTML containing the links you want, and adding the name of the method to readonly_fields. This is even mentioned in the official readonly_fields documentation but I discovered this by accident a few years back.

The method name doesn’t have to be added anywhere else, not to fields nor do you have to define fieldsets for this to work. Just adding it to readonly_fields appends it to the end of the form, before any eventual inlines you’re using.

from django.template.loader import render_to_string
from app import models

@admin.register(models.Class)
class ClassAdmin(admin.ModelAdmin):
    list_display = ["name", "language_code"]
    readonly_fields = ["admin_show_custom_districts"]

    @admin.display(description=_("districts")
    def admin_show_custom_districts(self, obj):
        return render_to_string(
            "admin/admin_show_custom_districts.html",
            {"custom_districts": obj.customdistrict_set.all()},
        )

Weeknotes (2025 week 03)

2025-01-15T12:00:00Z

Weeknotes (2025 week 03)

Claude AI helped me for the first time

django-imagefield prefers processing thumbnails, cropped images etc. directly when saving the model and not later on demand; it’s faster and also you’ll know it immediately when an image couldn’t be processed for some reason instead of only later when people actually try browsing your site.

A consequence is that if you change formats you have to remember that you have to reprocess the images. The Django app comes with a management command ./manage.py process_imagefields to help with this. I have added parallel processing based on concurrent.futures to it some time ago so that the command completes faster when it is being run on a system with several cores.

A work colleague is using macOS (many are, in fact), and he always got multiprocessing Python crashes. This is a well known issue and I remember reading about it a few years ago. I checked the docs and saw that the concurrent.futures page doesn’t mention macOS, but multiprocessing does. So, I hoped that a simple rewrite of the management command using multiprocessing might fix it.

Because I was in a rush and really didn’t want to do it I turned to an AI assistant for doing this boring work. To my surprise it immediately produced a version which I could easily fix by hand to produce a working version. Of course, the initial response was totally broken, removed code it wasn’t supposed to, and even the syntax was invalid. I didn’t expect more though, but what was surprising was that it actually felt like I had to do less work at this time.

The assistant also helped adding a --no-parallel flag to the management command. The output was even more broken than the output of the change mentioned above, but again, I could easily fix it to achieve what I wanted.

The fact that I know the code and git well certainly helped, the assistant would really have helped without that knowledge.

In the end, switching to multiprocessing didn’t help, but adding the --no-parallel flag allowed them to run the processing themselves by not spawning any additional threads or processes.

The energy use and the stealing of copyrighted material done by the AI companies is still really bad. It does feel somewhat OK to use an AI assistant in an area where I’m proficient as well and where I probably also supplied training material (without being asked if I wanted this) though. It’s making me slightly faster, and doesn’t allow me to do things I really couldn’t otherwise.

Releases

FeinCMS 24.12.3: I have added a TinyMCE 7 integration to FeinCMS.
django-imagefield 0.21.1: See above.

Weeknotes (2024 week 51)

2024-12-20T12:00:00Z

Weeknotes (2024 week 51)

Building forms using Django

I last wrote about this topic in April. It has resurfaced on Mastodon this week. I’m thinking about writing a feincms3-forms demo app, but I already have too much on my plate. I think composing a forms builder on top of django-content-editor is the way to go, instead of replacing the admin interface altogether – sure, you can always do that, but it’s so much less composable…

Releases

blacknoise 1.2: No real changes, added support for Python 3.13 basically without changing anything. It’s always nice when this happens.
django-imagefield 0.21
django-prose-editor 0.10: I rebuilt django-prose-editor from the ground up and wrote about that two weeks ago. The 0.10 release marks the final point of this particular rewrite.
django-js-asset 3.0: See the blog post from this week

Object-based assets for Django's forms.Media

2024-12-18T12:00:00Z

Object-based assets for Django’s forms.Media

The pull request for adding object-based script media assets into Django is in a good state and I hope it will be merged soon. I have been using object-based assets long before Django actually added support for them in 4.1 (since 2016, that’s before Django 1.10!) by using a gross hack. Luckily I have been able to clean up the code when Django 4.1 landed.

I have been asking myself at times why I haven’t proposed the change to Django myself despite having been a user of something like this for such a long time. After all, I have been happily contributing issue reports, bug fixes and tests to Django. The process of adding new features sometimes is terribly frustrating though even when looking (and cheering) from the sidelines. It feels bad that adding another package to the list of packages I maintain so clearly seems to be the better way to get things done compared to proposing a new feature for Django itself. I hope processes change somewhat.

But I digress.

The ProseEditorWidget in django-prose-editor wants to ship CSS, JavaScript and some JSON to the browser for the widget. So, of course I used object-based media assets for this instead of widget HTML templates. Media assets are deduplicated and sorted by Django. If different editor presets use differing lists of assets they are smartly merged by forms.Media using a topological sort. You get those niceties for free when using forms.Media and everything just works, so what’s not to like?

The only thing which isn’t to like is that Django, at the time of writing, doesn’t provide any classes helping with this. You can put strings into forms.Media or you can put objects with a __html__() method in there. The latter of course is all that’s needed to support more advanced use cases – and that’s exactly what django-js-asset now provides, and what django-prose-editor uses.

django-js-asset has long supported a JS class with support for additional attributes, for example:

from js_asset import JS

forms.Media(js=[
    JS("asset.js", {"id": "asset-script", "data-answer": "42"}),
])

Since 3.0 the package also comes with a CSS and JSON class:

from js_asset import CSS, JS, JSON

forms.Media(js=[
    JSON({"cfg": 42}, id="widget-cfg"),
    CSS("widget/style.css"),
    CSS("p{color:red;}", inline=True),
    JS("widget/script.js", {"type": "module"}),
])

This produces the following HTML:

<script id="widget-cfg" type="application/json">{"cfg": 42}</script>
<link rel="stylesheet" href="/static/widget/style.css">
<style>p{color:red;}</style>
<script src="/static/widget/script.js" type="module"></script>

The code which is proposed for Django supports the JavaScript use case but with a slightly different API:

from django.forms import Script

forms.Media(js=[
    Script("widget/script.js", type="module"),
])

This looks slightly nicer as long as you don’t use e.g. data attributes, because then you have to do:

from django.forms import Script

forms.Media(js=[
    Script("widget/script.js", **{"data-cfg": ...}),
])

I always forget that Python supports passing keyword arguments names which aren’t valid Python identifiers (but only when using **kwargs). I personally don’t care much either way, and when my packages can finally drop compatibility with Django versions which do not support all these functionalities yet I’ll finally be able to retire django-js-asset. That won’t happen any time soon though, if only because I like supporting old versions of Django because I have so many Django-based websites running somewhere.

Weeknotes (2024 week 49)

2024-12-06T12:00:00Z

Weeknotes (2024 week 49)

Django Steering Council elections

I have been thinking long and hard about running for the Django Steering Council. I think there are a few things I could contribute since I’ve been using Django for 16 or more years, and have been working on, maintaining and publishing third-party apps almost all this time. I have also contributed a few small features to Django core itself, and contributed my fair share of tests and bugfixes. The reason why I haven’t been more involved was always that I feared the review process with what I perceive to be a too unrestrained perfectionism. Teaching people is good, but I fear that those who teach are self-selected survivors of the process, which come to appreciate the perfectionism a bit too much. It’s somewhat the same as with the Swiss naturalization process – the hurdles are very high, and some of those who weather the process maybe are or grow to be too fond of it.

An important point is that this has nothing to do with being nice (or not). Everybody has always been great, maybe with the exception of myself back when I didn’t understand that the problem wasn’t the individuals but the way everyone has agreed things should be done.

I’m not the only one who thinks that we should improve the process somewhat. So, I’m definitely going to look out for candidates who think this is important.

There are a few reasons why I’m not running myself at this time. A somewhat important reason is that my candidacy wouldn’t help diversity at all. This shouldn’t discourage anyone else with the same background from running – we cannot change the world all at once. More importantly, I have more personal reasons for being hesitant to accept new commitments. That being said, I’m looking forward to be more involved in the community in other ways. And also, it’s not now or never.

Releases

feincms3-cookiecontrol 1.5.4: No functional changes, only code golfing. It’s nice to have a working cookie banner with a solution for embedding third party content only when people consent in less than 4KiB of minified (not compressed!) JavaScript.
django-admin-ordering 0.20: Objects can now be reordered using arrow buttons instead of drag drop or manually changing the ordering field’s value. This should make the package more accessible. It’s always a joy when people contribute such useful improvements.
django-prose-editor 0.10a?: See the recent blog post.

Rebuilding django-prose-editor from the ground up

2024-12-04T12:00:00Z

Rebuilding django-prose-editor from the ground up

The django-prose-editor package provides a HTML editor based upon the ProseMirror toolkit for the Django administration interface and for the frontend.

The package has been extracted from a customer project and open sourced so that it could be used in other projects as well. It followed a very restricted view of how rich text editors should work, which I have initially added to the FeinCMS repository when documenting the design decisions more than 15 years ago (Note that I didn’t edit the paragraph, it’s reproduced here as it was back then, with all the errors and heedlessness.)

All of this convinced me that offering the user a rich text editor with too much capabilites is a really bad idea. The rich text editor in FeinCMS only has bold, italic, bullets, link and headlines activated (and the HTML code button, because that’s sort of inevitable – sometimes the rich text editor messes up and you cannot fix it other than going directly into the HTML code. Plus, if someone really knows what he’s doing, I’d still like to give him the power to shot his own foot).

My personal views are unchanged. I have to recognize though that forcing this idea upon everyone isn’t workable and that this would mean that I’d have to find a different editor for most projects just because people really want or need more rope. Going back to an editor which allows everything was out of the question, so I had to look around for a way to allow project-specific extensions for the editor.

Of course that’s problematic, since Django packages and Python virtualenvs do not offer a good way of shipping CSS and JavaScript which should be available for a frontend bundler to process. The existing Django staticfiles app is great, works well, but it’s not a bundler – and it shouldn’t be.

So, I started shopping around for ways to make ProseMirror extensible while keeping extensions clean and well localized. Instead of inventing another plugin ecosystem I settled on Tiptap which uses ProseMirror under the hood. The abstractions are pleasantly leaky – if you know how to work with ProseMirror’s API, you can use Tiptap’s API without any issues. That was important for me, since I already have a somewhat large selection of plugins which I do not want to reimplement from the ground up.

I had already looked at Tiptap a few years back, but ultimately stayed with ProseMirror because I liked some behaviors better (such as not including trailing spaces in marks) and because I didn’t need the extensibility which at the time only made the resulting bundle much bigger.

Now, things have improved a lot, and I’m really happy with Tiptap and the development version of django-prose-editor. Writing an editor extension in project code is great, and my editor core stays nice. Also the list of readily available extensions is large, and most of the things just work.

Weeknotes (2024 week 47)

2024-11-20T12:00:00Z

Weeknotes (2024 week 47)

I missed a single co-writing session and of course that lead to four weeks of no posts at all to the blog. Oh well.

Debugging

I want to share a few debugging stories from the last weeks.

Pillow 11 and Django’s `get_image_dimensions`

The goal of django-imagefield was to deeply verify that Django and Pillow are able to work with uploaded files; some files can be loaded, their dimensions can be inspected, but problems happen later when Pillow actually tries resizing or filtering files. Because of this django-imagefield does more work when images are added to the system instead of working around it later. (Django doesn’t do this on purpose because doing all this work up-front could be considered a DoS factor.)

In the last weeks I suddenly got recurring errors from saved files again, something which shouldn’t happen, but obviously did.

Django wants to read image dimensions when accessing or saving image files (by the way, always use height_field and width_field, otherwise Django will open and inspect image files even when you’re only loading Django models from the database…!) and it uses a smart and wonderful¹ hack to do this: It reads a few hundred bytes from the image file, instructs Pillow to inspect the file and if an exception happens it reads more bytes and tries again. This process relies on the exact type of exceptions raised internally though, and the release of Pillow 11 changed the types… for some file types only. Fun times.

The issue had already been reported as #33240 and is now tracked as #8530 on the Pillow issue tracker. Let’s see what happens. For now, django-imagefield declares itself to be incompatible with Pillow 11.0.0 so that this error cannot happen.

rspack and lightningcss shuffled CSS properties

rspack 1.0 started reordering CSS properties which of course lead to CSS properties overriding each other in the incorrect order. That was a fun one to debug. I tracked the issue down to the switch from the swc CSS minimizer to lightningcss and submitted a reproduction to the issue tracker. My rust knowledge wasn’t up to the task of attempting to submit a fix myself. Luckily, it has been fixed in the meantime.

rspack problems

I have another problem with rspack where I haven’t yet tracked down the issue. rspack produces a broken bundle starting with 1.0.0-beta.2 when compiling a particular project of mine. I have the suspicion that I have misconfigured some stuff related to import paths and yarn workspaces. I have no idea how anyone could have a complete understanding of these things…

Update: The problem was #8027, experiments.css is quite broken for now.

Bundlers are complex beasts, and I’m happy that I mostly can just use them.

Closing thoughts

Debugging is definitely a rewarding activity for me. I like tracking stuff down like this. Unfortunately, problems always tend to crop up when time is scarce already, but what can you do.

Releases

Quite a few releases, many of them verifying Python 3.13 and Django 5.1 support (if it hasn’t been added already in previous releases). The nicest part: If I remember correctly I didn’t have to change anything anywhere, everything just continues to work.

django-admin-ordering 0.19: I added support for automatically renumbering objects on page load. This is mostly useful if you already have existing data which isn’t ordered yet.
feincms3-data 0.7: Made sure that objects are dumped in a deterministic order when dumping. I wanted to compare JSON dumps by hand before and after a big data migration in a customer project and differently ordered dumps made the comparison impossible.
django-prose-editor 0.9: I updated the ProseMirror packages, and put the editor into read-only mode for <textarea disabled> elements.
feincms3-language-sites 0.4: Finally released the update containing the necessary hook to validate page trees and their unique paths before moving produces integrity errors. Error messages are nicer than internal server errors.
django-authlib 0.17.2: The value of the cookie which is used to save the URL where users should be redirected to after authentication wasn’t checked for validity when setting it, only when reading it. This meant that attackers could produce invalid header errors in application servers. No real security problem here when using authlib’s code.
feincms3 5.3: Minor update which mostly removes support for outdated Python and Django versions.
django-imagefield 0.20: See above.

wonderfully ugly ↩

Weeknotes (2024 week 43)

2024-10-23T12:00:00Z

Weeknotes (2024 week 43)

I had some much needed time off, so this post isn’t that long even though four weeks have passed since the last entry.

From webpack to rspack

I’ve been really happy with rspack lately. Converting webpack projects to rspack is straightforward since it mostly supports the same configuration, but it’s much much faster since it’s written in Rust. Rewriting things in Rust is a recurring theme, but in this case it really helps a lot. Building the frontend of a larger project of ours consisting of several admin tools and complete frontend implementations for different teaching materials only takes 10 seconds now instead of several minutes. That’s a big and relevant difference.

Newcomers should probably still either use rsbuild, Vite or maybe no bundler at all. Vanilla JS and browser support for ES modules is great. That being said, I like cache busting, optimized bundling and far-future expiry headers in production and hot module reloading in development a lot, so learning to work with a frontend bundler is definitely still worth it.

Dark mode and light mode

I have been switching themes in my preferred a few times per year in the past. The following ugly bit of vimscript helps switch me the theme each time the sun comes out when working outside:

let t:light = 0
function! FiatLux()
    if t:light == 0
        :set background=light
        let t:light = 1
    else
        :set background=dark
        let t:light = 0
    endif
endfunction
nnoremap <F12> :call FiatLux()<CR>

I’m using the Ptyxis terminal emulator currently, I haven’t investigated yet if there’s a shortcut to toggle dark and light mode for it as well. Using F10 to open the main menu works fine though, and using the mouse wouldn’t be painful either.

Helping out in the Django forum and the Discord

I have found some pleasure in helping out in the Django Forum and in the official Django Discord. I sometimes wonder why more people aren’t reading the Django source code when they hit something which looks like a bug or something which they do not understand. I find Django’s source code very readable and I have found many nuggets within it. I’d always recommend checking the documentation or maybe official help channels first, but the code is also out there and that fact should be taken advantage of.

Releases

django-content-editor 7.1: Fixed a bug where the ordering and region fields were handled incorrectly when they appear on one line in the fieldset. Also improved the presentation of inlines in unknown regions and clarified the meaning of the move to region dropdown. Also, released the improvements from previous patch releases as a new minor release because that’s what I should have been doing all along.
form-designer 0.27: A user has been bitten by slugify removing cyrillic characters because it only keeps ASCII characters around. Here’s the wontfixed bug in the Django issue tracker: #8391. I fixed the issue by removing the slugification (is that even a word?) when generating choices.

Weeknotes (2024 week 39)

2024-09-25T12:00:00Z

Weeknotes (2024 week 39)

CSS for Django forms

Not much going on in OSS land. I have been somewhat active in the official Django forum, discussing ways to add Python-level hooks to allow adding CSS classes around form fields and their labels. The discussion on the forum and on the pull request goes in the direction of allowing using custom BoundField classes per form or even per project (instead of only per field as is already possible today). This would allow overriding css_classes, e.g. to add a simple class="field". Together with :has() this would probably allow me to skip using custom HTML templates in 99% of all cases.

I have also been lurking in the Discord, but more to help and less to promote my packages and ideas :-)

Releases

django-user-messages 1.1: Added Django 5.0, 5.1 to the CI, and fixed the migrations to no longer mention index_together at all. It seems that squashing the migrations wasn’t sufficient, I also had to actually delete the old migrations.
blacknoise 1.1: Starlette’s FileResponse has gained support for the HTTP Range header, allowing me to remove my homegrown implementation from the package. The blacknoise implementation is now half as long as it was in 1.0.
django-fhadmin 2.3: No new features, only tweaks to the styling and behavior prompted by updates to Django’s admin interface.
django-cabinet 0.17: I have pruned the CI matrix and accepted a pull request adding a ru translation. I feel conflicted about that since I strongly believe that everything is political, but I don’t know if rejecting translations helps anyone.

django-content-editor now supports nested sections

2024-09-13T12:00:00Z

django-content-editor now supports nested sections

django-content-editor (and it’s ancestor FeinCMS) has been the Django admin extension for editing content consisting of reusable blocks since 2009. In the last years we have more and more often started automatically grouping related items, e.g. for rendering a sequence of images as a gallery. But, sometimes it’s nice to give editors more control. This has been possible by using blocks which open a subsection and blocks which close a subsection for a long time, but it hasn’t been friendly to content managers, especially when using nested sections.

The content editor now has first-class support for such nested sections. Here’s a screenshot showing the nesting:

Finally it’s possible to visually group blocks into sections, collapse those sections as once and drag and drop whole sections into their place instead of having to select the involved blocks individually.

The best part about it is that the content editor still supports all Django admin widgets, as long as those widgets have support for the Django administration interface’s inline form events! Moving DOM nodes around breaks attached JavaScript behaviors, but we do not actually move DOM nodes around after the initialization – instead, we use Flexbox ordering to visually reorder blocks. It’s a bit more work than using a ready-made sortable plugin, but – as mentioned – the prize is that we don’t break any other Django admin extensions.

Simple patterns

I previously already reacted to a blog post by Lincoln Loop here in my post My reaction to the block-driven CMS blog post.

The latest blog post, Solving the Messy Middle: a Simple Block Pattern for Wagtail CMS was interesting as well. It dives into the configuration of a Wagtail stream field which allows composing content out of reusable blocks of content (sounds familiar!). The result is saved in a JSON blob in the database with all the advantages and disadvantages that entails.

Now, django-content-editor is a worthy competitor when you do not want to add another interface to your website besides the user-facing frontend and the Django administration interface.

The example from the Lincoln Loop blog post can be replicated quite closely with django-content-editor by using sections. I’m using the django-json-schema-editor package for the section plugin since it easily allows adding more fields if some section type needs it.

Here’s an example model definition:

# Models
from content_editor.models import Region, create_plugin_base
from django_json_schema_editor.plugins import JSONPluginBase
from feincms3 import plugins

class Page(models.Model):
    # You have to define regions; each region gets a tab in the admin interface
    regions = [Region(key="content", title="Content")]

    # Additional fields for the page...

PagePlugin = create_plugin_base(Page)

class RichText(plugins.richtext.RichText, PagePlugin):
    pass

class Image(plugins.image.Image, PagePlugin):
    pass

class Section(JSONPluginBase, PagePlugin):
    pass

AccordionSection = Section.proxy(
    "accordion",
    schema={"type": "object", {"properties": {"title": {"type": "string"}}}},
)
CloseSection = Section.proxy(
    "close",
    schema={"type": "object", {"properties": {}}},
)

Here’s the corresponding admin definition:

# Admin
from content_editor.admin import ContentEditor
from django_json_schema_editor.plugins import JSONPluginInline
from feincms3 import plugins

@admin.register(models.Page)
class PageAdmin(ContentEditor):
    inlines = [
        plugins.richtext.RichTextInline.create(models.RichText),
        plugins.image.ImageInline.create(models.Image),
        JSONPluginInline.create(models.AccordionSection, sections=1),
        JSONPluginInline.create(models.CloseSection, sections=-1),
    ]

The somewhat cryptic sections= argument says how many levels of sections the individual blocks open or close.

To render the content including accordions I’d probably use a feincms3 renderer. At the time of writing the renderer definition for sections is a bit tricky.

from feincms3.renderer import RegionRenderer, render_in_context, template_renderer

class PageRenderer(RegionRenderer):
    def handle(self, plugins, context):
        plugins = deque(plugins)
        yield from self._handle(plugins, context)

    def _handle(self, plugins, context, *, in_section=False):
        while plugins:
            if isinstance(plugins[0], models.Section):
                section = plugins.popleft()
                if section.type == "close":
                    if in_section:
                        return
                    # Ignore close section plugins when not inside section
                    continue

                if section.type == "accordion":
                    yield render_in_context("accordion.html", {
                        "title": accordion.data["title"],
                        "content": self._handle(plugins, context, in_section=True),
                    })

            else:
                yield self.render_plugin(plugin, context)

renderer = PageRenderer()
renderer.register(models.RichText, template_renderer("plugins/richtext.html"))
renderer.register(models.Image, template_renderer("plugins/image.html"))
renderer.register(models.Section, "")

Closing thoughts

Sometimes, I think to myself, I’ll “just” write a “simple” blog post. I get what I deserve when using those forbidden words. This blog post is neither short or simple. That being said, the rendering code is a bit tricky, the rest is quite straightforward. The amount of code in django-content-editor and feincms3 is reasonable as well. Even though it may look like a lot you’ll still be running less code in production than when using comparable solutions built using Django.

Weeknotes (2024 week 37)

2024-09-11T12:00:00Z

Weeknotes (2024 week 37)

I have helped mentoring Aman Pandey who has worked all summer to add async support to django-debug-toolbar. Tim has released an alpha which contains all of the work up to a few days ago. Test it! Let’s find the breakages before the final release.

Dropping Python 3.9 from my projects

I have read Carlton’s post about the only green Python release and have started dropping Python 3.9 support from many of the packages I maintain. This is such a good point:

[…] I’m also thinking about it in terms of reducing the number of Python versions we support in CI. It feels like a lot of trees to support 5 full versions of Python for their entire life. 🌳

Releases

django-debug-toolbar 5.0.0a0: See above.
form-designer 0.26.2: The values of choice fields are now returned as-is when sending mails or exporting form submissions instead of only returning the slugified version.
django-authlib 0.17.1: The role-based permissions backend had a bug where it wouldn’t return all available permissions in all circumstances, leading to empty navigation sidebars in the Django administration. This has been fixed.
feincms3 5.2.3: Bugfix release, the page moving interface is no longer hidden by an expanded navigation sidebar. I almost always turn off the sidebar in my projects so I haven’t noticed this.
django-prose-editor 0.8.1: Contains the most recent ProseMirror updates and bugfixes.
django-content-editor 7.0.10: Now supports sections. Separate blog post coming up!

Weeknotes (2024 week 35)

2024-08-28T12:00:00Z

Weeknotes (2024 week 35)

Getting deep into htmx and django-template-partials

I have been skeptical about htmx for some time because basically everything the library does is straightforward to do myself with a few lines of JavaScript. I am a convert now because, really, adding a few HTML attributes is nicer than copy pasting a few lines of JavaScript. Feels good.

The combination of htmx with django-template-partials is great as well. I didn’t know I had been missing template partials until I started using them. Includes are still useful, but replacing some of them with partials makes working on the project much more enjoyable.

I haven’t yet had a use for django-htmx but I may yet surprise myself.

Releases

django-authlib 0.17: django-authlib bundles authlib.little_auth which offers an user model which uses the email address as the username. I have also introduced the concept of roles instead of permissions; now I have reorganized the user admin fieldset to hide user permissions altogether. Group permissions are still available as are roles. I’m personally convinced that user permissions were a mistake.
feincms3-forms 0.5: Allowed setting a maximum length for the bundled URL and email fields through the Django administration interface.
django-content-editor 7.0.7: Fixed a bug where plugins with several fieldsets weren’t collapsed completely.
django-imagefield 0.19.1: Allowed deactivating autogeneration of thumbnails completely through the IMAGEFIELD_AUTOGENERATE setting. This is very useful for batch processing. Also, documented all available settings.
django-prose-editor 0.8: Added support for translating the interface elements and for restricting the available heading levels in the UI.
form-designer 0.25: Fixed the type of the author field for the send-to-author processing action.
feincms3 5.2.2: Added support for embedding SRF play external content. They do not support oEmbed unfortunately.
feincms3-cookiecontrol 1.5.3: Added support for SRF play embeddings as well. The difference is that the feincms3-cookiecontrol embedding requires consent before embedding external content.