Matthias Kestenholz

Workbench, the Django-based agency software

2024-04-24T12:00:00Z

Workbench, the Django-based agency software

I get the impression that there’s a lot of interesting but unknown software in Django land. I don’t know if there’s any interest in some of the packages I have been working on; if not this blog post is for myself only.

(Hi)story time

As people may know I work at Feinheit, an agency which specializes in digital communication services for SMEs, campaigns for referendums, and website and webapp development. At the time of writing we are a team of about 20-25 communication experts, graphic designers, programmers and project managers.

We have many different clients and are working on many different projects at the same time and are billing by the hour¹. Last year my own work has been billed to more than 50 different customers. In the early days we used a shared file server with spreadsheet files to track our working hours. Luckily we didn’t often overwrite the edits others made but that was definitely something which happened from time to time.

We knew of another agency who had the same problems and used a FileMaker-based software. Their solution had several problems, among them the fact that it became hard to evolve and that it got slower and slower as more and more data was entered into it over the years. They had the accounting know how and we had the software engineering know how so we wrote a webapp based on the Django framework. As always, it was much more work than the initial estimate, but if we as programmers didn’t underestimate the effort needed we wouldn’t have started many of the great projects we’re now getting much value and/or enjoyment from, hopefully both. The product of that work was Metronom. The first release happened a little bit later than harvest but it already came with full time tracking including an annual working time calculator, absence management, offers, invoices including PDF generation etc, so it was quite a bit more versatile while still being easier to use than “real” business software solutions.

I personally was of the opinion that the product was good enough to try selling it, but for a variety of reasons (which I don’t want to go into here) this never happened and we decided that we didn’t want to be involved anymore.

However, this meant that we were dead-end street with a software that didn’t belong to us anymore, which wasn’t evolving to our changing requirements. I also didn’t enjoy working on it anymore. Over the years I have tried replacing it several times but that never came to pass until some time after the introduction of Holacracy at our company. I noticed that I didn’t have to persuade everyone but that I, as the responsible person for this particular decision, could “just”² move ahead with a broad interpretation of the purpose and accountabilities of one of my roles.

Workbench

Workbench is the product of a few long nights of hacking. The project was started as an experiment in 2015 and was used for sending invoices but that wasn’t really the intended purpose. After long periods of lying dormant I have brought the project to a good enough state and switched Feinheit away from Metronom in 2019.

I have thought long and hard about switching to one of the off-the-shelf products and it could very well be that one of them would work well for us. Also, we wouldn’t have to pay (in form of working hours) for the maintenance and for enhancements ourselves. On the other hand, we can use a tool which is tailored to our needs. Is it worth the effort? That’s always hard to answer. The tool certainly works well for the few companies which are using it right now, so there’s no reason to agonize over that.

At the time of writing, Workbench offers projects and services, offers and invoices incl. PDF generation, recurring invoices, an address book, a logbook for rendered services, annual working time reports, an acquisition funnel, a stupid project planning and resource management tool and various reports. It has not only replaced Metronom but also a selection of SaaS (for example Pipedrive and TeamGantt) we were using previously.

The whole thing is open source because I don’t want to try making agency software into a business anymore, I only want to solve the problems we have. That being said, if someone else finds it useful then that’s certainly alright as well.

The license has been MIT for the first few years but I have switched to the GPL because I wanted to integrate the excellent qrbill module which is also licensed under the GPL. As I wrote elsewhere, I have released almost everything under the GPL in my first few open source years but have switched to BSD/MIT later when starting to work mainly with Python and Django because I thought that this license is a better fit³ for the ecosystem. That being said, for a product such as this the GPL is certainly an excellent fit.

Final words?

There are a few things I could write about to make this into a series. I’m putting a few ideas here, not as an announcement, just as a reminder for myself.

Better time tracking
Patterns for creating recurring invoices
Feature switches and configurability
Coffee time

Rather, in six minute increments. It’s even worse. ↩
Of course it’s never that simple, because the responsibility is a lot. The important point is: It would have been a lot of work anyways, the big difference is that it was sufficient to get consent from people; no consensus required. ↩
That’s not meant as a criticism in any way! ↩

Building forms with the Django admin

2024-04-12T12:00:00Z

Building forms with the Django admin

The title of this post was shamelessly copied from Jeff Triplett’s post on Mastodon.

Why?

Many websites need a simple way of embedding forms, for example as a contact form or for simple surveys to collect some data or inputs from visitors. Django’s forms library makes building such forms straightforward but changing those forms requires programming skills and programmer time. Both of them may not be readily available. More importantly, sometimes it’s just nice to give more tools to web publishers.

The simple way to build something like this is to use a form builder such as Google Forms, Typeform, Paperform or anything of the sort. Those options work nicely. The downsides are that embedded forms using those services load slowly, look differently, cost a lot or collect a lot of data on users, or all of those options. Because of that there’s still a place for building such functionality locally.

If I wanted to use PHP and WordPress I could just use WPForms and call it a day. Since I do not actually want that this blog post is a bit longer.

The early days: form-designer

One of the first Django-based third party apps I published was the form-designer. The first version was uploaded to PyPI in 2012 but it had already been used in production for more than two years at that point in time. I had used Git submodules for the deployment back then, before switching to Python virtualenvs some time later (and never looking back!)

The form-designer is still maintained actively. Because of Django’s stability and because of the fact that the app doesn’t do all that much it doesn’t require much development at all.

The form designer supports a selection of standard HTML5 input fields out of the box and also has an optional django-recaptcha integration. All fields support some basic configuration such as setting a title, a help text, marking the field as required etc. Submissions can be sent to a configurable email address and can be saved in the database and later exported as an XLSX file. It’s also possible to define your own actions.

More flexibility needed: feincms3-forms

A few years back I mentioned feincms3-forms in a weeknotes entry. The reasons why form-designer wasn’t sufficient for a project back then are outlined in the blog post:

feincms3-forms – A new forms builder for the Django admin interface

For a current project we needed a forms builder with the following constraints:

Simple fields (text, email, checkboxes, dropdowns etc.)

Custom validation and processing logic

It should be possible to add other content, e.g. headings and explanations between form fields

The form_designer fulfilled a few of these requirements but not all. It still works well but I wanted a forms builder based on django-content-editor for a long time already. Also, I really like the feincms3 pattern where the third party app only provides abstract models. Yeah, it is much more work to start with but the flexibility and configurability is worth it – especially since it’s possible to write straightforward code to handle special cases[^2] instead of configuring even more settings.

The humble beginnings are here in the feincms3-forms repository. The test suite already shows how things work together but as of now no documentation exists and no release has been made yet. I hope it will be ready for a first beta release in the next few weeks 😄

Since then I have used feincms3-forms more often than form-designer, for building simple forms and also to build multi-step form wizards with custom fields, custom validation, configurable steps etc. The README now actually explains why the project exists and how it could be used.

It still doesn’t come close to WPForms in terms of included functionality; a big feature which is missing is conditional logic because I haven’t yet had a use for it.

The feincms3-forms forms support all types of content between form fields (basically everything django-content-editor supports). Plugins for form fields are more flexible and can add as many input fields to the form as they want, you’re not restricted to single values or single input fields.

Packages

Weeknotes (2024 week 14)

2024-04-06T12:00:00Z

Weeknotes (2024 week 14)

I’m having a bit of a slow week with the easter weekend and a wisdom tooth extraction. I’m recovering quite quickly it seems and I’m glad about it.

This weeknotes entry is short and quick. I’m trying to get back into the habit of writing them after a mediocre start this year.

20th Anniversary Celebration of Young Greens Switzerland

I have attended the celebration of Young Greens Switzerland. I have been a founding member and have been active for close to ten years. A lot of time has passed since then. It has been great to reminisce about old times with friends and, more importantly, to see how the torch is carried on.

Releases

blacknoise 0.0.5: blacknoise is an ASGI app for static file serving inspired by whitenoise. It only supports a very limited subset of whitenoise’s functionality, but it supports async.
html-sanitizer 2.4.1: The lxml library moved the HTML cleaner into its own package, lxml-html-clean; this release adds support for that. I didn’t know that the HTML cleaner is viewed as being problematic by the lxml maintainers. I’m having another look at nh3 and will maybe switch html-sanitizer’s guts from lxml to nh3 in the future.
django-tree-queries 0.18: django-tree-queries now supports ordering siblings by multiple fields and even allows descending orderings.
django-cabinet 0.14.2: This release fixes the CKEditor 4 filebrowser popup when using Django 5 or better.

The Django admin is a CMS

2024-03-27T12:00:00Z

The Django admin is a CMS

The post Why is the Django Admin “Ugly”? and the discussion on Mastodon around it finally motivated me to write down my thoughts regarding the recurring theme in Django land that the Django administration interface isn’t a CMS (Content Management System).

I think that this is misguided and needlessly limits the discourse around what the admin’s current functionality is and the ideas what it could be and already is.

A web content management system is about website authoring for users who do not need to be web programming experts in their own rights. Django was created at the Lawrence Journal-World newspaper. The admin itself was created to allow quickly spinning up new websites, where the admin interface was used by content managers to fill in the content while programmers finalized the rest of the website. So obviously the admin interface was a system used to manage content¹ from the beginning.

Sure, the Django admin site documentation states:

One of the most powerful parts of Django is the automatic admin interface. It reads metadata from your models to provide a quick, model-centric interface where trusted users can manage content on your site. The admin’s recommended use is limited to an organization’s internal management tool. It’s not intended for building your entire front end around. [emphasis added]

In other words, the Django documentation also points out that the admin is powerful and that it allows trusted users to manage content¹.

Yes, it will be very painful if you try to do everything on top of the Django admin site. The warnings against using the Django admin for more than it was designed to are necessary and I totally support them. As soon as you’re getting into workflows, into complex permission scenarios (sad noises) or similar things the admin definitely isn’t for you. But, the admin nicely solves 90% of the problems with 10% of the effort. And it’s very good at that.

And sure, if you try building your own frontend on top of the Django admin you’re in for a bumpy ride, but that much should be obvious.

Many third party apps for Django actually target the Django admin interface itself, and not one of the (excellent!) Django-based CMS such as Wagtail. This means that by building on the Django admin instead of one of the CMS you’re running less code, by using more libraries instead of frameworks (on top of frameworks) you’re keeping maintenance lower, and you’re a part of a larger community², which brings the potential benefit of being able to profit more from the general Django packages ecosystem.

Since you’re depending on smaller pieces of additional software it will generally be possible to upgrade to new Django versions quicker. This isn’t true for all packages of course, and I’m a reluctant maintainer of some of them. Anecdotes aren’t data, but I see that some larger CMS systems are definitely having a hard time keeping up with Django’s release schedule.

I’m not trying to say that the Django admin is a better CMS than other Django-based CMS, or any other CMS. I’m saying it’s a trade off and you should be mindful of the downsides of choosing a larger system. And I’m saying that the people who tell you that you shouldn’t be using the Django admin interface are wrong in the first approximation.

The fact that it’s so easy to spin up an additional site and with minimal effort and still be able to work with clean database schemas and all the great tools Django (and Python) offers is important for those of us who are working on many different projects with limited financial resources, because the website often is for example just a small part of a campaign.

Sorry-not-sorry for my choice of words. ↩↩
The assumption that the communities of these Django-based CMS projects are a subset of the Django community itself shouldn’t be too controversial. ↩

blacknoise – ASGI app for static file serving

2024-03-20T12:00:00Z

blacknoise – ASGI app for static file serving

Note

This blog post consists of the blacknoise README at the time of publishing.

blacknoise is an ASGI app for static file serving inspired by whitenoise and following the principles of low maintenance software.

This is pre-alpha software and everything is subject to change. I’m not even sure if blacknoise should exist at all or if the energy wouldn’t be better spent improving whitenoise or other tools. Feedback and contributions are very welcome though!

Using blacknoise with Django to serve static files

Install blacknoise into your Python environment:

pip install blacknoise

Wrap your ASGI application with the BlackNoise app:

from blacknoise import BlackNoise
from django.core.asgi import get_asgi_application
from pathlib import Path

BASE_DIR = Path(__file__).parent

application = BlackNoise(get_asgi_application())
application.add(BASE_DIR / "static", "/static")

BlackNoise will automatically handle all paths below the prefixes added, and either return the files or return 404 errors if files do not exist. The files are added on server startup, which also means that BlackNoise only knows about files which existed at that particular point in time.

Improving performance

BlackNoise has worse performance than when using an optimized webserver such as nginx and others. Sometimes it doesn’t matter much if the app is behind a caching reverse proxy or behind a content delivery network anyway. To further support this use case BlackNoise can be configured to serve media files with far-future expiry headers and has support for serving compressed assets.

Compressing is possible by running:

python -m blacknoise.compress static/

BlackNoise will try compress non-binary files using gzip or brotli (if the Brotli library is available), and will serve the compressed version if the compression actually results in (significantly) smaller files and if the client also supports it.

Far-future expiry headers can be enabled by passing the immutable_file_test callable to the BlackNoise constructor:

def immutable_file_test(path):
    return True  # Enable far-future expiry headers for all files

application = BlackNoise(
    get_asgi_application(),
    immutable_file_test=immutable_file_test,
)

Maybe you want to add some other logic, for example check if the path contains a hash based upon the contents of the static file. Such hashes can be added by Django’s ManifestStaticFilesStorage or by appropriately configuring bundlers such as webpack and others.

License

blacknoise is distributed under the terms of the MIT license.

Podcasts I like listening to

2024-03-18T12:00:00Z

Podcasts I like listening to

I discovered listening to podcasts about one year ago. Previously, I never knew why anyone would want to listen to people talk when they could listen to music or nothing, but that has changed a bit.

So, here’s a list of podcasts I’m currently listening to on a regular basis.

Tech Won’t Save Us

I have recently stumbled over Tech Won’t Save Us, a Podcast which is critical of the technological “progress” offered by Silicon Valley elites. It’s a great antidote for the generative AI hype.

I still have some stupid hope that AGI will solve our problems because maybe people will trust a computer more than scientists that something has to be done about the combined crisis we’re facing as humans. Who knows. I do not want to be too negative about it though, there are positive news if you’re looking in the right places. I myself do not write about those bigger issues as much as I used to. I support those more important issues elsewhere.

Generative AI is certainly not helping me in my work. I do not want a computer to generate code which I have to review and maintain when I still enjoy writing code myself. Maybe that will change at some point in the future. When that happens I’ll probably retire and become a gardener or something.

The Ezra Klein Show

This show seems to be a favorite of many people. I joined for the (relatively) critical perspectives on AI and stayed for the insights into politics and into the near east conflict. I think it’s great that people from different sides get a voice on the show, even if Ezra doesn’t agree with them.

I sometimes wish the questions they ask on the show were a little bit more critical.

Hard Fork

I like the perspectives and the bantering on this podcast. It’s a good way for me to stay informed about what’s going on in AI/ML land, among other things. It’s a lighthearted listen I often look forward to.

Django Chat

I work with Django all the time and Django Chat is a great way to learn more about Django, about the people involved and also about the surrounding ecosystem. A wholehearted recommendation!

Talk Python To Me

Good interviews, interesting guests and topics. Just a great way to learn about libraries, tools etc. which I cannot wait to use in my own projects.

Datenschutzplaudereien

Required listening for people working with customer data in Switzerland. Swiss german knowledge required.

Others

A few other podcasts I’m listening to more or less regularly:

Wir müssen reden. Public Eye spricht Klartext: The podcast from public eye, an organization which uncovers human right violations perpetrated by Swiss companies around the globe. I’m not yet sure if I like the podcast, but I like and support the organization very much.
Hanselminutes: The more polifor the wide and diverse array of guests.tical podcasts have displaced Hanselminutes in my listening schedule, but I’d definitely recommend it to anyone who wants to learn more about tech. Recommended because of the diverse array of guests, among many other reasons.
Syntax: At times very interesting, at times not much new for me. I still recommend it.
Weird Studies: Really weird. I liked the discussion of Hellraiser and The Thing, and have listened to some of the other episodes. Religiosity and spiritualism was always around me when I was a child, and it’s interesting to revisit some of the ideas about reality from a different angle after years and years of not engaging with these questions at all. (Writing this down does sound a bit like a midlife crisis on the horizon, but I don’t think that’s it. I have studied a little bit of philosophy and ethics and have stayed interested in these topics ever since.)
I don’t think the german “Welt” is a good newspaper at all. I do like the «Aha! History» and «Aha! Zehn Minuten Alltagswissen» podcasts because they are short and often examine topics I don’t know much about.
Sternstude Philosophie: It’s a bit slow and definitely more geared towards viewers than listeners, but definitely interesting.

Weeknotes (2024 week 11)

2024-03-16T12:00:00Z

Weeknotes (2024 week 11)

Estimates

Jacob wrote an excellent post on breaking down tasks. I did like the post a lot. Maybe I’ll write a longer reply later, but for now just this. There definitely are good reasons for the pushback against estimation, and it’s really not just that some people lack professionalism.

Releases

django-cabinet 0.14.1: Mini release containing a Turkish translation. It’s always nice if software is used.
feincms3 4.6: Fixed a bug where the move form wouldn’t use a potentially overridden ModelAdmin.get_queryset method.
form-designer 0.24: Updated the package for django-recaptcha 4.0.
html-sanitizer 2.3.1: Fixed an edge case sanitization bug (luckily without security implications).
django-content-editor 6.4.2: django-content-editor now again supports transitioning plugin fieldsets when opening and closing thanks to CSS grid’s ability to animate the maximum height of an element. Also, the initialization in 6.4 was badly broken.
django-prose-editor 0.2: See the announcement blog post from Wednesday.

Eleven months of static site generation

2024-03-15T12:00:00Z

Eleven months of static site generation

Almost eleven months have gone by since I last wrote about the static site generator script I’m using for this website.

What has changed in the meantime?

The build script is 40 lines shorter.
Code snippets are now highlighted using pygments.
The script now not only reports how many files have been written but also the categories and their post count.
The script now supports a folder with static assets so that images etc. do not have to be hotlinked from elsewhere. Hotlinking images hosted in GitHub issues doesn’t work anymore it seems.
The execution time isn’t reported anymore.
The script now supports scheduled publishing, mainly by leveraging GitHub action’s scheduling support.

Has it been worth it?

The general advice seems to not write your own static site generator because you’ll spend time tweaking the generator instead of actually writing posts. This definitely doesn’t have to be the case. Sometimes it’s good to hack on a small projects just for the joy of it. I certainly enjoy the fact that my code (arguably 🙃) does something useful each time I publish a new post.

django-prose-editor – Prose-editing component for the Django admin

2024-03-13T12:00:00Z

django-prose-editor – Prose-editing component for the Django admin

During the last few days I have been working on a prose-editing component for the Django admin which will replace the basically dead django-ckeditor in all of my projects. It is based on ProseMirror, in my opinion the greatest toolkit for building prose editors for the web.

Here’s a screenshot:

django-prose-editor is in active development; it’s available on PyPI and is developed in the open here on GitHub. The version at the time of writing is 0.2, and it’s not yet used in production environments, only in staging/preview environments. That will soon change though.

Researching alternatives to django-ckeditor

I have spent a lot of time evaluating alternatives. All of them are great choices, and I don’t want to bash any of them. But, I didn’t feel good betting on any of the choices from the WYSIWYG editors grid on Django Packages.

A few packages have potential licensing issues. CKEditor 5’s change to the GPL is what basically killed django-ckeditor. The upcoming TinyMCE 7 version will also change to the GPL according to the TinyMCE GitHub repository. Froala only has a free trial. The django-summernote app doesn’t have a dedicated maintainer, so I wouldn’t bet on it. django-prose uses Trix; there are various reasons why I didn’t want to bet on Trix, among them my personal experience that it sometimes gets into a buggy state where only a full page reload unfreezes the editor.

Other packages are basically Markdown editors. I like Markdown and use it for my blog. I don’t think Markdown is a good choice for a CMS which is used by people of many different skill levels. I don’t want to teach people to use the Markdown link syntax or the heading syntax even if the editor helps out a bit.

Some of the remaining choices are using JavaScript widgets which haven’t been updated for a really long time or they are really code editors, not WYSIWYG editors, also a no go.

Deciding on going with ProseMirror

I have worked with ProseMirror on and off since October 2015, soon after the crowdfunding ended. It is used in a project where people can write their own book with a standardized pipeline and process, where the technical side of the project is implemented using Django, ProseMirror and LaTeX. A hacked prosemirror-example-setup still is good enough for this project.

The ProseMirror deep dive came much later, only a few years back, when implementing an editor with more custom functionality such as annotations, different ways of marking up text and even interactive elements within the text, for example to use it as a cloze in teaching materials.

The learning curve is steep. I haven’t worked with another library which was so hard to get started with. It is my conviction that the reason for this is that rich-text editing is actually a hard problem. The ProseMirror architecture and implementation definitely makes sense when it finally clicks.

As additional bonuses the ProseMirror community is nice and ProseMirror is used in a few large software projects which make me believe that the software has a non-zero probability of being maintained in the long run.

Current plans

The only thing which is configurable right now is the server-side sanitizing of submitted HTML content. I plan on allowing some configurability, for example to disable links when the content will only be used inside a card teaser which itself is already a link. Too much configurability isn’t a good thing though in my mind, so I’ll probably be slow to add features and rather keep complexity low.

I’m definitely interested to hear from people who want to use the package but cannot do so right now or who would want some additional features. Issues and pull requests are welcome!

My 2024 Development Setup

2024-02-28T12:00:00Z

My 2024 Development Setup

I have been inspired by Jeff Triplett’s post to write this down. The main value – if there’s value at all in this post – lies in my ability to revisit it later and see if anything changed.

Desk

I’m using a standing desk since 2015 or 2016. I’m actually doing most of my work standing; when I’m tired or maybe after a sports session I sit down, but when I do that I mostly choose the sofa or a different table and get back to the standing desk as soon as possible. I have started using the standing desk when I was in therapy for severe back pain but I have come to like it a lot since then. I do have a chair but I don’t really use it all that often.

The greatest thing about a standing desk is that when I go to an event which involves a lot of standing around (or dancing!) I don’t get tired as fast as before :-)

Hardware

My main machines are two Thinkpad notebooks, a Lenovo X1 carbon and a Lenovo X1 nano. I bought the latter because my previous notebook broke during a warranty repair by a technician and I really really had to have a notebook for the coming days and I already had ordered the former, but it wasn’t to be delivered for several weeks. That was annoying. The good thing about it is that I normally do not have to carry around anything when going to the office, since I leave one of them in the office most of the time. I’m using a 27” WQHD monitor when I work. I have used a 32” 4k monitor for some time, but I have noticed that I’m starting to spend time searching for windows. I don’t like curved or widescreens monitors at all (I have tried them). I thought I’d never reach the point when I would reduce the available screen size voluntarily, but here I am.

I also have a desktop machine at home, but I mostly use it for gaming, and only a little development on the side. It has a 11th gen i7, 32 GB ram and a GeForce RTX 30 series graphics card.

This also means that I’m switching computers all the time, and basically everything has to be synced to the cloud all the time. I’m using git since the spring of 2006 and push everything all the time anyway. And when I’m not, I’m mostly using web-based software. This means that I mostly don’t care about backing up my data, I let the cloud companies and the NSA do it.

Software

After many years on macOS and a few years on Windows with WSL I am back to using Linux without a VM. I am using Fedora because I actually like the unmodified GNOME 3, and because I prefer flatpaks to snaps. RPM/DNF are alright, but I’m still more accustomed to DEB/APT.

I have left macOS because of the touch bar and the terrible keyboard, and Windows because I don’t want ads everywhere on my computer.

Development

I use a terminal emulator (gnome-terminal on Linux, alacritty on Windows/WSL) to run tmux, and start everything I need directly in there.

I have tried escaping vim many times but have always failed. (No, not exiting vim, thanks for your help.) The various VIM modes in popular editors such as VisualStudio Code just do not cut it. I’m currently editing code in neovim with a few plugins. The only really important plugin is ctrlp.vim. Ctrl-P with fuzzy matching is a great way to open files. I like vi, and I tell people that I use it when I’m asked but I do not suggest to people that they should use it as well. The learning curve isn’t worth it except maybe if you do a lot of system administration, I don’t know.

I have a VisualStudio Code installation laying around for using Live Share. I think it’s a great way for helping others solve a particular problem in a project they are working on, much better than screen sharing and probably even better than sitting at a table together, just because having two keyboards and cursors is so nice.

I mostly only use virtualenvs created using python3 -m venv venv for local development, no containers or anything. I definitely see the value of containers for deployment but I haven’t yet gotten around to setting things up for local development. This means that my machine always (also) runs a PostgreSQL and a redis server and maybe some other tools. Python 3 upgrades are mostly painless. NodeJS upgrades have sometimes been painful, but I avoid most of the pain by upgrading relatively late.

Weeknotes (2024 week 07)

2024-02-16T12:00:00Z

Weeknotes (2024 week 07)

This is a short weeknotes entry which mainly contains a large list of releases. The reason for the large list is that I haven’t published a weeknotes entry in weeks.

Releases

form-designer 0.23: Only small changes, mainly updated the package for current Django and Python versions.
feincms3-cookiecontrol 1.4.6: A minor change: Swallow exceptions which happen during startup when clobbering the scripts data fails. As an aside: I find it funny that I have discovered the .f3cc class in some cookie banner blocklists. It feels good to be recognized even if this maybe isn’t the nicest way, but it works for me since I actually do not like cookie banners either. At least feincms3-cookiecontrol doesn’t inject anything without users’ consent, and doesn’t require a third party service to run.
django-simple-redirects 2.2.0: Minor release which adds a search field to the admin changelist. django-simple-redirects is a repackaged version of django.contrib.redirects without the django.contrib.sites dependency.
speckenv 6.2: django_cache_url now supports parsing redis configuration for a leader-replica redis installation with a read-write leader host and read-only replica hosts. I use the same configuration format as django-cache-url does.
django-debug-toolbar 4.3: I haven’t done much here, just some reviewing here and there. I enjoy the Djangonaut Space contributions a lot.
django-cabinet 0.14: I have removed the constraint which enforces unique names for subfolders. Enforcing the uniqueness does make sense, but it also makes bulk-updating the media library using serialized data more painful than it should be. It’s a clear case of worse is better for me. If people want to confuse themselves I’m not going to stop them (anymore, in this case) but it makes the rest of the code so much easier to write that it’s not even funny.
html-sanitizer 2.3: This release contains a nice contribution which removes some whitespace which has been added by the sanitizer when merging adjacent tags of the same type, e.g. <strong>abc</strong><strong>def</strong>.
django-ckeditor 6.7.1: See above.
django-json-schema-editor 0.0.11: Fixed a crash which happened when not providing the optional (!) configuration. Shit happens. I should really have a test suite for this package.
feincms3 4.5.2: Disables the CKEditor version check.
django-content-editor 6.4: The first release since December 2022! Very stable software. The editor now restores the collapsed state of inlines and the scroll position when using “Save and continue editing”. This is especially useful if editing an object with many content blocks.

django-ckeditor

2024-02-14T12:00:00Z

django-ckeditor

It has finally happened. The open source version of CKEditor 4 does not contain fixes for known problems, see the CKEditor 4.24.0 LTS announcement.

I totally get why the CKEditor developers did this and can only thank them for all the work that went into the editor.

I wish I didn’t have to do the migration work to move basically everything to a different editor. The CKEditor 4 LTS version is only expected to be supported until the end of 2026 and I have a few projects which will be around far longer than this (or at least I hope so). Therefore, buying the LTS package would only delay the inevitable. CKEditor 5 is a completely different editor and uses the GPL license, so that’s not really an option either. TinyMCE is well known and I have been using it much earlier in my career, but reimplementing plugins isn’t fun to do.

I would prefer moving everything to ProseMirror or some other structured editor, but we have so much legacy content contained in HTML blobs which do not use any schema at all that this isn’t workable unfortunately.

Stay tuned for updates – they will come since I unfortunately cannot just ignore this problem.

Which brings me to django-ckeditor. CKEditor shows a very annoying popup to users when it detects a newer version with security fixes, see the GitHub issue. It’s not a bad idea but users cannot do much about it, so I opted to disable the version check (and warning) in django-ckeditor and replaced it with a Django system check which annoys developers instead.

The release containing these changes is available as django-ckeditor 6.7.1 on PyPI.

Weeknotes (2024 week 03)

2024-01-17T12:00:00Z

Weeknotes (2024 week 03)

Djangonaut Space

I wish all participants a good time and much success. I do not have anything to do with it really but I enjoy the idea a lot and maybe there will be a pull request or two to review.

Kubernetes

After years and years of hosting all sites on VPS I have finally reached the point where the old setup is more annoying to work with than switching to a new one. I have searched long for a solution which wasn’t as limited as some PaaS and as complex as going full Kubernetes, and where I can still delegate the responsibility of actually keeping things up and running to other people. In the end I have now accepted that such a thing doesn’t exist; either you have the limitations of a ready made solution, the limitation of having to open many many support tickets or the problem of having to learn Kubernetes (or something similar) with its extremely steep learning curve.

After spending days with it I’m slowly getting to the point where setting up local development environments and deploying changes is fun again. I’m using the GitOps paradigm; while I’m still building and uploading Docker (podman) images from the local development environment everything else is automated and goes through a Git based process. That’s much nicer than clicking around in some interface or copy pasting obscure commands into the console.

The biggest problem I encountered was (perhaps unsurprisingly) managing secrets as a team. It seems to me that while sealed secrets work great as an individual developer they don’t really offer straightforward solutions to avoid different people overwriting and resetting each others secrets when updating them. I’m a happy user of the external secrets operator and using some cloud service to actually store those secrets.

I have started using granian in production. I like the idea of a Rust-based ASGI/WSGI server. Nothing mission critical yet. My idea is to build confidence in the software stack.

Compulsory social measures

The more I learn about how Switzerland treats its citizens the more I wonder about the ways in which humans can mistreat other humans in a so called civilized and peaceful society.

It’s not exactly a new topic for me, but working on platforms which help remember and which help introducing people to the history certainly causes a heightened awareness for issues such as these.

Releases

feincms3-sites 0.20.2: It previously wasn’t possible to filter the list of sites to only show those sites which do not have a default language. This has been fixed.
django-json-schema-editor 0.0.10: It’s now actually possible to use the JSON editor outside inlines! That was a fun bug… not. Apart from the mentioned bug this new release mostly contains fixes to the styles. We’re slowly getting there.
django-mptt 0.16: I didn’t do anything except for the changelog and the release. That’s alright. The release contains a few minor fixes.

Weeknotes (2024 week 01)

2024-01-03T12:00:00Z

Weeknotes (2024 week 01)

First weeknotes post for 2024! Happy new year!

Looking back on 2023

Writing

I have published almost 40 posts last year. That’s almost as many posts as I published in the time period from 2014 to 2023. Coworking to write more does work.

I already had a quite active blog from 2005 to 2008 with a few posts after that; everything before 2014 was in german and mainly concerned with green politics and climate change. I’m still very interested in these topics but I don’t feel as if I have much to add to the conversation, even though it’s the more important issue.

Open Source

Not much changed here. I enjoy basically everything I do in open source land, and co-maintaining the Django Debug Toolbar with Tim is a joy.

I still wish that some of my projects had more impact in Django land, especially those who augment the Django administration interface to be a lightweight CMS which requires very little maintenance and work in the long run. I think it’s great that one of the core components, django-content-editor, hasn’t required a release in more than one year. It doesn’t have to be expanded because it just works. It would be great if there was a good way to distinguish between software which basically doesn’t require any updates and software which is abandoned.

The only project which doesn’t bring me much joy is django-mptt. Most people accept that there are no guarantees, but some people are just rude in the way in which they expect others to do the work. The first reaction was to return the blow and I’m glad that I didn’t give in to the temptation to do that. It’s basically never worth it to do that in writing.

Family

We had a good 2023 together and I’m very much looking forward to a just as good 2024.

Plans for 2024

I have bought a ticket for the DjangoCon Europe 2024 in Spain and I’m very much looking forward to that.

Maybe we’ll visit a music festival again this summer. After listening to a lot of metal music in the last ten or more years I rediscovered the dark side of D’n’B. Good times.

Advent of Code

I have finished the Advent of Code with some help from the Subreddit. Almost all of the puzzles were fun to think about. I didn’t have fun solving each and everyone of them, especially not the second part of a few of the later days. I feel good checking out solutions from other people in the subreddit, and maybe adding newly gained ideas to my code or even running someone else’s code 1:1 on my data after studying the algorithms used and hopefully learning something.

I had a good time and enjoyed shutting down the computer after that.

Releases

FeinCMS 23.12: A few minor changes to the thumbnailing code. It’s always nice to hear from people who are still using this project.

Weeknotes (2023 week 50)

2023-12-15T12:00:00Z

Weeknotes (2023 week 50)

django-imagefield

The path building scheme used by django-imagefield has proven problematic: It’s too likely that processed images will have the same path.

I have changed the strategy used for generating paths to use more data from the source; it’s now possible (and recommended!) to set IMAGEFIELD_BIN_DEPTH to a value greater than 1; 2 or 3 should be sufficient. The default value is 1 which corresponds to the old default so that the change won’t be backwards incompatible. However, you’ll always get a deprecation warning if you don’t set a bigger value yourself. The default will probably change in the future.

Advent of Code

I have always felt a bit as an imposter because I do not have any formal CS education; not so much in the last few years but certainly earlier in my career. I have enjoyed participating in the Advent of Code 2022 a lot and I have definitely learned to know when to use and how to use a few algorithms I didn’t even know before. I’m again working through the puzzles in my own pace and have managed to solve almost all of them up to today this year. There still are some puzzles where I don’t even know how to start the second part 😅.

Hosting

We’re still hosting most sites on virtualized servers, without any containers or any of the new stuff. I’m finally reaching the point where the downsides of this approach start to drag new projects down and the workarounds start looking worse than maybe switching to containers or even Kubernetes. Wish me luck, I’m more confused than I’ve been in years.

Health

To absolutely nobody’s surprise the family and myself have continued to be sick in the last two weeks. Nothing really bad happened, so we’re still lucky.

There’s unfortunately no way to solve a societal problem individually, so that will probably continue to be our life for now.

Releases

django-imagefield 0.18: See above.
feincms3-sites 0.20.1: Added additional validation (cleaning) checks. Showing error messages is preferrable to crashing with IntegrityError exceptions after all.
django-js-asset 2.2.0: Hatchling seems to dislike it if the project name and the Python module name do not match. I actually like django-js-asset’s Python module to be js_asset but I’m beginning to rethink this decision.
django-json-schema-editor 0.0.4: See the post from this week.

django-json-schema-editor

2023-12-13T12:00:00Z

django-json-schema-editor

I have extracted a JSON editing component based on @json-editor/json-editor from a client’s project and released it as open source. It isn’t the first JSON editing component by far but I like it a lot for the following reasons:

It works really well.
It supports editing arrays of objects using a tabular presentation. Tabular isn’t always better, but stacked definitely isn’t always better as well.
The data structure is defined as JSON schema,the data which is being entered is validated on the server using the fastjsonschema library. Having a schema and schema-based validation fixes most problems I have with less structured data than when using only Django model fields (without JSON).

Here’s a screenshot of the editing component used as a django-content-editor plugin:

Within the first few days of having released the package it has already proven useful in several other projects. A pleasant (but not totally unexpected) surprise.

Weeknotes (2023 week 48)

2023-11-30T12:00:00Z

Weeknotes (2023 week 48)

A few weeks have passed since the last update. The whole family was repeatedly sick with different viruses etc… I hope that the worst is over now. Who knows.

12-factor Django storage configuration

I should maybe write a longer and separate post about this, but speckenv has gained support for the Django STORAGES setting. No documentation yet, but it supports two storage backends for now, the file system storage and django-s3-storage, my go-to library for S3-compatible services.

Using it looks something like this:

from speckenv import env
from speckenv_django import django_storage_url

STORAGES = {
    "default": django_storage_url(
        env(
            "STORAGE_URL",
            default="file:./media/?base_url=/media/",
            warn=True,
        ),
        base_dir=BASE_DIR,
    ),
    "staticfiles": {
        "BACKEND": "django.contrib.staticfiles.storage.ManifestStaticFilesStorage",
    },
}

Then, if you want to use S3 you can put something like this in your .env file:

STORAGE_URL=s3://access-key:secret@bucket.name.s3.eu-central-1.amazonaws.com/media/

Or maybe something like this, if you want to serve media files without authentication:

STORAGE_URL=s3://access-key:secret@bucket.name.s3.eu-central-1.amazonaws.com/media/?aws_s3_public_auth=False&aws_s3_max_age_seconds=31536000

Releases

speckenv 6.1.1: See above.
feincms3-meta 4.6: York has contributed support for emitting structured data records. Looks nice. No documentation yet.
django-tree-queries 0.16.1: .values() and .values_list() queries are now handled better and more consistently than before.

Weeknotes (2023 week 44)

2023-11-02T12:00:00Z

Weeknotes (2023 week 44)

Unmaintained but maintained packages

There’s a discussion going on in the django-mptt issue tracker about the maintenance state of django-mptt. I have marked the project as unmaintained in March 2021 and haven’t regretted this decision at all. I haven’t had to fix inconsistencies in the tree structure once since switching to django-tree-queries. And if that wasn’t enough, I get little but only warm and thankful feedback for the latter, so that’s extra nice.

Despite marking django-mptt as unmaintained I seem to be doing a little bit of maintenance still. I’m still using it in old paid projects and so the things I do to make the package work for me is paid work. I’m not personally invested in the package anymore, so I’m able to tell people that there are absolutely no guarantees about the maintenance, and that feels good.

Read the Docs

I do understand why the .readthedocs.yaml file is now necessary. I wish that I wouldn’t have to do all the busywork of adding one to projects. I have just resubscribed to the Read the Docs Gold Membership which probably has expired at some point in the past. Read the Docs is excellent and everybody who can should support them.

Releases

feincms3 4.5, feincms3-sites 0.20 and feincms3-language-sites 0.3: Fixed the check which only allows adding an application through the CMS to the page tree (yes, that’s right) once; feincms3 worked fine, feincms3-language-sites by accident but feincms3-sites didn’t.
towel 0.31: Towel is one of my oldest packages which is still being used in real-world projects. Towel is a tool for building CRUD-type applications and is designed to keep you DRY while doing that. The project has been heavily inspired by a Django-based agency software I built many years back. The package even has docs! I’m still quite proud of the mostly transparent support for multitenancy, but apart from that I haven’t used it in many new projects.

Weeknotes (2023 week 43)

2023-10-27T12:00:00Z

Weeknotes (2023 week 43)

Switching to ruff format

ruff v0.1.2 includes the beta version of the ruff formatter. It formats the code almost the same way as black, but much much faster. I have started switching projects I’m working on and the experience has been nothing but great.

100% test coverage

I have again fixed a bad bug in code which had 100% test coverage. It’s not news that 100% test coverage isn’t a recipe for bug-free code, and it’s probably not even worth optimizing for that metric in code. It’s nothing more but a proxy for what you should really measure. Nevertheless it certainly does feel good to achieve high coverage values!

Releases

No releases at all.

Customize the Django admin to differentiate environments

2023-10-19T12:00:00Z

Customize the Django admin to differentiate environments

We often have the same website running in different configurations:

Once as a production site.
Once as a place where editors update and preview the content. The content is later automatically (and maybe partially) transferred from this environment to the production environment.
Once as a stage environment to stabilize the code.
And maybe additional environments for local development.

The Django admin panel mainly uses CSS variables for styling since theming support was introduced in Django 3.2 (by yours truly with a lot of help from others). This makes it simple and fun to customize the colors of all interface elements in a straightforward way without having to write loads of CSS.

If you have a ENVIRONMENT context variable available (as we do) you could add the following template as admin/base.html to your project, giving you a red color scheme for the production environment (to discourage people from updating content) and a nice scheme for the preproduction environment which clearly deviates from the standard color scheme used everywhere else:

{% block extrahead %}
  {{ block.super }}
  <style>
#site-name::after {
  content: " ({{ ENVIRONMENT }})";
  font-size: 60%;
}
  </style>
  {% if ENVIRONMENT == 'production' %}
    <style>
:root {
  --primary: #aa0000;
  --secondary: #810000;
  --accent: yellow;
}
    </style>
  {% elif ENVIRONMENT == 'preproduction' %}
    <style>
:root {
  --primary: #30b181;
  --secondary: #1f7957;
  --accent: #cdffea;
}
    </style>
  {% endif %}
{% endblock %}