• models.py

• settings.py

• urls.py

Apparently, you need these 3 files to start Django project the usual way. Most python microframework like Flask or Bottle allows you to start your new app in just a single file. Imagine you can do something like this:-

from siteplan import App, run
from django.http import HttpResponse

def index(request):
    return HttpResponse("Hello world")

from django.urls import path
urlpatterns = [
    path("test/", index),
]

conf = {}
app = App(conf, urls=urlpatterns)

if __name__ == "__main__":
    run(app)

Trying to run Django from a single file was one of my favorite past time activities but most failed, due to the constraints mentioned above.

I have almost gotten it work with siteplan, except when it comes to defining your model. In the above example, if you try to define a models like this:-

from siteplan import App, run
from django.http import HttpResponse
from django.db import models

class TestModel(models.Model):
    name = models.CharField(max_length=255)

def index(request):
    return HttpResponse("Hello world")

It will fail with this error right away:-

 File "/home/kamal/.cache/pypoetry/virtualenvs/siteplan-G2sCj5Qw-py3.11/lib/python3.11/site-packages/django/conf/__init__.py", line 63, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

So you can only import django.db.models after your app has been fully initialized. My idea was to create a dummy siteplan.models and later on attach the models that the user has defined in their app.py to siteplan.models.

Even though not ideals, just for some experimentation I try to defer the models initialization by defining the models in a function which then gets passed to the App instance. This way I evaluate the models definition after I have configured the settings.

def create_models():
    from django.db import models

    class TestModel(models.Model):
        name = models.CharField(max_length=255)

        __module__ = "siteplan.models"

    return [TestModel]

conf = {}
app = App(conf, urls=urlpatterns, models=create_models)

Then in App class's __init__:-

        default_conf.update(conf)
        base_conf.update(default_conf)

        if not settings.configured:
            settings.configure(**base_conf)

        from siteplan import models as models_module
        for mod in self.models():
            setattr(models_module, mod.__name__, mod)

While this get rid of the settings error, we got another error:-

 File "/home/kamal/.cache/pypoetry/virtualenvs/siteplan-G2sCj5Qw-py3.11/lib/python3.11/site-packages/django/apps/registry.py", line 136, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

This means we can only initialize the models after AppConfig.ready() method being called. This is tricky since AppConfig is a different class and we can only tell Django the path to that class.

I took a step back and searched around to see if it is possible to create django models dynamically. Turns out in some ways it is possible to create models dynamically even since django 1.0!

Baserow has been utilizing the method to the fullest as described in this blog post.

Even though I'm already at the losing end, but just for the sake of experimentation, I try to move initializing the models to the __call__ method of Siteplan. This finally works but definitely can't be used as the method will be called on every request.

Being able to define the models is one thing, but how to create the database tables? In normal Django, we generate a migrations file and then call the migrate command. Thanks to the Baserow's article above, I learned that we don't need to generate migrations files in order to create the database tables:-

    def __call__(self, environ, start_response):
        ...
        from django.db import connection
        for mod in self.models():
            setattr(models_module, mod.__name__, mod)

            with connection.schema_editor() as schema_editor:
                schema_editor.create_model(mod)

        return application(environ, start_response)

Django Schema Editor allows us to create database tables without migrations files!

If I still want to proceed, I can let user define the models either in dummy class or in yaml, and then use the same approach used by baserow to generate the models dynamically.

Some other unrelated notes of what I bumped into in the course of this exercise:-

• Django fastapi bridge

• Django-ninja - fastapi like functionality on top of Django Rest Framework

• Django ORM now has async queryset method! While not fully red yet, works are on the way to make everything async down to the database driver.

So it talks about using JavaScript without any build system. Aaah, a topic that is close to my heart. Ever remembered the days when you can just add some <script src="..." tag in your HTML and call it a day?

I'll just list a few things that are interesting to me here:-

• Julia's Vue project template - it just index.html + script.js

• Twind - I'm excited about this one. Always wish I can just use tailwindcss by simply adding the <script src= tag. This makes it possible. It's a compiler that will compile tailwind utility class into CSS on the fly.

• Fresh - Deno web framework. TIL that Deno is not fond of a build system.

• Supabase - ok, this one is not really related to the topic but I saw it on the Fresh chat demo. It claims to be an open-source Firebase alternative. Basically, it gave you hosted PostgreSQL database with a nice Firebase-like API to access it.

• HTMX no build step.

We're using pytest to run the tests and our conftest.py resembles what is mentioned in this Pyramid testing documentation. For example, we have this fixture defined in the file:-

@pytest.fixture(scope="session")
def app_env(ini_path: str) -> AppEnvType:
    """Initialize WSGI application from INI file given on the command line."""
    env = bootstrap(ini_path)

    # build schema
    alembic_cfg = Config("etc/alembic.ini")
    command.upgrade(alembic_cfg, "head")
    return env

This mean, in our test we just need to request this fixture and manipulate it before executing the function under tests.

def test_something(app_env, paramiko: mock.MagicMock,
                   celery_db: Session, demo_celery: None, ....):
    app_env["registry"].settings["special_config"] = False
    with pytest.raises(
        ValueError, match=r"not enough values to unpack \(expected 2, got 1\)"
    ):
        func_to_test()

But that special_config flag never set to False. Going through the documentation again, I'm pretty sure that's all I need to manipulate the settings before running my function under tests. But it looks like the settings are coming from somewhere else and not from my fixture.

After hours of pulling my hair, I did what those in desperation usually did. I start randomly removing stuff to see what could break. I removed celery_db and demo_celery from the fixture request. And something interesting happened. The test failed because of missing stuff.

src/kai/article/tests/test_article_tasks.py:586: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/lib/python3.9/site-packages/celery/local.py:191: in __call__
    return self._get_current_object()(*a, **kw)
src/kai/tasks/__init__.py:112: in __call__
    self.settings = app.conf["settings"]
.venv/lib/python3.9/site-packages/celery/utils/collections.py:449: in __getitem__
    return self.__missing__(key)

That app.conf things caught my eye. I started tracing where that app comes from. As we can see in the snippet above, it is being used in the modules src/kai/tasks/__init__.py I can see in the modules:-

from kai.celeryapp import app

And in src/kai/celeryapp.py:-

app = Celery()
app.user_options["preload"].add(add_preload_arguments)
app.steps["worker"].add(StructLogInitStep)

# Plaster parts
def setup(ini_path: str) -> None:
    """Given ini file, load settings and setup Celery."""
    loader = plaster.get_loader(ini_path)
    settings = loader.get_settings("celery")
    # TODO: this line can fail, but celery will swallow exception
    resolver = DottedNameResolver()
    celery = resolver.resolve(settings.get("use"))

    celery(app, loader)

So that's it! The function under tests is a celery task and it turns out that the celery task, is using a separate app instance, different from the main app. That's why my changes to the settings in the registry have no effect at all. It uses a different registry.

Knowing this, the fix to my test is simply:-

def test_something(paramiko: mock.MagicMock,
                   celery_db: Session, demo_celery: None, ....):
    from kai.celeryapp import app
    app.conf["settings"]["special_config"] = False
    with pytest.raises(
        ValueError, match=r"not enough values to unpack \(expected 2, got 1\)"
    ):
        func_to_test()

I'll just take the code example from the documentation linked above. Your feed's code will look like this:-

from django.contrib.syndication.views import Feed
from django.urls import reverse
from policebeat.models import NewsItem

class LatestEntriesFeed(Feed):
    title = "Police beat site news"
    link = "/sitenews/"
    description = "Updates on changes and additions to police beat central."

    def items(self):
        return NewsItem.objects.order_by('-pub_date')[:5]

    def item_title(self, item):
        return item.title

    def item_description(self, item):
        return item.description

    # item_link is only needed if NewsItem has no get_absolute_url method.
    def item_link(self, item):
        return reverse('news-item', args=[item.pk])

And will hook that feed class into urls.py like this:-

from django.urls import path
from myproject.feeds import LatestEntriesFeed

urlpatterns = [
    # ...
    path('latest/feed/', LatestEntriesFeed()),
    # ...
]

All is well and good until there's a new requirement that the feeds listed should be based on the user's current language. So how do we know what is the user's current language?

User's language can be inferred from the Request object as request.LANGUAGE_CODE. But no where in code above we can see a request object.

Usually, we will see the request object when we implement the views function, such as in the code below:-

from django.http import HttpResponse

def show_feed(request):
    return HttpResponse("Hello world")

I'm just too lazy to show the example above as CBV.

But our feed's code above looks nothing like views, be it a CBV or FBV. Depending on your knowledge and understanding of Django, you might see a hint that the above code is also a views or you will have no idea at all.

The most obvious hint would be the code in the urls.py where we hook up our Feed class to a url path. Let's see it again:-

urlpatterns = [
    # ...
    path('latest/feed/', LatestEntriesFeed()),   # ...
]

The second parameter to the path() function must be a views (or a urls include but let's save that for the other day). So LatestEntriesFeed() will act as a views. How can that be? We defined it as a class of Feed.

Some might be confused that we're passing class LatestEntriesFeed to the path() function. This is a common mistake that I observed among beginners or junior developers. The parentheses at the end make a difference.

path('latest/feed/', LatestEntriesFeed),

This means you're passing a class to the path function. While this:-

path('latest/feed/', LatestEntriesFeed()),

You're passing an instance of that class. Writing it like the below maybe will make it much clearer:-

feed_views = LatestEntriesFeed()
urlpatterns = [
    # ...
    path('latest/feed/', feed_views),   # ...
]

NOTES:-

While reading the source code for django syndication framework, I noticed the use of get_language() function from django.utils.translation. So the rest of this article is just purely an academic exercise ... :(

There are 2 ways we can figure out where the request object exists. First is by using breakpoint() to step up the caller of our method and see at which point request object exists. The second is by reading the source code.

Let us take a look at the first method. Put a breakpoint in our items() method.

class LatestEntriesFeed(Feed):
    title = "Police beat site news"
    link = "/sitenews/"
    description = "Updates on changes and additions to police beat central."

    def items(self):
        breakpoint()
        return NewsItem.objects.order_by('-pub_date')[:5]

When we make a request again, we should get something like this on our console:-

> /workspace/kai-site/src/kai_site/feeds.py(14)items()
-> return BlogPage.objects.live().order_by('-date')
(Pdb) l
  9        
 12         def items(self, request):
 13             breakpoint()
 14  ->         return BlogPage.objects.live().order_by('-date')
 15  
 16         def item_title(self, item):
 17             return item.title
 18  
 19         def item_description(self, item):

Step one level up will give us this:-

(Pdb) u
> /workspace/.pyenv_mirror/poetry/virtualenvs/kai-site-gqAITAgE-py3.10/lib/python3.10/site-packages/django/contrib/syndication/views.py(103)_get_dynamic_attr()
-> return attr(obj)

Step up again and we will see this:-

-> for item in self._get_dynamic_attr("items", obj):
(Pdb) l
175                 try:
176                     description_tmp = loader.get_template(self.description_template)
177                 except TemplateDoesNotExist:
178                     pass
179  
180  ->         for item in self._get_dynamic_attr("items", obj):
181                 context = self.get_context_data(
182                     item=item, site=current_site, obj=obj, request=request
183                 )
184                 if title_tmp is not None:
185                     title = title_tmp.render(context, request)

That's it! We can see the request object. So our Feed class does has the request object somewhere but it's not available to our method.

If the Feed class is implemented as CBV, then our instance's self will have the request object. This syndication framework I guess was implemented before the CBV implementation. The pattern used here is one of the proposed CBV implementation but as history told us, the class method approach is finally the one that gets implemented.

Btw, let's get back to the question of how our feed class can act like a Views. If you look into the source code of the syndication framework, we can see this line of code:-

class Feed:
    feed_type = feedgenerator.DefaultFeed
    title_template = None
    description_template = None
    language = None

    def __call__(self, request, *args, **kwargs):
        try:
            obj = self.get_object(request, *args, **kwargs)
        except ObjectDoesNotExist:
            raise Http404("Feed object does not exist.")
        feedgen = self.get_feed(obj, request)
        response = HttpResponse(content_type=feedgen.content_type)
        if hasattr(self, "item_pubdate") or hasattr(self, "item_updateddate"):
            # if item_pubdate or item_updateddate is defined for the feed, set
            # header so as ConditionalGetMiddleware is able to send 304 NOT MODIFIED
            response.headers["Last-Modified"] = http_date(
                feedgen.latest_post_date().timestamp()
            )
        feedgen.write(response, "utf-8")
        return response

The class implemented the magic method __call__ which will turn an instance of that class into a callable. In Python, a callable is an object that we can call (doh ...). The most common callable is a function. A class is also callable. Calling a class will return an instance of that class.

Now let's get into the actual question we have - how do we get the current language from our items() method? By default, the items() method get called without any parameter. But if we add a second parameter to the method, it will pass any object returned by the get_object() method. The default implementation in the Feed class only returns None so we need to override the method.

class LatestEntriesFeed(Feed):
    title = "Police beat site news"
    link = "/sitenews/"
    description = "Updates on changes and additions to police beat central."

    def items(self, request):
        language = request.LANGUAGE_CODE
        return (NewsItem.objects.filter(language=language)
                .order_by('-pub_date')[:5])

    def item_title(self, item):
        return item.title

    def get_object(self, request, *args, **kwargs):
        return request

As we can see above, get_object() get passed a request object. The actual purpose of this method is to return our Feed object but since we don't have any, we just return the request object. The object then will get passed to items() the method as second parameter. If we do have a feed object to return, then can attach the request object to that object instead.

And if there's no translation yet, add link for creating new translation like this:-

So I was scouring wagtail github repo looking for the url name to be used in the template. The url name for showing the translated version is easy, since it just an edit page. We can find it in wagtail/admin/urls/pages.py through the EditView class.

The url to submit for new translation however can't be found anywhere. Then it comes to my mind maybe it's not implemented in wagtail but in the wagtail-localize app. I looked into that app repo but can't find anything as well. Btw you can find urls for wagtail-localize app in wagtail_localize/wagtail_hooks.py. You can thank me later.

Fortunately I remembered that we can list all urls through django-extensions show_urls command.

poetry run python manage.py show_urls

That will list all urls defined in all installed app. You will see something like this:-

Turn out the urls is define in simple_translation app. The templates then looks like this:-

<fieldset>
    {% if self.heading %}
        <legend class="w-sr-only">{{ self.heading }}</legend>
    {% endif %}
    <div class="{{ self.classname }}">
    {% for translation in self.instance.get_translations %}
    <li><a href="{% url 'wagtailadmin_pages:edit' translation.id %}">{{ translation.title }}</a></li>
    {% empty %}
    <a href="{% url 'simple_translation:submit_page_translation' self.instance.id %}">Add New Translation</a>
    {% endfor %}
    </div>
</fieldset>

On how to add custom tab like above, I guess that would be for another post ;)

In our case, it was the Site settings not properly configured. It looked like this when the error appeared:-

You need to change that localhost to your actual domain such as yoursite.com and the Preview would then works.

class BlogIndexPage(RoutablePageMixin, Page):
    intro = models.CharField(max_length=250)
    content_panels = Page.content_panels + [
        FieldPanel('intro')
    ]

    @path("t/<str:tag>/", name="blog_tag")
    def blog_tag(self, request, tag):
        blogpages = BlogPage.objects.child_of(self).live()
        blogpages = blogpages.filter(tags__name=tag)
        return self.render(
            request,
            context_overrides={
                "blogpages": blogpages,
            },
            template="kai_site/blog_tag_index_page.html"
        )

And then in the blog page views templates we will list all tags the page is associated with:-

            ...
            {% if page.tags.all.count %}
            <div class="tags">
                <h3>Tags</h3>
                {% for tag in page.tags.all %}
                    <a href="{% routablepageurl blog_index url_name='blog_tag' tag=tag %}"><button type="button">{{ tag }}</button></a>
                {% endfor %}
            </div>
            {% endif %}
        </article>

And blog_index is defined as:-

class BlogPage(Page):
    date = models.DateField("Post date")
    intro = models.CharField(max_length=250)
    tags = ClusterTaggableManager(through=BlogPageTag, blank=True)
    ...
    def get_context(self, request):
        context = super().get_context(request)
        context["blog_index"] = self.get_parent().specific
        return context

However I got the following error:-

File "/workspace/.pyenv_mirror/poetry/virtualenvs/kai-site-gqAITAgE-py3.10/lib/python3.10/site-packages/django/template/library.py", line 237, in render
    output = self.func(*resolved_args, **resolved_kwargs)
  File "/workspace/.pyenv_mirror/poetry/virtualenvs/kai-site-gqAITAgE-py3.10/lib/python3.10/site-packages/wagtail/contrib/routable_page/templatetags/wagtailroutablepage_tags.py", line 25, in routablepageurl
    routed_url = page.reverse_subpage(url_name, args=args, kwargs=kwargs)
AttributeError: 'Page' object has no attribute 'reverse_subpage'

Checking what the page instance is shows that:-

>>> type(page)
<class 'wagtail.models.Page'>
>>> page.__class__
<class 'wagtail.models.Page'>

So I'm getting the base class, not my subclass BlogIndexPage. Took some time debugging with me getting off-tangent thinking that the blog page was created with the wrong parent. But peeking into the database shows everything is in order.

It only after looking into wagtail source code for the Page models I found this:-

    def get_specific(self, deferred=False, copy_attrs=None, copy_attrs_exclude=None):
        """
        Return this page in its most specific subclassed form.
        By default, a database query is made to fetch all field values for the
        specific object. If you only require access to custom methods or other
        non-field attributes on the specific object, you can use
        ``deferred=True`` to avoid this query. However, any attempts to access
        specific field values from the returned object will trigger additional
        database queries.

So that's it!

>>> type(page)
<class 'wagtail.models.Page'>
>>> page.__class__
<class 'wagtail.models.Page'>
>>> page.specific
<BlogIndexPage: Blog>

Page.get_parent() only return the instance with the base class as its type. We need to use the .specific property to get our sub-class as its type.

Have ever bumped into this when you try to ssh into a new host first time?

$ ssh kamal@51.251.110.111
The authenticity of host '[51.251.110.111] ([51.251.110.111]:22)' can't be established.
ECDSA key fingerprint is SHA256:xouheGRcbdddfdfdfaaaaaaaaaaaaacxxxxxxcadd*
*Are you sure you want to continue connecting (yes/no/[fingerprint])? yes*

In a lot of tutorials around the net, usually, you’ll just be told to type yes, and that’s where all the problems begin.

Unlike TLS/SSL which certificate that requires third party (CA) to prove its validity, SSH doesn’t use any third party to prove the authenticity of the key. Instead, they employ a mechanism called Trust on First Use or ToFU. And that’s what the prompt you see above.

For an ssh connection to establish, it must first make sure 2 things are correct:-

1. The key that the user uses to access the server is correct and allowed - this is basically by adding the public key into the user’s authorized_key file on the server.

2. The client can verify the identity of the server is correct. That’s what the prompt above is. It asks you to verify that you trust that indeed the server that you want to log in.

To verify that, you have to take the server key fingerprint (shown in the prompt) and then compared it with the key that you know belongs to the server. Only if you have verified that it matched, then you should type “yes”.

In the old days, that was not a problem. Usually, you will set up a server in the data center by being present physically in the data center. After setting up the server, you can run a command to see what is its key fingerprint, write it down and go back to the office. Then you will try to ssh first time to the server from your office’s computer. You got the same prompt as above, so you compare the fingerprint presented to the fingerprint that you wrote down on paper while in the data center. It matched and now you have a secure connection from your office to the server in the data center.

But today in all cloud environments, we are no longer present physically in the data center to copy the server’s key fingerprint. Some cloud providers will tell you the fingerprint when you provision the instance in their management console. This is the most ideal situation, assuming we already access their management console securely, with HTTPS for example.

Another mechanism is by providing virtual console access (similar to you plugging the keyboard and monitor into the server) so can you check the fingerprint yourself. But do not mistake this for the web terminal that some cloud provides, most of them just ssh clients, which means you still need to authenticate via ssh, which brings us back to the original problem.

AWS however doesn’t provide such an ideal solution. The only way we can discover the fingerprint is by inspecting the launch log, or the boot log, I’m not sure. But it is such a hassle that people simply type yes when prompted to verify the server fingerprint.

Now we have talked about the problem, I’ll just link to this article that proposes the use of a certificate that will be verified by CA instead to replace the old key authentication.

The general ideas in gettext translation are:-

1. Mark the string as translatable. There are a lot of ways to accomplish this. In program code, the best approach is to use a function. Why? Because first, it is syntactically correct. Function can always return value so it makes perfect sense as you can return the translated string from the function. But if you want to be fancy, you can just use any marker and then run the pre-processing script on that code to generate the translation you want.

2. Parse the code and extract the string marked as translatable. In django, you can see it implemented here.

Basically, you just run this command:-

xgettext -d django -L Python -kugettext_lazy test_trans.py

where test_trans.py is the file you want to translate. django.po will be generated in the current directory.

3. Translate the message file and compile it into a message object to optimize looking for translated string later on.

Now, to really understand how it work in practice, save the guess work and get your hand dirty in the console:-

09:41:40 {master} ~/git/kai-app$ python manage.py shell
>>> from django.utils.translation import gettext_lazy, gettext, activate
>>> gettext('About LaLoka Labs')
'About LaLoka Labs'
>>> activate('ja')
>>> gettext('About LaLoka Labs')
'About LaLoka Labs'

It's not translated, so what is the problem here? Aaah, we forgot to generate the message object.

python manage.py compilemessages

from django.utils.translation import gettext_lazy, gettext, activate >>> gettext('About LaLoka Labs')
'About LaLoka Labs'
>>> activate('ja')
>>> gettext('About LaLoka Labs')
'LaLoka Labsについて'

It's work!

Now, what is the difference between ugettext and ugettext_lazy ? The docs is here - https://docs.djangoproject.com/en/4.1/topics/i18n/translation/#lazy-translation.

It said:-

Use the lazy versions of translation functions in django.utils.translation (easily recognizable by the lazy suffix in their names) to translate strings lazily – when the value is accessed rather than when they’re called.

These functions store a lazy reference to the string – not the actual translation. The translation itself will be done when the string is used in a string context, such as in template rendering.

This is essential when calls to these functions are located in code paths that are executed at module load time.

Hmm, what does that mean?

If you try it in console, you'll see:-

>>> gettext_lazy('About LaLoka Labs')
<django.utils.functional.lazy.<locals>.__proxy__ object at 0x7f9c34880c50>

Notice the difference. Unlike the previous example, this one does not return the translated string.

So why do you need gettext_lazy over gettext?

If you go further down the docs, it shows an example of translating some model's attributes. You should already know that models definition is loaded once when you start django. If you use gettext, which means the string will get translated at that time. But what if you access models attribute at runtime (later on), in a different translation context, like changing from en to ja? You'll not get the translated ja string because it has already been translated before. So using gettext_lazy will ensure the string is translated when you see it, not when the program gets loaded into memory.

We have had some cases in the past where the translatable strings did not pick up by makemessages. Turns out someone aliased the gettext_lazy function to gettext_lz. If you look at the django source I linked above, makemessages run this command to extract the string:-

        elif self.domain == "django":
            args = [
                "xgettext",
                "-d",
                self.domain,
                "--language=Python",
                "--keyword=gettext_noop",
                "--keyword=gettext_lazy",
                "--keyword=ngettext_lazy:1,2",
                "--keyword=pgettext:1c,2",
                "--keyword=npgettext:1c,2,3",
                "--keyword=pgettext_lazy:1c,2",
                "--keyword=npgettext_lazy:1c,2,3",
                "--output=-",
            ]

_lz is not defined anywhere there. _ is recognized by default by python gettext module.

Some more examples to help you understand when to use gettext and gettext_lazy:-

class Person(models.Model):
    name = models.CharField(help_text=_lz('This is the help text'))

    def say_something(self):
        return _("Hello world")

name above will be executed when django started, either through runserver or apache restart, so it needs to be lazy. say_something() only executed by user, for example when they call person.say_something() in console, so it doesn't need to be lazy.

Now, can anyone tell what this means (this is from an old django project settings.py):-

gettext = lambda s: s
LANGUAGES = (
    ('en', ugettext('English')),
    ('ja', ugettext('日本語')),
)

Some would answer that to avoid circular import since the translation module depends on settings.py so we can't import gettext from the settings module itself.

However, in latest version of django, this workaround is not needed anymore and django.utils.translation can be imported from settings.py. Here's what the comment says about the workaround:-

# Here be dragons, so a short explanation of the logic won't hurt:
# We are trying to solve two problems: (1) access settings, in particular
# settings.USE_I18N, as late as possible, so that modules can be imported
# without having to first configure Django, and (2) if some other code creates
# a reference to one of these functions, don't break that reference when we
# replace the functions with their real counterparts (once we do access the
# settings).

You can thank me later.

And if you just look at /etc/systemd/system/sshd.service you'll feel kind of magic as well as there's nothing there related to EIC. That is just a symlink to /lib/systemd/system/ssh, which has all the meat there.

Other than sshd.service, syslog.service also symlink to /lib/systemd/system/rsyslog.service.

Ok, so /etc/systemd/system is supposed to be the place where you want to override the system's service unit definition, which is usually in /lib/systemd/system. /etc/systemd/system will have precedence over /lib/systemd/system.

And the correct way to override is not by editing the .service file but instead by creating a directory called servicename.service.d/ directory at the same level and include *.conf file in that directory. Within that .conf file you can override any individual service section attributes such as ExecStart=.

So for example in /lib/systemd/system/ssh.service.d/ec2-instance-connect.conf, ExecStart is overriden with this command instead:-

[Service]
ExecStart=
ExecStart=/usr/sbin/sshd -D -o "AuthorizedKeysCommand /usr/share/ec2-instance-connect/eic_run_authorized_keys %%u %%f" -o "AuthorizedKeysCommandUser ec2-instance-connect" $SSHD_OPTS

If you just look in /etc/ssh/sshd_config or /etc/systemd/system/sshd.service, you will feel like a fool because in sshd_config, AuthorizedKeysCommand is commented:-

# Expect .ssh/authorized_keys2 to be disregarded by default in future.#AuthorizedKeysFile     .ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none#AuthorizedKeysCommandUser nobody

and in /etc/systemd/system/sshd.service, ExecStart looks like this:-

ExecStart=/usr/sbin/sshd -D $SSHD_OPTS

Nothing in there suggested the use of EC2 Instance Connect. And systemd selling proposition is standardization 😊

Let me check how Amazon Linux does it. I think this is very Ubuntu/Debian specific.

ssh provides Include /etc/ssh/sshd_config.d/*.conf as override mechanism but I think a common dilemma faced by package maintainers or distro builders is whether to use the program's specific mechanism or use the system's mechanism.

And I think it's clear package maintainers prefer system mechanisms.

Amazon linux simply use sshd_config file.

AuthorizedKeysCommand /opt/aws/bin/eic_run_authorized_keys %u %f AuthorizedKeysCommandUser ec2-instance-connect

Much clearer. no hidden magic.

The command to start a new django project is django-admin startproject. But I stumbled there because I don't have the command django-admin. This is one thing that always confuses the newbie and make them stuck.

I paused a bit to reflect on how an inexperienced Django developer would think in this situation. There are many ways to go from here and it might feel daunting to some people. This is because we're facing different levels of concepts here but to inexperienced developers, it is just one bag of confusing things.

Django here exists as both a toolkit (in form of the django-admin command) and libraries (when you do import django ... in your app).

One quick way is to create a new virtual environment in the current directory I'm working on.

python -mvenv venv

Then I can install django inside that newly created venv:-

venv/bin/pip install django
venv/bin/django-admin startproject

I will also continue to use that venv to run my django project:-

cd myproject
../venv/bin/python manage.py runserver

So both the tooling (django-admin) and the libraries (django) in the same virtualenv. There's also another way to go, by treating django-admin as toolkit and install it "globally" within my $HOME environment. For this purpose I usually use pipx to install all the common tools I'm using for development.

pipx install django
django-admin startproject myproject
python -mvenv venv
venv/bin/pip install django
cd myproject
../venv/bin/python manage.py runserver

Above, I'm treating django-admin as toolkit and installing it to global $HOME so it is always available whenever I need it. But for myproject I'm installing a separate django in venv specific for that project.

So what else that I'm treating as toolkit?

• Black

• Flake8

• Pre-commit

• ...

For futher reading, head ups to this stackoverflow question.

• Query parameters get in different order .
• You can't pass parameters with same key to simulate arrays .
• API Gateway already decoding the query params so if your params has some esoteric encoding, tough luck for you .

The suggested workaround seems if you need to access raw data, don't pass it as query params, put it in the request body instead.

The problem is that, if your code under test also need to test the effect of using transaction then you can't use TestCase. Django provides TransactionTestCase for this purpose. In this class, each test will have database setup and teardown process - tables created and migrations runs at the beginning of the test and being dropped when the test finish. This is applied to each test case so you can imagine how slow it is.

For more information you can refer to the documentation page.

django-q

Django-q is a lightweight task queue that we usually use in our Django project. We prefer it compared to celery and it support a number of brokers such as redis, SQS, IronMQ and RDBMS (through Django ORM).

One feature django-q provides is the sync parameter when adding task to the queue. The parameter is meant to bypass the queue and immediately executing the task function, which is something you usually do in tests.

In the beginning the sync implementation is quite simple, basically just calling the task function, something like:-

if sync:
    run(task_func, *args, **kwargs, ...)

But in recent implementation this has been changed so that it better simulates the environment when the task was executed within the queue framework. So the implementation is something likes:-

if sync:
    _sync(task)

and the _sync() function defined as:-

def _sync(pack):
    """Simulate a package travelling through the cluster."""
    from django_q.cluster import worker, monitor

    task_queue = Queue()
    result_queue = Queue()
    task = SignedPackage.loads(pack)
    task_queue.put(task)
    task_queue.put("STOP")
    worker(task_queue, result_queue, Value("f", -1))
    result_queue.put("STOP")
    monitor(result_queue)
    task_queue.close()
    task_queue.join_thread()
    result_queue.close()
    result_queue.join_thread()
    return task["id"]

So in the original implementation the task function simply executed, no different than just calling the function, which mean it still executed within the same process as the tests. All is well here but in reality, this is not how the task function will be executed. So the _sync() function above would simulate how the task would enter and popped out of the queue to be executed.

The side effect is that the current transaction also will be closed and remember again how django TestCase works? Long story shot our tests will break. So when writing tests that need to utilized the sync parameter you have to use TransactionTestCase.

Web framework in Deno. What's special I think it geared towards frontend developer. One thing caught my eyes is the use of pages routing. So you drop a file in ./pages/about.tsx and it will available as /about/ url. Reminds me to PHP ;)

You can also drop few other file formats in /pages/ such as markdown and that will be rendered by Aleph.

Aleph looks interesting and made me thinking of building something similar in Python. Imagine being able to drop file written in Jinja2 in ./pages/ and automatically get it rendered.

marshmellow

Thinking about python web framework inspired by Aleph about lead me to search for serializer library, something similar to Django Rest Framework (DRF) serializer but more framework agnostic.

https://marshmallow.readthedocs.io/en/stable/why.html

twtxt

gemini

https://gemini.circumlunar.space/

myst

https://myst-parser.readthedocs.io/en/latest/