Django: Creating models dynamically

My original idea with siteplan was to allow a quick way to start and run a new django project (or app). The official way is to always start a minimal app that contains at least the following:

• models.py

• settings.py

• urls.py

Apparently, you need these 3 files to start Django project the usual way. Most python microframework like Flask or Bottle allows you to start your new app in just a single file. Imagine you can do something like this:-

from siteplan import App, run
from django.http import HttpResponse

def index(request):
    return HttpResponse("Hello world")

from django.urls import path
urlpatterns = [
    path("test/", index),
]

conf = {}
app = App(conf, urls=urlpatterns)

if __name__ == "__main__":
    run(app)

Trying to run Django from a single file was one of my favorite past time activities but most failed, due to the constraints mentioned above.

I have almost gotten it work with siteplan, except when it comes to defining your model. In the above example, if you try to define a models like this:-

from siteplan import App, run
from django.http import HttpResponse
from django.db import models

class TestModel(models.Model):
    name = models.CharField(max_length=255)

def index(request):
    return HttpResponse("Hello world")

It will fail with this error right away:-

 File "/home/kamal/.cache/pypoetry/virtualenvs/siteplan-G2sCj5Qw-py3.11/lib/python3.11/site-packages/django/conf/__init__.py", line 63, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

So you can only import django.db.models after your app has been fully initialized. My idea was to create a dummy siteplan.models and later on attach the models that the user has defined in their app.py to siteplan.models.

Even though not ideals, just for some experimentation I try to defer the models initialization by defining the models in a function which then gets passed to the App instance. This way I evaluate the models definition after I have configured the settings.

def create_models():
    from django.db import models

    class TestModel(models.Model):
        name = models.CharField(max_length=255)

        __module__ = "siteplan.models"

    return [TestModel]

conf = {}
app = App(conf, urls=urlpatterns, models=create_models)

Then in App class's __init__:-

        default_conf.update(conf)
        base_conf.update(default_conf)

        if not settings.configured:
            settings.configure(**base_conf)

        from siteplan import models as models_module
        for mod in self.models():
            setattr(models_module, mod.__name__, mod)

While this get rid of the settings error, we got another error:-

 File "/home/kamal/.cache/pypoetry/virtualenvs/siteplan-G2sCj5Qw-py3.11/lib/python3.11/site-packages/django/apps/registry.py", line 136, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

This means we can only initialize the models after AppConfig.ready() method being called. This is tricky since AppConfig is a different class and we can only tell Django the path to that class.

I took a step back and searched around to see if it is possible to create django models dynamically. Turns out in some ways it is possible to create models dynamically even since django 1.0!

Baserow has been utilizing the method to the fullest as described in this blog post.

Even though I'm already at the losing end, but just for the sake of experimentation, I try to move initializing the models to the __call__ method of Siteplan. This finally works but definitely can't be used as the method will be called on every request.

Being able to define the models is one thing, but how to create the database tables? In normal Django, we generate a migrations file and then call the migrate command. Thanks to the Baserow's article above, I learned that we don't need to generate migrations files in order to create the database tables:-

    def __call__(self, environ, start_response):
        ...
        from django.db import connection
        for mod in self.models():
            setattr(models_module, mod.__name__, mod)

            with connection.schema_editor() as schema_editor:
                schema_editor.create_model(mod)

        return application(environ, start_response)

Django Schema Editor allows us to create database tables without migrations files!

If I still want to proceed, I can let user define the models either in dummy class or in yaml, and then use the same approach used by baserow to generate the models dynamically.

Some other unrelated notes of what I bumped into in the course of this exercise:-

• Django fastapi bridge

• Django-ninja - fastapi like functionality on top of Django Rest Framework

• Django ORM now has async queryset method! While not fully red yet, works are on the way to make everything async down to the database driver.