Today I was fixing one of our Pyramid app's test. Due to some changes in business logic, I need to change some settings for this particular test to pass. Should be a straightforward fix.
We're using pytest to run the tests and our
conftest.py resembles what is mentioned in this Pyramid testing documentation. For example, we have this fixture defined in the file:-
def app_env(ini_path: str) -> AppEnvType: """Initialize WSGI application from INI file given on the command line.""" env = bootstrap(ini_path) # build schema alembic_cfg = Config("etc/alembic.ini") command.upgrade(alembic_cfg, "head") return env
This mean, in our test we just need to request this fixture and manipulate it before executing the function under tests.
def test_something(app_env, paramiko: mock.MagicMock, celery_db: Session, demo_celery: None, ....): app_env["registry"].settings["special_config"] = False with pytest.raises( ValueError, match=r"not enough values to unpack \(expected 2, got 1\)" ): func_to_test()
special_config flag never set to False. Going through the documentation again, I'm pretty sure that's all I need to manipulate the settings before running my function under tests. But it looks like the settings are coming from somewhere else and not from my fixture.
After hours of pulling my hair, I did what those in desperation usually did. I start randomly removing stuff to see what could break. I removed
demo_celery from the fixture request. And something interesting happened. The test failed because of missing stuff.
src/kai/article/tests/test_article_tasks.py:586: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ .venv/lib/python3.9/site-packages/celery/local.py:191: in __call__ return self._get_current_object()(*a, **kw) src/kai/tasks/__init__.py:112: in __call__ self.settings = app.conf["settings"] .venv/lib/python3.9/site-packages/celery/utils/collections.py:449: in __getitem__ return self.__missing__(key)
app.conf things caught my eye. I started tracing where that
app comes from. As we can see in the snippet above, it is being used in the modules
src/kai/tasks/__init__.py I can see in the modules:-
from kai.celeryapp import app
app = Celery() app.user_options["preload"].add(add_preload_arguments) app.steps["worker"].add(StructLogInitStep) # Plaster parts def setup(ini_path: str) -> None: """Given ini file, load settings and setup Celery.""" loader = plaster.get_loader(ini_path) settings = loader.get_settings("celery") # TODO: this line can fail, but celery will swallow exception resolver = DottedNameResolver() celery = resolver.resolve(settings.get("use")) celery(app, loader)
So that's it! The function under tests is a celery task and it turns out that the celery task, is using a separate
app instance, different from the main app. That's why my changes to the settings in the registry have no effect at all. It uses a different registry.
Knowing this, the fix to my test is simply:-
def test_something(paramiko: mock.MagicMock, celery_db: Session, demo_celery: None, ....): from kai.celeryapp import app app.conf["settings"]["special_config"] = False with pytest.raises( ValueError, match=r"not enough values to unpack \(expected 2, got 1\)" ): func_to_test()