Tales from Django Middleware
• Mark Eschbach
Over the past two days I’ve been working on modifying the django-request-logger to honor the annotations of not logging the request entity nor the response entity. The two particular business cases which support this charge are security and throughput issues. If you are accepting a request or response entity which validate a password then the user password really shouldn’t be logged. Throughput is an issue with extremely large request or response entities. For some requests I’ve seen the response time take up to 92% in logging.
Yesterday I learned the particular test fixture used in the request logger does not report exceptions risen in the
settings module. This was quiet unfortunate as I spent most of my non-meeting times yesterday chugging away at this.
It was only after JG mentioned it was really strange the module is acting like something was not imported I realized
all errors were being consumed.
Prior to the PR which will push
0.6.1 out we did not directly test the Django Rest Framework. Until I got
integrated into our main application there was not a reason to. Luckily a negative test pointed out the
Django Request Logger needed to deal with nonexistent route gracefully. Getting DRF integrated was interesting and
this was my first experience integrating it into an existing project.
The following was added to the libraries
djangorestframework==3.8.2. In the future
I would like to expand the test matrix to test against multiple versions of DRF & Django but we just have pinned
versions right now. Next up was creating a new DRF viewset and wiring it in.
from rest_framework import viewsets, routers class UnannotatedDRF(viewsets.ReadOnlyModelViewSet): @no_logging("DRF explicit annotation") def list(self, request): return HttpResponse(status=200, body="DRF Unannotated") router = routers.SimpleRouter(trailing_slash=False) router.register(r"widgets", UnannotatedDRF, base_name="widget")
Last up was probably the most annoying. This was my first time really modifying the URLs for a Django application.
From best I can tell a URL in Django is a pair of
(regex, func) group to consume a give URI from within the service.
Nice and easy to hook up new route handlers. However this became a serious stumbling block in regards on how to
router object. Initially I desired to add the the whole router under
/drf to make the tests clear on
what was being tested. However after working on the tests for a bit I eventually resigned to just having DRF work off
the root URI. Perhaps with more experience in the future I will be able to clean that up and collapse it. For now
this is what I had settled on:
urlpatterns = [ url(r'^somewhere$', general_resource), url(r'^test_class$', TestView.as_view()), url(r'^test_func$', view_func), url(r'^test_msg$', view_msg), url(r'^dont_log_empty_response_body$', dont_log_empty_response_body), ] + router.urls
Once I got a few tests which exhibited the issue && I fixed then BAM pull request. Then Travis indicated a failure while I was cleaning up the PR text. Turns out Python 3.7-dev is upset.
pyenv life has been a bit easier. Just
pyenv install 3.7-dev and wait. For a while. Not bad in terms
of getting a software environment done. Until it spins for a while then fails. Fails because of
zipimport.ZipImportError: can't decompress data; zlib not available.
Looking back at Getting multiple version of Python installed on OSX
CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" prefixed on OSX. Nope, no luck there. On to using
a Linux machine.
Alright, so I setup a Linux machine with a Fuse SSH mount. Time to start debugging.
After some time, I’m not sure why the routes are being sucked in. We currently do not use Python
3.7-dev and I am
not so certain we should be supporting pre-release builds. Unforunately I’m out of time and will have to return to the
future comparability at another time.