May the 4th be with You
• Mark Eschbach
I hope your May the 4th is going well.
Spent most of the day exploring monitoring tools. New Relic is expensive but more on that later.
Anyways, I’m trying to figure out how to rotate security keys within Django for talking with ElasticSearch. As my adventures earlier this week have shown this wasn’t a picnic. I had a fun time battling character encoding issues and need to use a third party library to sign the requests. So the security token is extracted from AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
with http://169.254.170.2
prepended.
Normally boto
would probably take care of this for you, however our primary application is still using boto2
. I should probably look into upgrading the tools at some point. After reviewing the AWS ECS IAM Role documentation there is an Expiration
field there. I’m hoping this field contains the date the Token
is no longer valid. Totally not expirenced enough to know if the access key and secret key also expire after the same period of time.
I’m guessig I can’t just update the HAYSTACK_CONNECTIONS
dictionary as in theory HayStack doesn’t build a new object once initialized. I could be wrong though. Let’s go to the source!
Based on the stack trace the exception is being risen at haystack/indexes.py line 278. The backend
object is retrieved from _get_backend @ indexes.py line 241. In my case I believe the value of using
is None
so we fall into the conneciton_router
. I’m guessing this object is responsible for managing read/write connections. The connection_router
is imported from the module root haystack/init.py line 52. For those unknowning like myself I’ll spare you the rabbit hole and leave it as the connection_router
is an object responsible for deciding the name of the connection to be used. Probably fairly useful for splitting read-write loads.
Onto tracking down how the dictionary haystack.connections
is populated. The field is assigned on haystack/init.py line 44 from loading.ConnectionHandler
. Interestingly the ConnectionHandler
has na interesting method #reload. Within the method there are some strange references to thread_local
so I’m hoping that will not be an issue. Famous last words there.
Timing is the next problem. The field in theory gives us a specifc time to wake up and update the token. Ideally I don’t waste time with each request checking if the time has elapsed. Might be less effort for right now. I might be trying to optimize early :-). Much time has passed since I last worked with dates in Python. StackOverflow seems to have the answer on parsing the AWS date. Comparison looks fairly staright forward with the binary infix operator <
.
Let’s give it a whirl in the console! Primarly because I don’t want to keep going back and forth with the deployments. Oh right, datetime.datetime.now() < date
is complaining about time zones. I’ve waged this debate with Django before. Ah! StackOverflow to the rescue: datetime.utcnow().replace(tzinfo=pytz.utc)
. Properly worked! Sweet!
Unforunately I’m still a bit off kilter with the way in which Python handles Object-Oriented programming. I think it works great when pushing people towards composition, however there are times where without delegation I must overload.