ALB and k8s: Routing between namespaces
• Mark Eschbach
ALBs are bound to a specific namespace. This is quiet unforutnate since each ALB costs at least $18 a month and you miss out on a lot of awesome features. Sure failure domains are isolated however if you publish a number of small serivces this is rather obnoxious.
A suggestion which looks promising is to have the ALB target the nginx ingress controller and dispatch from there. A little sad the ALB does not support this out of the box but will have to do for now. Perhaps in the future I will consider other ingress mechanisms to replace nginx if it exhibits broken connection issues on restart or is not monitorable still.
nginx Installation
Need to prime nginx’s stable repository first:
helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
After a quick glance throught the configuration some options definitely need to chnage. Here is my initial pass at the configuration for the chart. Configuration knobs
controller:
service:
type: ClusterIP
prometheus:
create: true
Most important is setting the controller to use the ClusterIP
as this is what the ALBs will target. If unset this
will create a new ELB via the LoadBalancer
service configuration which is definitely not what is desired.
Caveat: Default Host
Well, after seraching for a while turns out nginx does not support a critical use case: default hosts. This is really disappointing but I can work around the issue I suppose. All sites must be named and nginx provides it’s own default host. Easy enough to resolve by just directing specific hosts to the nginx controllers, although this makes me sad.