Our initial implementation of the VPC design failed in an interesting way: we ran out of NACLs pretty quick. This was primarily due to using a decimal based addressing scheme instead of sticking with the more standard binary scheme. For example a network might be in the 10.100/16 group instead of being at 10.16/12 group. This would open up more networks and allow for easier group between network nodes.

I’m wondering what the best break down of the network is. If we arbitrarily cap the number of regions we are willing to run in at 16 then that would consume 4 bits. Alright, it’s not arbitrary, AWS currently runs in 14 different regions. Hmm, that only really leaves a two regions, or one if we want an on-prem system too. So perhaps we give them 5 bits? That would be a total regional couont of 32. I’m really torn on having a global service identifier too. I think it might be time to read up on prior work in this arena.

All of the AWS aritcles themselves have some interesting attributes however all seem severly flawed. Something is probably missing from my understanding, however they keep dividing subnets, which keep getting smaller. So my searching reveals more and more articles merely suggesting you should follow the default reference design, not really talking in depth about the tradeoffs of different designs. I don’t really blame them, changing VPCs require reprovisioning all nodes attached to them; however I would have expected more detail from AWS itself.

Based on my previous musings I’m going to run that by my accomplices. In truth we’ll probably need more than 16 service classes but that is just conjecture. In reality you have at least hte following:

  • Command and Control: Bastion hosts, any automated CNC software designed to modify AWS, etc.
  • Datastores: Vault, Postgres, if you are adventerous you’ll probably have a few more like CouchDB, MongoDB, Cassandra, etc.
  • Application: Nodes designed to perform application work loads.
  • Network management layer: NAT, etc.

I’m torn on using Application Load Balancers in a seperate class. In truth I view them as apart of the application group itself and would put them under the application specific subgroup. In my proposal I think I’m going to reorder bits 0-4 of octect 2 and octect 3 entirely. There is some interesting quetsioning if we really need region, however I think it will cleanly map in and allow for easier system federation as we scale out. So let’s try this.

+--------------------------------+--------------------------------+--------------------------------+--------------------------------+
| Octet 1                        | Octet 2                        | Octet 3                        | Octet 4                        |
+--------------------------------+----------------+---------------+----------------+---------------+--------------------------------+
| Private Network (10/8)         | Region         | SRV-C  | SRV  | SRV   |  Avail-zones   | Host  | Host                           |
+--------------------------------+----------------+--------+------+----------------+---------------+--------------------------------+
| 8-bits                         | 4-bits         | 2-bits | 4-bi | ts    | 4-bits         | 10-bi | ts                             |
+--------------------------------+----------------+--------+------+----------------+---------------+--------------------------------+

The application load balancers are kind of interesting. The ALB/ELBs need to be on their own subnets matched to AZs however in theory we need farther fewer than the application services. Alrighty, good network connection. Time to see if I can get Terraform going again. I’ll come back to this later.