Enterprises continue to feed their clouds with increasingly sensitive information, yet according to McAfee’s latest report the security issues are building alongside this trend. The study, titled ‘Enterprise Supernova: The

Continue ReadingMore sensitive data moves to the enterprise cloud – but the security risk widens with it

News

Nicole Kobie

Appointment service and NHS 111 directory are the first major services to make the move

…

Read MoreNHS shifts two national services to the cloud

2019 saw a bumper year of data centre merger and acquisition deals with the total number passing 100 for the first time, according to Synergy Research. The record number has

Continue ReadingData centre M&A broke the 100 deal barrier in 2019 – driven by private equity

In this post, I’d like to show readers how to use Pulumi to create a VPC endpoint on AWS. Until recently, I’d heard of VPC endpoints but hadn’t really taken the time to fully understand what they were or how they might be used. That changed when I was presented with a requirement for the AWS EC2 APIs to be available within a VPC that did not have Internet access. As it turns out—and as many readers are probably already aware—this is one of the key use cases for a VPC endpoint (see the VPC endpoint docs). The sample code I’ll share below shows how to programmatically create a VPC endpoint for use in infrastructure-as-code use cases.

For those that aren’t familiar, Pulumi allows users to use one of a number of different general-purpose programming languages and apply them to infrastructure-as-code scenarios. In this example, I’ll be using TypeScript, but Pulumi also supports JavaScript and Python (and Go is in the works). (Side note: I intend to start working with the Go support in Pulumi when it becomes generally available as a means of helping accelerate my own Go learning.)

Here’s a snippet of TypeScript code that will create a VPC endpoint (I’ve randomized the resource IDs of the AWS resources below):

const ec2ApiEndpoint = new aws.ec2.VpcEndpoint("test-vpc-endpoint", { privateDnsEnabled: true, serviceName: "com.amazonaws.us-west-2.ec2", securityGroupIds: [ "sg-1db01e1f9be081d76" ], subnetIds: [ "subnet-0db757befe2c48624" ], vpcEndpointType: "Interface", vpcId: "vpc-0e2c348614e845763", tags: { Name: "test-vpc-endpoint" }
});

Let’s walk through this a bit (the full API reference is here):

• The privateDnsEnabled: true line enables us to use the “standard” EC2 API URL endpoint within the VPC and have it resolve to a private IP address out of the VPC/subnets. This relies on Route 53, and is necessary for use cases where you (the user) don’t have any control over how the API endpoint is being called from within the VPC (it may be a bit of commercial software you can’t modify). In this case, any software trying to reach “ec2.us-west-2.amazonaws.com” will resolve to a private IP address instead of the public IP address.
• The serviceName specifies which service is available via this endpoint (naturally, this is for EC2).
• The vpcEndpointType specifies “Interface”, which means that an ENI is created to allow traffic to reach the service represented by this endpoint. The other option is “Gateway”, which would require you to modify route tables, and is only supported for some services (like S3). This article provides an example of a gateway endpoint.

The other settings are all pretty self-explanatory. This being TypeScript, you could—of course—use references to variables or outputs from other portions of your Pulumi code instead of hard-coding values as I have in this example (and that would be highly recommended). This could be easily incorporated into a large portion of code that created a new VPC, new subnets, etc.

And that’s it! I know this is a fairly simple example, but I hadn’t found very many articles showing how this is done, and thought that sharing this may prove helpful to others. Feel free to contact me on Twitter if you have corrections (in the event I made a mistake), suggestions for improvement, or other feedback.

…

Read MoreCreating an AWS VPC Endpoint with Pulumi

I recently had a need to manually load some container images into a Linux system running containerd (instead of Docker) as the container runtime. I say “manually load some images” because this system was isolated from the Internet, and so simply running a container and having containerd automatically pull the image from an image registry wasn’t going to work. The process for working around the lack of Internet access isn’t difficult, but didn’t seem to be documented anywhere that I could readily find using a general web search. I thought publishing it here may help individuals seeking this information in the future.

For an administrator/operations-minded user, the primary means of interacting with containerd is via the ctr command-line tool. This tool uses a command syntax very similar to Docker, so users familiar with Docker should be able to be productive with ctr pretty easily.

In my specific example, I had a bastion host with Internet access, and a couple of hosts behind the bastion that did not have Internet access. It was the hosts behind the bastion that needed the container images preloaded. So, I used the ctr tool to fetch and prepare the images on the bastion, then transferred the images to the isolated systems and loaded them. Here’s the process I followed:

1. On the bastion host, first I downloaded (pulled) the image from a public registry using ctr image pull (the example I’ll use here is for the Calico node container image, used by the Calico CNI in Kubernetes clusters):

   ctr image pull docker.io/calico/node:v3.11.2
   

   (Note that sudo may be needed for all these ctr commands; that will depend on your system configuration.)

   I repeated this process for all the other images the isolated systems also needed.

2. Still on the bastion host, I exported the pulled images to standalone archives:

   ctr image export calico-node-v3.11.2.tar \
   docker.io/calico/node:v3.11.2
   

   The general format for this command looks like this:

   ctr image export <output-filename> <image-name>
   

   If you don’t know what the image name (according to containerd) is, use ctr image ls.

3. After transferring the standalone archives to the other systems (using whatever means you prefer; I used scp), then load (or import) the images into containerd with this command:

   ctr image import <filename-from-previous-step>
   

   Repeat as needed for additional images. It appears, by the way, that using wildcards in the ctr image import command won’t work; I had to manually specify each individual file for import.

   If you need these images to be available to Kubernetes, you must be sure to add the -n=k8s.io flag to the ctr image import command, like this:

   ctr -n=k8s.io images import <filename-from-previous-step>
   

4. Verify that the image(s) are present and recognized by containerd using ctr image ls.

   If you specified the k8s.io namespace when importing the images in the previous step—so as to make the images available to Kubernetes—then you can verify that CRI (Container Runtime Interface, the means by which Kubernetes talks to containerd) sees these images by running crictl images (again, sudo may be required, based on your configuration).

That should do it for you!

How I Tested

To test this procedure, I used multiple Ubuntu 18.04 instances on AWS. One instance had Internet access (the bastion host); the others did not. The instances had been built from AMIs prepared using Packer and Ansible with playbooks from the Kubernetes “image-builder” repository (specifically, the playbooks used to build the Kubernetes Cluster API images), so containerd and all the other CLI tools mentioned in this article were already installed and available for use.

Additional Resources

I did find a few sites with some containerd information while searching, and wanted to include them here for credit and additional context:

Getting started with Containerd

crictl User Guide

I hope the information in this post is helpful. If you have questions, comments, suggestions, or corrections, you are welcome to contact me on Twitter, or drop me an e-mail (my e-mail address isn’t too hard to find).

…

Read MoreManually Loading Container Images with containerD