I use Terraform to create all infrastructure in Azure at work. It's a fantastic tool, and one which has saved a lot of time. However, there is always a lot to learn, and with that comes gotchas that you want to avoid in future. So to save future-me some Googling in a few months, here are the most time consuming gotchas I encountered on a recent project while trying to create hybrid connections for app services in Azure using Terraform.

Lesson 1: Always use a hostname - not an IP address

Ok, so this one is not strictly a Terraform issue, but one to watch for nonetheless. After creating all three elements in Azure and connecting the hybrid connection in Azure to an internal resource using Hybrid Connection Manager, eveything appeared to be connected, but I was getting an error message

Err Connection refused

My error was attempting to use the IP address of our intenral resource, instead of the hostname. Hybrid connections will try and resolve the IP address using DNS, and consequently look to the internet rather than an internal network to resolve this.

Lesson 2: Always use lowercase URL

Again, not strictly a hybrid connection issue, this was caused by my use of a hostname http://xxx-123-ukQConnect:1234. Notice the QC part there? Well I didn't for about 2 days! This would likely not have been an issue for most setups, but I am using Linkux containers in Azure to talk to the hybrid connection. Linux will be quite upset if you attempt to use casing in your URLs.

Lesson 3: There are some things Terraform just can't do.... yet!

TL:DR - Terraform does not create the resources you need to create a working hybrid connection for your web app in Azure.

The first thing to note is when creating a hybrid connection for app service, you need to create three resources

  1. Azure relay
  2. Azure Hybrid Connection
  3. Azure App Service Hybrid Connection

The default script provided in the docs creates these three elements, but once they have been provisioned in your resource group and configured, they will not connect to your interal resource.

In my investigation, deleting the Azure app service hybrid connection and then recreating it with the same settings using the Azure Portal fixed my connection issues.

When terraform creates the hybrid connection it uses the default SAS (Shared Access Signature) keys created on the relay called RootManageSharedAccessKey However, when the Azure Portal creates an app service hybrid connection it creates two new SAS keys at the hybrid connection level and uses those.

Two SAS keys are created

  • defaultListener
  • defaultSender

In the app service hybrid connection, the defaultListener key is set in the Gateway Connection String, so the value looks something like this

`Endpoint=sb://yourrelayname.servicebus.windows.net/;SharedAccessKeyName=defaultListener;SharedAccessKey=Generated SAS key value=;EntityPath=your hybrid connection name

This is important as when the hybrid connection tries to talk to resources sat behind it, it uses TLS 1.2 for security and it's shared access signature (SAS) keys for authentication and authorization.

To this end, I'm not sure if anyone has ever managed to get a hybrid connection working in Azure having scripted it using Terraform.

This is not meant to be a dig at Terraform or the team behind it. I think its an amazing tool, which has allowed our team to get further than we though possible with IAC, I'm sure the feature will eventually be added, indeed a better developer would file a bug report and submit a PR against it.... 🤔 Perhaps, an update of the documentation would also work?

The approach we have ended up with is adding the relay and it's hybrid connection using Terraform, and then updating the project's readme to manually create the app service hybrid connection using the Azure portal for now.