Azure Private Link has been available in Azure little bit over year now. It was published 16.9.2019 to Public Preview. 14.2.2020 it got it General Available (GA) status and after that there have been added many PaaS-services for it. In short, Azure Private Link connects your PaaS service such as SQL Server, Storage account or App Service to your subnet and gets a private IP for it. It uses resource called private endpoint to accomplish it. Then you need some DNS-configurations and everything works like a magic. I’ve seen that this isn’t so clear for everyone how private endpoints works so I’d like to clarify it little bit more.
When implementing Azure Private Link, remember always use the public facing name for applications to reach the resource. DNS handles rest.
Azure Private Link as a concept
When you create a private endpoint (the resource that is used in the Private Link -concept), you will change the public name resolution for the resource towards you are creating the private endpoint. Let’s look little bit that. I have a storage account called bloggerzstorage and it has a public IP address provided by Azure.
Enabling Azure Private Link
When I create a private endpoint, Azure is changing the public name resolution by adding there another CNAME record pointing towards the dedicated FQDN of private endpoint. The record name and zone depends of resource type (or sub-type) and you can find the reference of DNS zone naming from the Microsoft’s documentation. For the storage account’s blob sub-resource as in this example the FQDN will be <storage-account-name>.privatelink.blob.core.windows.net.
DNS-record for Azure Private Link
Currently the IP that your DNS is resolving is the public IP, because there isn’t anything available in private link CNAME.
Now you should add to your internal name resolution an A-record that points the private endpoint address to the private IP that is associated for the resource inside your Azure virtual network:
When you try to resolve it now from the client that is using the internal DNS-server, it will response the private IP address.
How this is possible then? It’s not magic. Azure’s public DNS servers (where e.g. blob.core.windows.net is hosted) is configured to prioritize privatelink.blob.core.windows.net if available. According to RFC’s 1912 paragraph 2.4 multiple CNAME’s isn’t encouraged. But it works. For prioritizing records, you can consult RFC 1035 paragraph 7.2, but as you know, the DNS server hoster can do what ever she wants.
And as a conclusion Private link address will be prioritized in name resolution.
Private DNS zone resource
In next chapter I’m telling my best practices to implement DNS resolution in different scenarios. Before that I’m introducing a resource from Azure called Private DNS zone. It’s a private DNS zone that can be only accessed from Azure backend. Not from internet, only behind the Azure’s public IP addresses.
You can create required private link DNS zone e.g. privatelink.blob.core.windows.net to there and use it from Azure. When creating a private endpoint, you can tell also to automatically add the A-record to the private DNS zone. Azure engineer does not need to have access to your DNS-infrastructure e.g. in Windows Server Active Directory. Also when you are deleting the private endpoint, the A-record is removed automatically when using Azure Private DNS Zones. Sounds nice? YES!
Name resolution options
There are several ways to achieve the name resolution, but I will now tell how I would implement it. Normally you have three options depending of your infrastructure:
- Azure VNET without custom DNS Servers (without Active Directory)
- On-premises and/or cloud workloads with own DNS servers (e.g. with Active Directory) and multiple internet routes
- On-premises and/or cloud workloads with own DNS servers (e.g. with Active Directory) and internet gateway forced in on-premises
Azure VNET without custom DNS Servers (without Active Directory)
This is the most straight forward scenario. Just create a Private DNS Zone to Azure named by domain name that is going to be the private endpoint domain name for your resource for example privatelink.blob.storage.windows.net.
When creating the private endpoint, just select the dedicated zone for automatic DNS registration and all configured.
If the request to the public DNS-name (bloggerzstorage.blob.core.windows.net) is coming from VNet the Private DNS Zone answers the private endpoint’s internal IP, but if the request is coming from internet, the Azure’s own DNS answers the public IP.
- VM asks the public name bloggerzstorage.blob.core.windows.net
- Private DNS Zone Answers to request through Azure DNS
- VM connects to Private Endpoint
- DNS requests not allowed from internet
- You access storage account with a public IP provided by Azure
Remember always use the public facing name for applications to reach the resource. DNS handles rest.
On-premises and/or cloud workloads with own DNS servers (e.g. with Active Directory) and multiple internet routes
So if you have a route to the internet available on Azure you can also use Private DNS zones. You can route some of your internal networks to the internet from some other point (for example from on-premises datacenter), it really doesn’t matter. You have same benefits when having a full name resolution on Azure services. When you create a private endpoint, you can have an automatic DNS record creation to your Private DNS zone and it works.
In this scenario you have your own DNS Servers such as a Windows Server Active Directory integrated DNS that you want to use for name resolving. To get the name resolving working, install DNS servers also to the cloud. Without some server that acts as a DNS relay in the VNet, your DNS traffic is not routed to the internet (and Private DNS Zone) from Azure. After you have the relay server, just create a conditional forwarder to on-premises DNS for your public DNS-zones (e.g. blob.core.windows.net) and point those towards your Azure VM which is a DNS relay in cloud. Remember to put forwarders also for Azure DNS server to point Azure’s public DNS services in IP 168.63.129.16. You can use for example couple of domain controllers in cloud to act as a DNS relay.
DNS request that is asked from Azure DNS Services must be routed through some Azure Virtual Network, otherwise it is not allowed to Azure Private DNS zones. For lower latencies I suggest that you use your ISP provided DNS as a forwarder in on-premises DNS servers and Azure public DNS services in Azure based DNS servers. Ensure that always, when some VM tries to reach Azures DNS service 168.63.129.16 (also from on-premises), the traffic must be routed through the Azure Virtual Network through the DNS relay. You can do this with a route table in Azure and on-premises firewall or what ever you are using as a routing device in on-premises.
- VM asks the public name bloggerzstorage.blob.core.windows.net from local DNS server
- DNS Server forwards the request with conditional forwarder to Azure DNS that asks it from Azure’s public DNS servers and the DNS servers responses the private IP to the client
- DNS server responds the private IP to client
- VM connects to Private Endpoint
- DNS requests not allowed from internet
- You access storage account with a public IP provided by Azure
- Use your local ISP DNS as a forwarder for other DNS queries
Remember always use the public facing name for applications to reach the resource. DNS handles rest.
On-premises and/or cloud workloads with own DNS servers (e.g. with Active Directory) and internet gateway forced in on-premises
The last scenario is if your default route points to the on-premises and all traffic is routed to the internet from there (the legacy way). In this scenario you can’t use Azure Private DNS Zones since you can’t access them from the internet. Instead you have to create own DNS-zones to your DNS Server Infrastructure. In this scenario select no when private endpoint creation provides the ability to add a A-record to the Private DNS zones as there isn’t one.
Every time you are creating a private endpoint you have to manually add an A-record to your DNS and remove the record when removing the private endpoint. Of course you could create a script that for example uses Azure Event Grid to catch Private Endpoint creation or removal and modify DNS entries to the on-premises DNS servers, but is it clever? Avoid this, but it’s doable.
- VM asks the public name bloggerzstorage.blob.core.windows.net from local DNS server
- DNS Server answers locally from own privatelink.blob.core.windows.net zone the private IP
- VM connects to Private Endpoint
- You access storage account with a public IP provided by Azure
- Use local ISP DNS for public name forwarding
Remember always use the public facing name for applications to reach the resource. DNS handles rest.
Private DNS Zone in different subscription?
You can have a Private DNS Zone also centrally in different subscription and just delegate proper permission to it. Private Link creates itself DNS records to Private DNS Zone if the Private DNS integration is enabled and removes it, when the private endpoint connection is deleted.
Integration creation with code
User who is creating the integration with code needs next permission to Private DNS Zones.
- Microsoft.Network/privateDnsZones/join/action
Permission is enough if private endpoints are created through the code.
Integration creation through the portal
User who is creating the integration through Azure portal, she needs also read permissions to zones (to be able to select zones in wizard, you can’t specify the resourceId of DNS zone in portal wizard). Give next permissions for user if they are creating the integration through the portal:
- Microsoft.Network/privateDnsZones/join/action
- Microsoft.Network/privateDnsZones/read.
When using auto registration of DNS name to Private DNS Zone, just select the correct subscription before selecting the DNS zone.
Conclusion
Three different scenarios, three different ways to implement a DNS name resolution for a private link service. Hopefully this clarifies at least some ones mind how does private link and DNS work together and how mission critical it is.
21 Comments
Oliver · 04.01.2021 at 14.31
Hi MArkus,
great summary….
We are currently using the last scenario with different Private Zones configured on an onpremise DNS.
That all worked pretty good until we now want to connect to a DB from an external company which also uses privatelink-FQDNs.
So the first Cname comes back from the uplink DNS, but then the onpremise DNS checks his local zone and if there is no dedicated entry and the resolving process stops…
Or is there any fancy option in DNS I have missed so far, that checks the uplink DNS even if there is a .privatelink.[Azure-Service] – Zone configured..
Thanks in advance for your help,
Oliver
Markus Lintuala · 08.01.2021 at 13.40
DNS takes always the closest option available. If the zone exists in your own DNS server, it does not look up it anywhere else. So if you don’t have a record in zone it will not be resolved for client.
You can create a private link also between tenants if you want to publish your DB from your home tenant to the external tenant without any VPN tunnels etc. Remember that you can’t monitor the access to DB on the network level in your tenant. When creating a private endpoint for external tenant, you have to manually approve it before it works in the external company. After it has been approved they can add a new A-record to their DNS and magic happens. 😉
Naveej · 23.07.2021 at 21.14
Hello Markus,
I have a situation where I need to resolve the private IP address from on-premises. I setup private endpoints, private DNS zone. According to your article, I have a private DNS Zone for Azure Synapse (privatelink.sql.azuresynapse.net) and have a private endpoint for my Synapse instance created with an A record in the private DNS Zone. I understand we need to setup a forwarder in Azure (which forwards to the 168.63.129.16) which will be receive forwards from my on-prem DNS for sql.azuresynapse.net.
Now, what happens, if there is another public Synapse instance (something.sql.azuresynapse.net) which I’m not managing is being resolved from on-premises. Would the resolution fail, because I don’t have an entry for something.sql.azuresynapse.net in my private DNS zone, or would it still get resolved to the public IP of that record?
Markus Lintuala · 27.07.2021 at 09.06
Hi Naveej,
If another Synapse instance does not have private endpoint, it will be resolved to public ip. When you are creating private endpoint for some resource, it adds the privatelink.xx.yy.zz dns record for it and internal IP can be resolved through that zone.
Daniel · 03.03.2021 at 14.19
Hi Markus!
Nice post, one of the few posts on the internet which actually explains the way it works.
I tried setting up a POC for Azure Files with ADDS authentication aswell, which seemed to work fine just on the begin.
2x DC on-prem, 1x DC in Azure.
On the DC’s on-prem I added conditional forwarders to my DC in Azure, and from the DC in Azure a conditional forwarder to the private DNS zone from Microsoft. All traffic should be moving according to that, yet there is only 1 machine on-premise where I can actually mount the File share with AD credentials, all the other machines I tried it on don’t work for me.
Do you maybe have an idea what could cause this? Thank you in advance!
Markus Lintuala · 03.03.2021 at 18.57
Hi Daniel,
Hmm… With these information unfortunately not yet idea. I suggest to check with a nslookup that are clients resolving the private IP with a public name (something.file.core.windows.net). Remember to clear the DNS caches with ipconfig /flushdns before trying the DNS resolution. If the DNS resolution works, ensure that network is clear to the private IP itself and firewall ports are open. Ping me via e-mail if you need more help.
dheep · 25.07.2021 at 05.55
Amazing explanation. For the past year we trying to work out how to integrate onprem dns to cloud services. This post gives a clear picture of available options. Thank you soo much for making time to write this Markus.
SS · 30.07.2021 at 12.16
Thank you so much for your contribution!!!
HT · 21.09.2021 at 00.30
Hi, I have set up a private endpoint with a private IP. When I am on a VM within the same VNET and I perform a nslookup on the resource, it still responds with a public IP. I do not have full admin privilege to troubleshoot but based on the nslookup details, it is responding from a DC controller in Azure. Can I guess that a conditional forwarder needs to be set up in that DC controller DNS as you have suggested in option 2 in your blog? Thank you.
Markus Lintuala · 03.10.2021 at 11.40
Hi, In case you have already Private DNS zones in Azure, you only need to forward DNS-traffic outbound from cloud DNS-servers (DC’s DNS forwarders in your case) to the Microsoft’s public DNS endpoint. Do not add conditional forwarders to cloud DC’s.
Victor · 11.10.2021 at 09.38
I have just added “table.core.windows.net” as a conditional forwarder, attempting to get privatelinks working for tables (we already use “blob.core.*” for other things). When I do a lookup on the configured PE I’m getting the public IP, as well as the public and private alias for the table resource, but not the private IP.
When I do a lookup on the existing “blob.core.*” resources, I get back just the private IP of the resource and the privatelink dns for the resource only. So it’s filtering properly.
I did a tail on the DNS we have running in azure and tested both options (for blob.core resources and table.ocre) and the intial DNS lookup/response is the same for both. Cannot fault anything so the query IS going over our expressroute.
Thoughts as to what I need to look at next from there?
Markus Lintuala · 27.10.2021 at 10.14
Hi Victor,
First try to resolve the name from Azure VM. If it is not responding private IP, look for your Private DNS zones and is it linked to your VNET, where your DNS Proxy resides? If Azure VM resolves the DNS name, then check your conditional access configurations.
Andy · 24.08.2022 at 12.40
Hi Markus,
Great articles thank you.
Any ideas of what is this blob.xxxx.store.core.windows.net URI in your nslookup? I find no information on the web regarding that.
Andy
Markus Lintuala · 30.08.2022 at 09.22
It’s probably Microsoft’s URL for the correct blob storage itself, nothing to care about because you have your own endpoints 🙂
Alex G · 06.10.2022 at 21.23
Precise and to the point. Thanks for putting this together, Markus!
Salah · 09.12.2022 at 05.24
Nice Article, my scenario is same as your last scenario where i have on premises DNS Servers only and all routed to on premises. it is not clear for me that should i create a DNS Zone with privatelink.blob.core.windows.net name or i should create DNS zone with the blob.core.windows.net, if DNS Zone will be created with privatelink.blob.core.windows.net then how client will be able to resolve abcstg.blob.core.windows.net to the private link ip address since there is no DNS Zone blob.core.windows.net in the internal DNS.
Thanks for your help
Markus Lintuala · 09.12.2022 at 18.26
Hi Salah, When you enable the private endpoint to storage account, it will get another public DNS record CNAME st112345.privatelink.blob.core.windows.net. When your client is requesting the public name st112345.blob.core.windows.net, the first response is from the longest CNAME so most specified CNAME is st112345.privatelink.blob.core.windows.net. Then your client is trying to solve its IP. If client is in internal network, it will find the private IP address from your own DNS servers that has the privatelink.blob.core.windows.net-zone available. If your client is not in internal network, it will not find that DNS-zone and continues to the next CNAME available in public DNS server (st112345.privatelink.blob.core.windows.net) that has of course the public IP address from Azure itself.
Hopefully this clarifies 🙂
crypto · 30.01.2023 at 15.39
Hi Markus,
One question. You refer to the Microsoft.Network/privateDnsZones/join/action and Microsoft.Network/privateDnsZones/read permissions under Integration creation through the portal. In a hub and spoke architecture, when spoke team members have no permissions over the hub resources (like private DNS zones) would creating a custom RBAC role with the two permissions mentioned by you allow them to effectively select the hub subscription, and have the DNS record created in the private DNS zones ? Thanks !
Markus Lintuala · 11.09.2023 at 22.29
Yes, you give permissions strict to only one private DNS zone or for the resource group of them. Then you have to state it governance document or some info for developers that use hub subscription and a specific zone under it to create DNS record linking.
Sven · 27.06.2023 at 10.48
Hi Markus
Is it possible to resolve the private address over the internet without a custom DNS server?
When following your first step, I still resolve the public addresses from Microsoft even with an Azure Private DNS Zone called “privatelink.file.core.windows.net” where the A record for the storage account is set to the private endpoint IP?
Is there something I am missing.
Thanks,
Sven
Markus Lintuala · 11.09.2023 at 22.27
Unfortunately not. Private link zones like privatelink.file.core.windows.net zone is restricted to use only in private DNS zones. You can not validate the public name for the domain to use public zones.