I recently had to deploy concourse 5 behind a web proxy. All seemed well after deploying it but then something surprising happened…
Surprising to me at least. Which mostly shows my lack of knowledge of using a proxy on a Linux machine. I normally try to avoid this since, in my experience, not all Linux applications behave nicely behind a proxy.
When you use BOSH to deploy Concourse setting the proxy is quite easy. First of all make sure you use the concourse-bosh-deployment repo to build your manifest. In the cluster/operations folder in that repo you’ll find the http-proxy.yml operator file. Include this in you deployment, supply the http(s)_proxy_url and no_proxy variable and you’re done. I made sure to list my local IP subnets in CIDR (10.0.0.0/8) notation so the proxy won’t be hit when accessing internal IPs…. more on this later.
After deployment I did some testing and all seemed well. Until I pushed a pipeline that is supposed to keep a BOSH deployment up to date. The bosh-deployment resource just refused to connect to the BOSH director and kept failing with this error:
Fetching info: Performing request GET 'https://<redacted>:25555/info': Performing GET request: Retry: Get https://<redacted>:25555/info: Forbidden
After quite a few hours troubleshooting I discovered that the BOSH cli running in the concourse resource was trying to connect through the proxy, not directly to the IP address. But I had the whole subnet listed in the no_proxy environment variable so that is supposed to do the trick right?
Wrong! As it turns out, the no_proxy variable only takes suffixes. So listing your internal domain suffixes works fine. When you want to connect on IP rather than FQDN you can list a single IP. But you CANNOT list a subnet. Avoiding the proxy for a whole subnet only works if you list all IP addresses in the subnet. But that’s impossible, there are simply too many. To make matters worse some linux applications seem to interpret the CIDR just fine. I did some testing with curl and that seemed to take it. But the concourse bosh deployment resource certainly does not.
So how to fix this? As any IT guy would do I googled the problem, landed at stackoverflow and found a solution. The trick for me was to list all digits as a suffix like so:
<span class="pln">no_proxy</span><span class="pun">=</span><span class="lit">0</span><span class="pun">,</span><span class="lit">1</span><span class="pun">,</span><span class="lit">2</span><span class="pun">,</span><span class="lit">3</span><span class="pun">,</span><span class="lit">4</span><span class="pun">,</span><span class="lit">5</span><span class="pun">,</span><span class="lit">6</span><span class="pun">,</span><span class="lit">7</span><span class="pun">,</span><span class="lit">8</span><span class="pun">,</span><span class="lit">9,automate-it.local</span>
Since no TLDs end with a number you’re now sure all IPs will avoid the proxy. Of course this means all IPs world-wide, not only local IP subnets but this works for me. For now at least.
The original article was posted on: automate-it.today
