### ArgoCD Failing to connect to repo-server

I faced this error in my argo cd applicationset controller logs:

```bash
argocd-applicationset-controller-d79554d98-qbp8t applicationset-controller time="2024-09-19T10:53:28Z" level=error msg="error generating params" error="error generating params from git: error getting directories from repo: error retrieving Git Directories: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp: lookup argocd-repo-server: i/o timeout\"" generator="&{0xc000575d40 argocd}"
argocd-applicationset-controller-d79554d98-qbp8t applicationset-controller time="2024-09-19T10:53:28Z" level=error msg="error generating application from params" applicationset=argocd/blogs error="error generating params from git: error getting directories from repo: error retrieving Git Directories: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp: lookup argocd-repo-server: i/o timeout\"" generator="{nil nil &GitGenerator{RepoURL:https://gitlab.com/x/homelab/gitops.git,Directories:[]GitDirectoryGeneratorItem{GitDirectoryGeneratorItem{Path:apps/blogs/*,Exclude:false,},},Files:[]GitFileGeneratorItem{},Revision:HEAD,RequeueAfterSeconds:nil,Template:ApplicationSetTemplate{ApplicationSetTemplateMeta:ApplicationSetTemplateMeta{Name:,Namespace:,Labels:map[string]string{},Annotations:map[string]string{},Finalizers:[],},Spec:ApplicationSpec{Source:nil,Destination:ApplicationDestination{Server:,Namespace:,Name:,},Project:,SyncPolicy:nil,IgnoreDifferences:[]ResourceIgnoreDifferences{},Info:[]Info{},RevisionHistoryLimit:nil,Sources:[]ApplicationSource{},},},PathParamPrefix:,Values:map[string]string{},} nil nil nil nil nil nil nil}"
```

And in my repo-server:

```bash
argocd-repo-server-797fcbd5c7-lp56l repo-server time="2024-09-19T10:49:23Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=Check grpc.service=grpc.health.v1.Health grpc.start_time="2024-09-19T10:49:23Z" grpc.time_ms=0.028 span.kind=server system=grpc
```

When you go the your repositories in the Argo CD frontend, you will notice theres a timeout trying to fetch the repositories.

### Debugging

First we need to see if we can resolve the address of `argocd-repo-server` from the namespace where argocd is deployed:

```bash
kubectl run -it --rm --image=busybox dns-test -n argocd -- nslookup argocd-repo-server
# ;; connection timed out; no servers could be reached
```

We can see that we cannot resolve addresses, I also tried resolving external addresses which also failed.

The next step is debugging `coredns`, I have done a rollout restart on that deployment:

```bash
kubectl rollout restart deployment/coredns -n kube-system
```

Once the deployment was restarted, I was able to get a response:

```bash
kubectl run -it --rm --image=busybox dns-test -n argocd -- nslookup argocd-repo-server
```

Response:

```bash
Server:		10.43.0.10
Address:	10.43.0.10:53
Name:	argocd-repo-server.argocd.svc.cluster.local
Address: 10.43.82.176
```

Then the argocd-applicationset-controller was working again.

### Thanks

For more [argocd](https://ruan.dev/tags/argocd) content, check out my blog:
- https://ruan.dev/tags/argocd