Runner is not healthy and will be disabled

Question

I hosts my repository with gitlab.com and I install runner in the DigitalOcean. It ran fine until today 16March2019 14:24 Thailand time.

# gitlab-runner status
Runtime platform                                    arch=amd64 os=linux pid=16937 revision=4745a6f3 version=11.8.0
gitlab-runner: Service is running!
# gitlab-runner unregister --all-runners
Runtime platform                                    arch=amd64 os=linux pid=16299 revision=4745a6f3 version=11.8.0
Running in system-mode.

WARNING: Unregistering all runners
ERROR: Unregistering runner from GitLab forbidden   runner=2bcd7af4
ERROR: Failed to unregister runner HerrRunner
# gitlab-runner list
Runtime platform                                    arch=amd64 os=linux pid=16346 revision=4745a6f3 version=11.8.0
Listing configured runners                          ConfigFile=/etc/gitlab-runner/config.toml
HerrRunner                                          Executor=shell Token=2bcd7af455f866ede7991992a68780 URL=https://gitlab.com/
# gitlab-runner --debug run
Runtime platform                                    arch=amd64 os=linux pid=16395 revision=4745a6f3 version=11.8.0
Starting multi-runner from /etc/gitlab-runner/config.toml ...  builds=0
Checking runtime mode                               GOOS=linux uid=0
Running in system-mode.

Configuration loaded                                builds=0
listenaddress: ""
sessionserver:
  listenaddress: ""
  advertiseaddress: ""
  sessiontimeout: 1800
metricsserveraddress: ""
concurrent: 1
checkinterval: 0
loglevel: null
logformat: null
user: ""
runners:
- name: HerrRunner
  limit: 0
  outputlimit: 0
  requestconcurrency: 0
  runnercredentials:
    url: https://gitlab.com/
    token: 2bcd7af455f866ede7991992a68780
    tlscafile: ""
    tlscertfile: ""
    tlskeyfile: ""
  runnersettings:
    executor: shell
    buildsdir: ""
    cachedir: ""
    cloneurl: ""
    environment: []
    preclonescript: ""
    prebuildscript: ""
    postbuildscript: ""
    shell: ""
    ssh: null
    docker: null
    parallels: null
    virtualbox: null
    cache:
      type: ""
      path: ""
      shared: false
      s3: null
      gcs: null
      s3cachepath: ""
      cacheshared: false
      serveraddress: ""
      accesskey: ""
      secretkey: ""
      bucketname: ""
      bucketlocation: ""
      insecure: false
    machine: null
    kubernetes: null
sentrydsn: null
modtime: 2018-08-12T18:07:07.963445119Z
loaded: true
  builds=0
Waiting for stop signal                             builds=0
Listen address not defined, metrics server disabled  builds=0
Listen address not defined, session server disabled  builds=0
Starting worker                                     builds=0 worker=0
Feeding runners to channel                          builds=0
Dialing: tcp gitlab.com:443 ...
ERROR: Checking for jobs... forbidden               runner=2bcd7af4
Feeding runners to channel                          builds=0
ERROR: Checking for jobs... forbidden               runner=2bcd7af4
Feeding runners to channel                          builds=0
ERROR: Checking for jobs... forbidden               runner=2bcd7af4
ERROR: Runner https://gitlab.com/2bcd7af455f866ede7991992a68780 is not healthy and will be disabled!
Feeding runners to channel                          builds=0
Feeding runners to channel                          builds=0
Feeding runners to channel                          builds=0
^CWARNING: Requested service stop: interrupt          builds=0
All workers stopped. Can exit now                   builds=0

Ultimate Goal

Get my runner up and run again

Question:

1. What does not healthy means?

2. I can't unregister my runner. How to fix this?

Solution 1

No idea. But seems like gitlab.com remove my runner token. Therefore I have to remove my runner, register, and run it again.

Solution 2

I was integrating my runner for the first time, and anyone familiar with gitlab will know this was never going to be a walk in the park. I spent many hours chasing down this error message only to find I was completely looking in the wrong direction.

ERROR: Checking for jobs... forbidden               runner=s3xBVnW8
ERROR: Checking for jobs... forbidden               runner=s3xBVnW8
ERROR: Checking for jobs... forbidden               runner=s3xBVnW8
ERROR: Runner https://gitlab.com/s3xBVnW8JZPTaocALN3i is not healthy and will be disabled!

I now believe my runner was telling me that the "runner" at gitlab.com is not healthy and has been rejected (by my runner). When I left it running and put a pipeline job in, it processed fine, there was no issue at my end, I had spent hours looking for nothing!

Solution 3

I faced with the same issue. Register you runner as described in the installation instructions. Go to admin/runners and click Show runner installation instructions button. The most important is Register runner session where you have to run register command that resolves the issue described. For instance, linux:

sudo gitlab-runner register --url YOUR_GITLAB_SERVER_URL --registration-token TOKEN

# YOUR_GITLAB_SERVER_URL like http://localhost:8100/ or https://mygitlabserver.com/

Follow instructions and you are good to go.

Once your runner is setup and running you can add modifications to the config file, stop and start it again after config changes.

You may find duplicate [[runners]] sections, that happens if you tried to configure it yourself and then with register command. That's fine, leave only one latest.

Comments

[Madura Dissanayake]

I was having same issue, any answer found ?

[ssi-anik]

So did I. when I was trying locally in a docker container, it kept saying gitlab-runner: "service" failed: exec: "service": executable file not found in $PATH and I tried to figure it out for hours. From alpine to ubuntu, from the latest to tagged versions, also found a few different issues. Later running the job worked fine! 🤦