How to solve the "remote: You are not allowed to upload code." error on GitLab CI/CD job?

Question

I am currently trying to use GitLab to run a CI/CD job that runs a Python file that makes changes to a particular repository and then commits and pushes those changes to master. I also have a role of Master in the repository. It appears that all git functions run fine except for the git push, which leads to fatal: You are not currently on a branch. and with using git push origin HEAD:master --force, that leads to fatal: unable to access 'https://gitlab-ci-token:xxx@xxx/project.git/': The requested URL returned error: 403. I've been looking over solutions online, one being this one, and another being unprotecting it, and couldn't quite find what I was looking for just yet. This is also a sub-project within the GitLab repository.

Right now, this is pretty much what my .gitlab-ci.yml looks like.

before_script:
  - apt-get update -y
  - apt-get install git -y
  - apt-get install python -y
  - apt-get python-pip -y

main:
  script:
    - git config --global user.email "xxx@xxx"
    - git config --global user.name "xxx xxx"
    - git config --global push.default simple
    - python main.py

My main.py file essentially has a function that creates a new file within an internal directory provided that it doesn't already exist. It has a looks similar to the following:

import os
import json

def createFile(strings):
    print ">>> Pushing to repo...";
    if not os.path.exists('files'):
        os.system('mkdir files');
    for s in strings:
        title = ("files/"+str(s['title'])+".json").encode('utf-8').strip();
        with open(title, 'w') as filedata:
            json.dump(s, filedata, indent=4);
    os.system('git add files/');
    os.system('git commit -m "Added a directory with a JSON file in it..."');
    os.system('git push origin HEAD:master --force');

createFile([{"title":"A"}, {"title":"B"}]);

I'm not entirely sure why this keeps happening, but I have even tried to modify the repository settings to change from protected pull and push access, but when I hit Save, it doesn't actually save. Nonetheless, this is my overall output. I would really appreciate any guidance any can offer.

 Running with gitlab-runner 10.4.0 (00000000)
      on cicd-shared-gitlab-runner (00000000)
 Using Kubernetes namespace: cicd-shared-gitlab-runner
 Using Kubernetes executor with image ubuntu:16.04 ...
 Waiting for pod cicd-shared-gitlab-runner/runner-00000000-project-00000-concurrent-000000 to be running, status is Pending
 Waiting for pod cicd-shared-gitlab-runner/runner-00000000-project-00000-concurrent-000000 to be running, status is Pending
 Running on runner-00000000-project-00000-concurrent-000000 via cicd-shared-gitlab-runner-0000000000-00000...
 Cloning repository...
 Cloning into 'project'...
 Checking out 00000000 as master...
 Skipping Git submodules setup
 $ apt-get update -y >& /dev/null
 $ apt-get install git -y >& /dev/null
 $ apt-get install python -y >& /dev/null
 $ apt-get install python-pip -y >& /dev/null
 $ git config --global user.email "xxx@xxx" >& /dev/null
 $ git config --global user.name "xxx xxx" >& /dev/null
 $ git config --global push.default simple >& /dev/null
 $ python main.py
 [detached HEAD 0000000] Added a directory with a JSON file in it...
  2 files changed, 76 insertions(+)
  create mode 100644 files/A.json
  create mode 100644 files/B.json
 remote: You are not allowed to upload code.
 fatal: unable to access 'https://gitlab-ci-token:xxx@xxx/project.git/': The requested URL returned error: 403
 HEAD detached from 000000
 Changes not staged for commit:
    modified:   otherfiles/otherstuff.txt
 no changes added to commit
 remote: You are not allowed to upload code.
 fatal: unable to access 'https://gitlab-ci-token:xxx@xxx/project.git/': The requested URL returned error: 403
 >>> Pushing to repo...
 Job succeeded

Solution 1

Here is a resource from Gitlab that describes how to make commits to the repository within the CI pipeline: https://gitlab.com/guided-explorations/gitlab-ci-yml-tips-tricks-and-hacks/commit-to-repos-during-ci/commit-to-repos-during-ci

Try configuring your gitlab-ci.yml file to push the changes rather than trying to do it from the python file.

Solution 2

I managed to do this via ssh on a runner by making sure the ssh key is added, and then using the full git url:

task_name:
  stage: some_stage
  script:
    - ssh-add -K ~/.ssh/[ssh key]
    - git push -o ci-skip [email protected]:[path to repo].git HEAD:[branch name]

If it is the same repo that triggered the job, the url could also be written as:

git@$CI_SERVER_HOST:$CI_PROJECT_PATH.git

Solution 3

The requested URL returned error: 403

• The HTTP 403 Forbidden client error status response code indicates that the server understood the request but refuses to authorize it.
• The problem is we cannot provide a valid authentication to git and hence our request is forbidden.
• Try this:Control Panel => User Accounts => Manage your credentials => Windows Credentials

It worked for me.However I'm not quite sure if it will work for you.

Solution 4

This method can be used to commit tags or files. You may also wish to consider using the CI CD variable API to store cross-build persistent data if it does not have to be committed to the repo https://docs.gitlab.com/ee/api/project_level_variables.html https://docs.gitlab.com/ee/api/group_level_variables.html

ACCESS_TOKEN below is a variable at the repo or an upbound group level that contains a token that can write to the target repos. Since maintainer can see these, it is best practice to create tokens on special API users who are least privileged for just what they need to do.

write_to_another_repo:
  before_script:
    - git config --global user.name "${GITLAB_USER_NAME}"
    - git config --global user.email "${GITLAB_USER_EMAIL}"
  script:
    - |
      echo "This CI job demonstrates writing files and tags back to a different repository than this .gitlab-ci.yml is stored in."
      OTHERREPOPATH="guided-explorations/gitlab-ci-yml-tips-tricks-and-hacks/commit-to-repos-during-ci/pushed-to-from-another-repo-ci.git" 
      git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@$CI_SERVER_HOST/$OTHERREPOPATH
      cd pushed-to-from-another-repo-ci
      CURRENTDATE="$(date)"
      echo "$CURRENTDATE added a line" | tee -a timelog.log
      git status
      git add timelog.log
      # "[ci skip]" and "-o ci-skip" prevent a CI trigger loop
      git commit -m "[ci skip] updated timelog.log at $CURRENTDATE"
      git push -o ci-skip http://root:$ACCESS_TOKEN@$CI_SERVER_HOST/$OTHERREPOPATH HEAD:master
      #Tag commit (can be used without commiting files)
      git tag "v$(date +%s)"
      git tag
      git push --tags http://root:$ACCESS_TOKEN@$CI_SERVER_HOST/$OTHERREPOPATH HEAD:master

View more solutions

Comments

[aaronsteers]

Can you clarify how you got the SSH key onto the CI runner? Did you have to preinstall the ssh key file via repository CI parameters, or some other means?

[James Trickey]

I manually added the ssh private key to the build agent. You may be able to add it as a runner file variable though. But I couldn't speak as to the security risk here.

[Admin]

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

[paiego]

How do you get this ACCESS_TOKEN from git or gitlab?

[paiego]

did get the Access Token? And did you get it from Gitlab, or Git?

[mati kepson]

@paiego you need to create the token by yourself under gitlab/profile/personal_access_tokens and next save the ACCESS_TOKEN as a variable inside project's settings/CICD/varibales and this does the trick ( also setting -o ci-skip)