Further Migration Notes

This page provides additional notes and explanations regarding the migration from an external GitLab instance to git.nrw. We will continue to expand this page.

1. Migration Types

Two methods are primarily available for data transfer: Direct Transfer and the classic file-based export/import of individual repositories.

Important note regarding private projects: Since Direct Transfer operates at the group level, purely private projects cannot be migrated directly this way. Before starting the migration, temporarily move private projects into a group on the source platform so that Direct Transfer can be used for them as well.

2. Transfer Scope

Direct Transfer migrates most of a repository. Exceptions here are artifacts and packages, as well as the container registry. Artifacts and packages are regenerated on git.nrw during the first run of the CI pipeline. The container registry, however, must be set up again.

A complete list of the data included in and excluded from Direct Transfer migration can be found in the GitLab documentation .

3. Preparing for Migration

The following steps help ensure a smooth migration and should be carried out periodically in your repositories anyway.

3.1. Cleaning Up the Repository

• Check file sizes: Check the repository for large files that aren’t strictly necessary (e.g. binary data, log files, temporary build artifacts).
• Cleanup tools: There are tools available to make finding and cleaning up large files easier. GitLab recommends git-filter-repo for this.
• Housekeeping: After deleting objects, they initially remain in the old commits. Manually trigger housekeeping in the GitLab project settings to free up server storage space via garbage collection. Note that GitLab’s garbage collection runs on a schedule, so the actual cleanup happens with a delay.

4. Groups

On git.nrw, top-level groups must be requested from your home institution. It may therefore be necessary to think through your group structure on git.nrw separately before migrating.

Direct Transfer migrates entire group structures from the source instance, starting from the top-level group. As a result, this group structure is migrated as a subgroup of the previously created top-level group on git.nrw. This creates an additional hierarchy level, which affects project paths.

See also our notes on group management .

5. User Management

GitLab cannot transfer users from one GitLab instance to another. However, during the transfer, so-called placeholder users are created in the migrated top-level group. All activity (e.g. commits) from the original user on the source GitLab is attached to these placeholders. By assigning the placeholders to real users on git.nrw, the previous relationships are restored. If a user no longer exists on git.nrw (e.g. has left the institution), the commit email addresses remain attached to the commits to ensure traceability.

Important: Internally, GitLab links data based on each user’s internal user ID. Changing a username or display name therefore has no effect on the association with the data in GitLab.

6. Handling the 2GB Repo Size Limit and Oversized Projects

On the target platform git.nrw, a storage quota of 2 GB per project applies (explicitly including LFS), whereby the following data is not counted toward the quota:

• Artifacts
• Containers
• Packages
• Snippets
• Uploads
• Wikis

Source: https://docs.gitlab.com/administration/settings/account_and_limit_settings/#repository-size-limit

Note: You can find exactly how the repo size limit is composed for your project on each project’s homepage, on the right-hand side under:

If the repo size limit is exceeded, the following happens:

• Projects within groups: During Direct Transfer, projects are transferred despite exceeding the repo size limit. The project is marked as read-only on git.nrw. Pushing or running a pipeline is only possible again once the repo size limit is met again.

• Private projects: If an import into a personal namespace (or a single project within a group) exceeds the limit, the import process fails and aborts. In this case, prior cleanup is required, or — for a pure transfer — moving the project into a group that is being migrated.

  Tip: If you need to work with large external data, take a look at git-annex . This allows you to interact with external data storage. Please note, however, that we are unable to offer support for git-annex.

7. After the Migration

A list of recommendations after the migration:

• Check the functionality of your processes and adjust them if necessary
  • Re-register any GitLab runners you use, if applicable
  • Set up the container registry again, if applicable
  • Adjust paths and secrets
  • etc.
• Update your local Git repositories: Update git remote URLs
• If your projects contain submodules with absolute URLs pointing to the old instance, you can transparently redirect these via a global Git configuration, without having to adjust .gitmodules in every repo:
  git config --global --add "url.https://gitlab.git.nrw/<group>/.insteadOf" "https://example-gitlab.com/<group>/"
• Make sure the new URL is communicated to interested parties
  • Prevent further edits to the project by other people on the source instance (e.g. by deletion or archiving)