Working with large files

Storing large files in Git is usually a bad idea. They're contained in its history forever, and they enlargen your repository size, which annoys your contributors and increases the cost of storage on Codeberg.

However, there is a good way to do this: Using Git LFS (Large File Storage), you can easily manage binary files, and remove them permanently when they are no longer necessary.

The files are still being tracked with Git, but the actual content is stored in a different place on the Codeberg servers.

Installation

Git LFS is a command line plugin for Git.

You can install it from the Git LFS Website, or via the package manager of your distribution. A more detailed installation description can be found at the Installation article of the Git LFS wiki.

After you install Git LFS, it needs to be enabled in your Git configuration.

This is achieved by calling:

git lfs install

You need to do this only once for every user you want to use Git LFS with.

Tracking large files with Git LFS

To track files with Git LFS, the file and/or path-names must be configured to be managed by Git LFS.

git lfs track $fileorpathname

You can add filenames, directories or even wild-card representations of files (e.g. images/** for all files in the images/ directory, or **.dat for all files ending with .dat).

This creates a .gitattributes file containing (apart from other non LFS-related stuff) the configuration of Git LFS. Be sure to add it to your repository as well.

git add .gitattributes

Now, you can work with Git as you would usually do. Add files with git add, commit with git commit, and then push with git push. Git LFS will transparently handle the use of LFS on the configured files for you. As long as your .gitattributes file is present in the remote repository, you can just use Git as usual.

Which files are currently tracked?

To find out which file patterns are currently configured to be tracked with LFS:

git lfs track

To list the actual files that are tracked with LFS:

git lfs ls-files

What is locking?

When you push your changes to Codeberg, Git LFS may tell you to enable locking:

Locking support detected on remote "origin". Consider enabling it with:
$ git config lfs.https://codeberg.org/your-user-name/your-repository.git/info/lfs.locksverify true

Once you enable this, Git LFS locks files that you are updating to prevent other users from updating them at the same time.

Codeberg LFS supports locking, and it's probably a good idea to activate it. Further details on the locking mechanism can be found in the Git LFS wiki article on File Locking.

Git LFS and the web interface

Unfortunately, Git LFS files cannot be edited in the web interface. However, that shouldn't be a problem as the files tracked by LFS should be large in terms of file size anyway.

Reduce local repository size

You can reduce the size of your local repository (the .git folder) by calling git lfs prune. This will remove all files that aren't in the current HEAD.

Info

This means that older versions of the files won't be stored on your local disk. If you checkout an older commit that still references the file, it will be downloaded from the LFS server. If the server side is not available, the requested version of the file will be inaccessible.

Enabling and Disabling LFS

Use git lfs migrate to enable or disable the tracking of files with LFS in an existing repository. All changes done by git lfs migrate are done in your local working copy. Nothing will be done on the server side unless you push your changes to the server.

Warning

git lfs migrate will rewrite the history of your repository.

Make sure no one else is working with the repository during your changes. Also keep in mind that all users must refresh their local repositories prior to doing changes to the altered repository.

Enabling LFS in an existing repository

If you already have a large file in your repository, it won't automatically be moved to LFS just by enabling the tracking.

You can first run git lfs migrate info to see which files take up the most space in your repository. Append --everything to see the sizes of all files (even the smaller ones).

Then you can run git lfs migrate import to import some or all files into using LFS.

Afterwards you need to call git push to push your changes to your repository.

As the man page of git-lfs-migrate(1) already contains a much more detailed description on how to import files, you are encouraged to read it to learn more about importing before starting the migration.

Disabling LFS

If you want to disable LFS in your repository or for specific files, you can use git lfs migrate export.

Note you must always specify a reference or --everything when using the export command so that Git LFS knows whether you want to stop using LFS for the files in a specific reference or in your whole repository.

Example:

git lfs migrate export --include=\*.rnd --everything

This disables LFS for all files ending with .rnd in your repository.

The man page of git-lfs-migrate(1) further explains how to export files in order to disable tracking them by LFS.

Further reading


Hey there! 👋 Thank you for reading this article!

Is there something missing, or do you have an idea on how to improve the documentation? Do you want to write your own article?

You're invited to contribute to the Codeberg Documentation at its source code repository, for example, by adding a pull request or joining in on the discussion in the issue tracker.

For an introduction on contributing to Codeberg Documentation, please have a look at the Contributor FAQ.

© Codeberg Docs Contributors. See LICENSE