Skip to main content
  1. Posts/

Resolving Git Large Repository Download Failures on AIX by Removing File Resource Limits

·368 words·2 mins· ·
Xianpeng Shen
Author
Xianpeng Shen

Recently, when downloading code from Bitbucket using AIX 7.1, I encountered this error:

fatal: write error: A file cannot be larger than the value set by ulimit.

$ git clone -b dev https://<username>:<password>@git.company.com/scm/vmcc/opensrc.git --depth 1
Cloning into 'opensrc'...
remote: Counting objects: 2390, done.
remote: Compressing objects: 100% (1546/1546), done.
fatal: write error: A file cannot be larger than the value set by ulimit.
fatal: index-pack failed

On AIX 7.3, I encountered this error:

fatal: fetch-pack: invalid index-pack output

$ git clone -b dev https://<username>:<password>@git.company.com/scm/vmcc/opensrc.git --depth 1
Cloning into 'opensrc'...
remote: Counting objects: 2390, done.
remote: Compressing objects: 100% (1546/1546), done.
fatal: write error: File too large68), 1012.13 MiB | 15.38 MiB/s
fatal: fetch-pack: invalid index-pack output

This is because the files in this repository are too large, exceeding the AIX limit on user file resource usage.

This can be checked using ulimit -a. More information about the ulimit command can be found at ulimit Command

$ ulimit -a
time(seconds)        unlimited
file(blocks)         2097151
data(kbytes)         unlimited
stack(kbytes)        32768
memory(kbytes)       32768
coredump(blocks)     2097151
nofiles(descriptors) 2000
threads(per process) unlimited
processes(per user)  unlimited

We can see that file has an upper limit of 2097151. Changing this to unlimited should solve the problem.

The limits file /etc/security/limits is accessible by the root user (ordinary users do not have access permissions).

# The following is part of the content of this file

default:
        fsize = 2097151
        core = 2097151
        cpu = -1
        data = -1
        rss = 65536
        stack = 65536
        nofiles = 2000

Changing fsize = 2097151 to fsize = -1 removes the file size limit. After making this change, log in again for the changes to take effect.

Executing ulimit -a again shows that the change has taken effect.

$ ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        32768
memory(kbytes)       32768
coredump(blocks)     2097151
nofiles(descriptors) 2000
threads(per process) unlimited
processes(per user)  unlimited

Now file(blocks) is unlimited. Let’s try git clone again.

git clone -b dev https://<username>:<password>@git.company.com/scm/vmcc/opensrc.git --depth 1
Cloning into 'opensrc'...
remote: Counting objects: 2390, done.
remote: Compressing objects: 100% (1546/1546), done.
remote: Total 2390 (delta 763), reused 2369 (delta 763)
Receiving objects: 100% (2390/2390), 3.80 GiB | 3.92 MiB/s, done.
Resolving deltas: 100% (763/763), done.
Checking out files: 100% (3065/3065), done.

Success!

Related

Branch Naming Convention
·205 words·1 min
This article introduces the conventional branch naming specification, including the purpose of branch names, key points, and basic rules for naming branches in Git. It also provides examples of branch prefixes and their meanings.
These settings in Bitbucket/GitHub recommends enable
·282 words·2 mins
Provides a list of recommended settings for Bitbucket and GitHub repositories, including enabling force push rejection, branch protection, tag management, merge checks, and commit message standards.
git sparse-checkout enable and disable
·127 words·1 min
This article explains how to enable and disable git sparse-checkout, including examples of how to configure sparse-checkout for specific directories and how to reset it.
Programmer's Self-Cultivation — Git Commit Message and Branch Creation Conventions
·1643 words·4 mins
This article introduces how to formulate and implement Git commit message and branch creation conventions to improve code quality and team collaboration efficiency.
How to Slim Down Your Git Repository
·1133 words·6 mins
How to remove unnecessary files and history from a Git repository to reduce its size, providing two methods using BFG Repo Cleaner or git filter-branch.
Git History Statistics Generator
·712 words·4 mins
GitStats, a Git history statistics generation tool written in Python, can generate detailed code submission statistical reports to help developers analyze project activity and contributor information.