XZ - Cannot allocate memory

Question

If i run this command:

XZ_OPT=-9 tar --xz -cvf files/compressed/xz/archive.tar.xz -C files/original/ .

Get this message:

xz: (stdin): Cannot allocate memory
tar: files/compressed/lzma//archive.lzma: Wrote only 4096 of 10240 bytes
tar: Error is not recoverable: exiting now

What type of memory it is? Or how do i set it to make it work.

EDIT: (aditional info)

Total file size that I want to compress: 18.92M

Gzip Bzip2 ZIP - works OK

xz --info-memory:

Total amount of physical memory (RAM):  595 MiB (623116288 B)
Memory usage limit for compression:     Disabled
Memory usage limit for decompression:   Disabled

ulimit -a:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 2312
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 2312
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Solution 1

In man xz, you'll find that -9 requires 674 MiB of memory for compression (and that it's only useful if you're compressing files bigger than 32 MiB). Try adding about this much swap to provide enough virtual memory for the operation (assuming you're using all your current memory for other purposes).

Solution 2

I know this is an old thread , but for a xz -9 there is a memory limit of 13Mib , so if you set it to use --memory=135000000 it should run if you have that much available.

Comments

[Stephen Kitt]

What does xz --info-memory say? And ulimit -a?

[Lajdák Marek]

Updated question.

[Lajdák Marek]

I need comression setting 9 because this is test... and after set XZ_DEFAULTS=--memlimit=150MiB archive was created but i have this message Adjusted LZMA2 dictionary size from 64 MiB to 4 MiB to not exceed the memory usage limit of 150 MiB - is it ok? Is possible turn off this message? (some parameter for ex. -q as quite or whatever)

[Avio]

Thanks, this seems to work. At first I tried with --memory=750000000 but I got: xz: Memory usage limit is too low for the given filter setup. xz: 1.250 MiB of memory is required. The limit is 716 MiB. Trying with --memory=1500000000 finally worked.