Can I expand the size of a file based disk image?
Solution 1
First you have to create an image file:
# dd if=/dev/zero of=./binary.img bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 10.3739 s, 101 MB/s
Then, you have to create a partition on it -- you can use whatever tool you want, fdisk
, parted
, gparted
, I prefer parted
, so:
# parted binary.img
You have to create a partition table first and then one big partition:
(parted) mktable
New disk label type? msdos
(parted) mkpartfs
WARNING: you are attempting to use parted to operate on (mkpartfs) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs. We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Partition type? primary/extended? primary
File system type? [ext2]? fat32
Start? 1
End? 1049M
Now let's see:
(parted) print
Model: (file)
Disk /media/binary.img: 1049MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 1049MB 1048MB primary fat32 lba
It looks good,
You want to enlarge it, so fist add some zeros to the image using dd:
# dd if=/dev/zero bs=1M count=400 >> ./binary.img
400+0 records in
400+0 records out
419430400 bytes (419 MB) copied, 2.54333 s, 165 MB/s
root:/media# ls -al binary.img
-rw-r--r-- 1 root root 1.4G Dec 26 06:47 binary.img
That added 400M to the image:
# parted binary.img
GNU Parted 2.3
Using /media/binary.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: (file)
Disk /media/binary.img: 1468MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 1049MB 1048MB primary fat32 lba
As you can see, the size of the image is different (1468MB). Parted can also show you free space in the image. If you want to see it just type print free
instead of print
. Now you have to add the extra space to the filesystem:
(parted) resize 1
WARNING: you are attempting to use parted to operate on (resize) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs. We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Start? [1049kB]?
End? [1049MB]? 1468M
and check it:
(parted) print
Model: (file)
Disk /media/binary.img: 1468MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 1468MB 1467MB primary fat32 lba
Pretty nice. If you want to shrink it, just do similar thing:
(parted) resize 1
WARNING: you are attempting to use parted to operate on (resize) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs. We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Start? [1049kB]?
End? [1468MB]? 500M
Now you can check if the partition is smaller:
(parted) print
Model: (file)
Disk /media/binary.img: 1468MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 500MB 499MB primary fat32 lba
Yes, it is.
If you try to resize the partition when data is on it, you have to pay attention to the size of the data because when you shrink it too much, you will get an error:
Error: Unable to satisfy all constraints on the partition
After shrinking the file system, you also have to cut some of the file off. But this is tricky. You could take the value from parted 500M (END):
# dd if=./binary.img of=./binary.img.new bs=1M count=500
But this leaves some space at the end of the file. I'm not sure why, but the image works.
And there's one thing about mounting such image -- you have to know an offset to pass to the mount command. You can get the offset from, for instance, fdisk:
# fdisk -l binary.img
Disk binary.img: 1468 MB, 1468006400 bytes
4 heads, 32 sectors/track, 22400 cylinders, total 2867200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f0321
Device Boot Start End Blocks Id System
binary.img1 2048 2867198 1432575+ c W95 FAT32 (LBA)
2048 (start) x 512 (sector size) = 1048576 , so you have to use the following command in order to mount the image:
# mount -o loop,offset=1048576 binary.img /mnt
Solution 2
Yes, this is possible - it works just like a partition. I tried the following, which worked:
Make the original file, mount it, check, unmount it
dd if=/dev/zero of=test.file count=102400
mkfs.ext3 test.file
mount test.file /m4 -o loop
df
umount /m4
Grow it
dd if=/dev/zero count=102400 >> test.file
mount test.file /m4 -o loop
df
resize2fs /dev/loop0
df
There is no reason why shrinking a file would not work similarly, but shrinking a file is always more difficult then growing a file (and, of-course, needs to be done when the block device is not mounted etc)
Have a look at this link which talks about using qemu-nbd to mount qcow2 images
Solution 3
Sparse files are a good choice for dynamic grow/resize disk images.
This will create a 1024M sparse file:
# dd if=/dev/zero of=sparse.img bs=1M count=0 seek=1024
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000565999 s, 0.0 kB/s
The image is not using any disk space,
# du -m sparse.img
0 sparse.img
but has the apparent size of 1024M.
# ls -l sparse.img
-rw-rw-r--. 1 root root 1073741824 Sep 22 14:22 sparse.img
# du -m --apparent-size sparse.img
1024 sparse.img
You can format and mount it as a regular disk image:
# parted sparse.img
GNU Parted 2.1
Using /tmp/sparse.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mktable
New disk label type? msdos
(parted) mkpartfs
WARNING: you are attempting to use parted to operate on (mkpartfs) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs. We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Partition type? primary/extended? primary
File system type? [ext2]? fat32
Start? 1
End? 1024M
(parted) print
Model: (file)
Disk /tmp/sparse.img: 1074MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 1024MB 1023MB primary fat32 lba
# du -m sparse.img
2 sparse.img
Now, resize using the same command to create just changing the seek parameter with the new size of image:
dd if=/dev/zero of=sparse.img bs=1M count=0 seek=2048
As you can see, the image is now 2048M and you can enlarge the partition using parted or other tool of your choice.
# du -m --apparent-size sparse.img
2048 sparse.img
# parted sparse.img
GNU Parted 2.1
Using /tmp/sparse.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print free
Model: (file)
Disk /tmp/sparse.img: 2147MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
16.4kB 1049kB 1032kB Free Space
1 1049kB 1024MB 1023MB primary fat32 lba
1024MB 2147MB 1123MB Free Space
(parted)
# du -m sparse.img
2 sparse.img
Now enjoy it!
Related videos on Youtube
Youstay Igo
Updated on September 18, 2022Comments
-
Youstay Igo over 1 year
So I have an object. And the object has 6 variables (a, b, c, d, e, f)
Something like this:
let obj={ a:0, b:0, ... ... }
I am passing this object as a function parameter and the function has to set the values of all the variables (a to f). The values are simply copied from other variables (lets call them u, v, w, x, y, z).
Based on the user's (my own) choice, a could be mapped to w or y or z etc and so on. Currently I am doing this manually like this:
obj.a=x; obj.b=y; obj.c=w; ... ...
The issue is, I want this mapping pattern to be very flexible. I want to change the entire mapping with the least change of coding and I want the coding to be as minimal/clean as possible. Currently there are 4 possible mapping options based on how I want to do it at runtime. But I want to add more options and as mentioned above, I want the mappings to be very change-friendly.
Should I declare a mapping string and use eval to automate the mapping? Then I would have to simply change the string arrangement and the values will be mapped automatically. Something like this:
var mapping="xyzuvw", values="abcdef"; for(var i=0;i<6;++i) //using 6 since mapping string is of 6 chars eval("obj."+values[i]+"="+mapping[i]);
As you can see, I only have to change (or create a new) mapping string and all the mapping sequence is automated.
The software is intended entirely for my own/private use.
Should I use eval() to automate this mapping process or should I manually edit (and create new) the mapping coding if I want to change the mapping pattern?
As in, will using eval() make the code prone to processing hiccups, runtime errors etc?
Will maintaining the code be easier in future with eval() or with manual mapping?
Edit to add:
For members who are referring me to Dynamically access object property using variabl, please note that I am trying to automate target variable name (whose value we want to copy) and not base variable name (whose value we want to change). As in, I want to automate whether obj.a = x; or whether obj.a=y;
Thanks.
-
just about 7 yearsTomasz: use the dd
seek=
param from the other answer -
ste about 6 yearsWhen you write "But this leaves some space at the end of the file. I'm not sure why, but the image works." I think that it is caused by the fact that you are using different units of measurement. In parted, you have 500MB (in parted 1MB is 1000 * 1000 bytes), so 500000000 bytes; in dd you copy 1Mx500=500M, where M stands for 1024*1024 bytes, so you copy 524288000 bytes. The difference should be the space you left at the end of the file. To fix, you could do with truncate as described here softwarebakery.com/shrinking-images-on-linux or change the units of meas. in parted or in dd
-
Abdillah about 5 yearsNow it's irrelevant to use
parted
withresize
: "Error: The resize command has been removed in parted 3.0" -
TonyH about 5 yearsNicely done. I think sparse files are underused.