How do I edit .mca files from Minecraft?

Question

Introduction

I want to combine my separate Minecraft worlds into a single world and it seemed like a relatively easy feat, but as I did research it evolved into the need to make a custom program.

The Struggle

I started by shifting the region files and combining them in one region folder, which seemed like the obvious solution and it almost worked. Note: I've opened the files and it seems entire sectors have their coordinates stored, not entities, hence the terrain itself is spatially mismatched with the region file name.

That led to quite a bit of lag when I opened the client and the regions failed to render. I read up on the Anvil file format and imagined a scheme for reading NBT files. I figured I could manually read out the bytes and edit them, but in my continued research I got conflicting answers as to whether region files are gzipped.

I finished enough code to read some raw bytes, but the byte values didn't come out as I expected.

According to the info I have on NBT files, they all start with a CompoundTag and a CompoundTag starts as a single byte valued as 10, or x0A. This is where I got my format information: https://minecraft.gamepedia.com/NBT_format

Here's a screenshot of what actually came out:

Note: The class description in the screenshot is not accurate. I just quickly filled in enough to read the bytes, not flesh out the UI function.

I assume these bytes coming out as non-sense is a sign that the file is compressed. I found this as a start to the gzip problem: http://gnuwin32.sourceforge.net/packages/gzip.htm

I imagine if I could get this installed it would unzip this .mca file and I could read the bytes as expected, but I don't understand the installation instructions. It says use the "Shell Commands, 'configure', 'make' and 'make install'". To me that sounds like Unix, but the file I downloaded is for Windows? There aren't any exe's, but there are quite a few C files. I don't have a C-compiler. . .

Note: I still have not got the gzip software to work.

Post Script

I've seen similar questions asked here, but all of them were either old (2016ish) with dead links to software that used to work, or they were recent and unanswered. I found one specific copy of this question asked 5 months ago, but I had to make an account to comment. Here's the link: How can read Minecraft .mca files so that in python I can extract individual blocks? His question is with regard to a Python implementation. He said he found an NBT library for Python, but it was rejecting his MCA files for being not-gzipped.

I've got a lead on understanding the problem because I have the NBTExplorer source code (see the answer I posted), but I'll have to update on how that pans out. As far as getting my world fixed, I think I have a viable solution now.

If anyone could point me to a finished Java library, with source code, that opens .mca's or a discussion board related to this topic that'd be cool. I'm still also interested in how file compression works, but that's probably outside this question's scope. I realize this isn't directly bug or error related; it's it was moreso that I didn't know what further steps to take to make a code that accomplishes this task.

Update

I found someone else's program to do this and posted it as an answer, but I'd still like to know how the file is converted from bytes to useable info. Using the manual edit method of the answer I posted, I will need at most 241,664 manual edits, so I still need a better solution.

Solution 1

First of: As far as I know there is no more information about "where the chunks are", stored in the region files. There are 32(x direction)*32(z-direction)= 1024 Chunks stored within one region file and each of it has its position of data within the file. So the chunks are just numbered within the file itself and the first 8192 bytes are just about if there is any data about that specific chunk or not, where its found within the file and when it got last updated. Where the complete region (those 1024 Chunks) are positioned within the world can be worked out within the file name where the regions themself are numbered in x and z direction.

So in your case you should be able to rename your region files in a way they stay togehter as they are in the original worlds and you should be able to merge them together.

Second: The NBT Format is not the first thing to look at when you want to decode the data. First of the Region files have their own structure: https://minecraft.gamepedia.com/Region_file_format and when you get to the actual data using Zlib (RFC1950) it's getting complicated...

Anyway if you want further information on how to decode I can give you some information (since the files https://www.rfc-editor.org/rfc/rfc1950.html and https://www.rfc-editor.org/rfc/rfc1951 about Zlib (RFC1950) are written in a hard way to understand - at least it was for me). But theres a point where I myself am struggeling right now which is why I came across this question.

Solution 2

I found an editor!

Now I can edit, but I don't know how the editing works. I haven't learned anything, but I did finally find someone else's editor. Not quite what I wanted because I wanted to know how to do this myself.

Update: To fix a region using this software I have to manually edit 2 fields, for up to 32x32 chunks, and I have 118 regions that I need to fix. **That's 241,664 potential manual edits! This solution is not viable on a reasonable timescale, but it's the best I have so far:

I found this page: https://fileinfo.com/extension/mca

Which linked to this page: https://fileinfo.com/software/nbtexplorer/nbtexplorer

Which linked to this page: https://github.com/jaquadro/NBTExplorer/releases

I installed the software and it automatically linked to the .minecraft folder, here's a screenshot of the GUI:

On the bright side, the application download page also has a download link for the source, so I intend to read that! I've opened two files so far to take a glance and they were not commented at all. They're also written in C# which I have never seen before, but I've heard it's very similar to Java, so maybe I'll learn that language too.

Solution 3

This Java library is quite nice for editing .mca and has some examples of doing so in the README

https://github.com/Querz/NBT

As for how the compression works, chunks can be individually compressed via either gzip or zlib, but in practice are generally all zlib compressed, which is implemented in Java through Inflater and Deflater. One annoying thing about the format for chunk data is it is only prefixed with the size of the compressed buffer, with no info on the size of the uncompressed buffer (so the uncompressed buffer must be estimated to be large enough or multiple buffers can be used to fill until the compressed buffer is completely "inflated").