Accessing field of Protobuf message of unknown type in Python
I know this is an old thread, but I'll respond anyway for posterity:
Firstly, as you know, it's not possible to determine the type of a protocol buffer message purely from its serialized form. The only information in the serialized form you have access to is the field numbers, and their serialized values.
Secondly, the "right" way to do this would be to have a proto that contains both, like
message Parent {
required int32 FormatVersion = 1;
optional bool SomeFlag [default = true] = 2;
optional Common CommonSettings = 3;
oneof letters_of_alphabet {
A a_specific = 4;
B b_specific = 5;
}
}
This way, there's no ambiguity: you just parse the same proto (Parent
) every time.
Anyway, if it's too late to change that, what I recommend you do is define a new message with only the shared fields, like
message Shared {
required int32 FormatVersion = 1;
optional bool SomeFlag [default = true] = 2;
optional Common CommonSettings = 3;
}
You should then be able to pretend that the message (either A
or B
) is in fact a Shared
, and parse it accordingly. The unknown fields will be irrelevant.
djf
Updated on June 05, 2022Comments
-
djf over 1 year
Let's say I have 2 Protobuf-Messages, A and B. Their overall structure is similar, but not identical. So we moved the shared stuff out into a separate message we called Common. This works beautifully.
However, I'm now facing the following problem: A special case exists where I have to process a serialized message, but I don't know whether it's a message of type A or type B. I have a working solution in C++ (shown below), but I failed to find a way to do the same thing in Python.
Example:
// file: Common.proto // contains some kind of shared struct that is used by all messages: message Common { ... } // file: A.proto import "Common.proto"; message A { required int32 FormatVersion = 1; optional bool SomeFlag [default = true] = 2; optional Common CommonSettings = 3; ... A-specific Fields ... } // file: B.proto import "Common.proto"; message B { required int32 FormatVersion = 1; optional bool SomeFlag [default = true] = 2; optional Common CommonSettings = 3; ... B-specific Fields ... }
Working Solution in C++
In C++ I'm using the reflection API to get access to the CommonSettings field like this:
namespace gp = google::protobuf; ... Common* getCommonBlock(gp::Message* paMessage) { gp::Message* paMessage = new gp::Message(); gp::FieldDescriptor* paFieldDescriptor = paMessage->GetDescriptor()->FindFieldByNumber(3); gp::Reflection* paReflection = paMessage->GetReflection(); return dynamic_cast<Common&>(paReflection->GetMessage(*paMessage,paFieldDescriptor)); }
The method 'getCommonBlock' uses FindFieldByNumber() to get hold of the descriptor of the field I'm trying to get. Then it uses reflection to fetch the actual data. getCommonBlock can process messages of type A, B or any future type as long as the Common field remains located at index 3.
My Question is: Is there a way to do a similar thing Python? I've been looking at the Protobuf documentation, but couldn't figure out a way to do it.
-
djf over 10 yearsThat would work, if I had an instance of obj. Maybe I should clarify my problem a little: I receive the message - in typical protobuf fashion - as a serialized blob (binary memory stream). How do I instantiate obj from that without knowing the underlying message type?
-
deddebme almost 9 yearsI see "Type" within message A and B. Don't you suppose to put in "Base" in those places?
-
kraptor almost 9 yearsIf you imported correctly, TYPE is an enum defined in the protobuf file so you are able to call it directly.
-
Kartik Ayyar almost 4 yearsAs noted in another answer, use oneof, as that natively at the protobuf level does something similar for you. You can query the type of the oneof field using APIs in different languages.