v1.12.0

Compare Source

This change includes a new feature related to source code information for compiled-in proto descriptors. It also includes some other bug fixes.

"github.com/jhump/protoreflect/desc"

Changes/fixes:

• The FindSymbol method of FileDescriptor would panic if given an empty string as the symbol name. This has been fixed. This case now returns a nil descriptor, indicating no element with the given name could be found.
• The various Load* functions will return descriptors that include comment information (and the underlying FileDescriptorProto will contain source code info) if the protoc-gen-gosrcinfo plugin was used to generate additional Go code for these proto files.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• If an option included a message type, and was printed as a compact message literal, it had an errant extra space before the final closing '}' symbol. This is fixed, and these are now printed with a single space between the last field value and the closing brace.

"github.com/jhump/protoreflect/desc/sourceinfo"

This is a brand new package which allows for registering source code information for a proto source file. It also includes some API for using/querying the source code info. This is useful to retain comments for compiled and linked-in descriptors. It is not expected that users would explicitly register source code info; instead use the protoc-gen-gosrcinfo plugin, in addition to protoc-gen-go, when compiling your proto sources.

"github.com/jhump/protoreflect/desc/sourceinfo/cmd/protoc-gen-gosrcinfo"

This is a brand new command which is a protoc plugin for augmenting compiled and linked-in descriptors with source code information. When used, the desc package can include source code info (and thus comments) when loading linked-in descriptors. The desc/sourceinfo package can be used to query for the source code info and can be useful for integrating this data with other things, such as gRPC server reflection.

v1.11.0

Compare Source

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• If an option on a method included comments and was the kind of option that protoprint is able to handle, they would fail to be included in the printed output. Options were supported on all other types, just not on methods. This has been remedied: methods now have the same support for options comments as other elements.
• The Printer type includes three new fields, to control formatting of complex options: ShortOptionsExpansionThresholdCount and ShortOptionsExpansionThresholdLength control when "short options" will be expanded to multiple lines, based on the number of options and the length of the options when rendered; MessageLiteralExpansionThresholdLength controls when message literals in option values will be expanded to multi-line form, based on if the length of the rendered string is too long.

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

• The LoadServiceDescriptors function now accepts an interface, not just the concrete type *grpc.Server. Since *grpc.Server implements the interface, this change should be mostly backwards compatible. However, if there are usages that rely on the precise signature, such as assigning to a function variable whose signature requires *grpc.Server, this is a minor breaking change. Such usage is not expected.

v1.10.3

Compare Source

This release contains several fixes to the desc/protoparse package and one fix to the grpcreflect package.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• If a custom option value contains a message literal, protoc accepts identifiers t, f, True, and False as synonyms for true and false. Use of these alternate spellings would be rejected by this package. This is now fixed so that this package accepts the same values as protoc.
• If a custom option refers to an enum which has values named true or false, this package would reject the option, misunderstanding the true or false identifier to be a boolean literal instead of an enum value name. This has been fixed.
• There were several cases where references to a custom option or extension, in an option name or in a message literal, were resolved differently by this package than by protoc. This lead to some variances -- some source files that protoc would accept but that this package would not (because it was unable to resolve a reference), and some source files that this package would accept but that protoc would reject (because this package was using different lexical scoping rules during resolution). This has been fixed and this package now resolves extension names the same way as protoc.
• When specifying a custom option value for a message whose type is google.protobuf.Any, protoc supports a special syntax that makes it possible to use a simple text format for the contained message, instead of having to include a byte string representation of a marshaled/encoded value. This package did not previously implement that syntax, so would reject proto sources that used it. This has been remedied, and the special syntax is now supported by this package.
• This package would previously accept a repeated extension for a message that used message-set wire format. However, only optional extensions are allowed for such messages. This has been fixed, and proto sources that try to define such a repeated extension will be rejected.
• Extensions are not allowed to set a json_name option, however this package was accepting proto sources that did so. This has been fixed, and proto sources that define an extension with the json_name option will be rejected.

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

• In some cases, servers implementing the reflection service has been observed to incorrectly include extra file descriptors in response to a file_containing_symbol request. Also, the reflection service does not actually specify any ordering requirements for responses that choose to include more than one file. But this package mistakenly assumed an ordering (based on an older implementation of the reflection service in the official Java runtime), which could cause such cases (responses with multiple or even superfluous files) to return the incorrect file descriptor. This has been fixed. Now all responses to file_containing_symbol, file_containing_extension and file_by_filename requests correctly support multiple files (even superfluous ones) in any order.

v1.10.2

Compare Source

This release contains several fixes to the desc/protoparse and dynamic packages.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• If a custom option has a type with a oneof in it, then option values in proto sources should not be allowed to set more than one of the fields. Previously, this package would accept such sources, and only the last field (in source location order) would end up being set in the option value. Now the package will report an error.
• This package would incorrectly accept numeric literals in formats that the Go language accepts but that the Protocol Buffer does not. For example: "0b0100", "0o1735", and "1_000_000" are not legal integer literals in proto source, but are legal Go integer literals and were incorrectly accepted by this package. Similarly, Go accepts "1_000.314" as a float literal and so did this package. But this is not allowed in proto source. These issues have been fixed and this package now accepts numeric literals using the same rules as protoc.
• This package would report an error if a numeric literal over-flowed what can be represented by a 64-bit floating point value (IEEE754 double precision). However, protoc accepts such sources and, like other floating point literals that cannot be perfectly represented, picks the nearest/best option. In the case of overflow, that value is infinity. This package now behaves like protoc and substitutes an infinite value for numeric literals that would otherwise overflow a 64-bit floating point value.
• A NUL character in a source file (code point zero, 0x00) could be misinterpreted as end-of-file. This character is not allowed in proto sources, but would not only be accepted in some cases but also result in incomplete compilation. Similarly, this package accepted such characters inside of comments, but that is also not allowed. This has been remedied and source files with such characters will be rejected.
• This package would incorrectly allow an integer literal like "09", treating it as decimal. However, in proto sources, an integer literal that starts with a zero must be octal (in the given example, "9" is out of range and should have resulted in error). This is fixed.
• This package would allow reserved ranges and extension ranges to start at zero. However one is lowest acceptable value for such ranges. This has been fixed.
• This package would incorrectly allow two different oneofs in the same message to have the same name. This has been fixed.
• This package would allow proto source files that try to directly import the same file more than once. While it is allowed for the transitive dependency graph of a file to include a file more than once, a single proto source file cannot reference the same file multiple times. This is fixed and now reported as an error.
• Custom option values that used the message literal syntax were required to use a colon separator between a repeated field name and an array literal (e.g. multiple values inside of [ and ]). However, protoc does not require the colon in this context. This package has been updated to match the behavior of protoc.
• Parsing a source file could lead to a runtime panic under some conditions of symbol collisions. For example, if an operation used a built-in/well-known import (such as google/protobuf/descriptor.proto) without supplying its own copy of source and then included a source file that defined a symbol with the same fully-qualified name as an element in that import, a panic would occur. This has been fixed and the conflict is reported as an error as expected.
• This package previously populated trailing comments differently from protoc for block elements (i.e. those with elements defined inside of brances { }, including messages, groups, oneofs, extend blocks, enums, services, and methods). This package used the comment immediately following the closing brace (}) as the element's trailing comment. However, protoc uses the comment immediately following the opening brace ({) as the trailing comment. The behavior in this package has been changed to match that of protoc.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• This package previously assumed that trailing comments for block elements (i.e. those with elements defined inside of brances { }, including messages, groups, oneofs, extend blocks, enums, services, and methods) should be rendered after the closing brace. However, this is not how trailing comments are parsed by protoc: they are the comment immediately following the opening brace instead. So this package has been updated to render the trailing comment for these elements after the opening brace.

"github.com/jhump/protoreflect/dynamic"

Changes/fixes:

• When unmarshaling JSON into a dynamic message, where one of the fields is of type google.protobuf.Value, this package would incorrectly process values that were JSON arrays. This should be allowed and result in the field's list_value field being set with the contents of the array. Instead, the field would be set to a single value equal to the last value in the JSON array. This has been fixed.
• When marshaling a dynamic message to JSON, if a field used a custom JSON name that included a character that was non-printable or outside of the 7-bit range (e.g. a control character, code points 0-31, or any code point >= 127), the result JSON output would be invalid. The characters would be encoded in a way such that the resulting key was not a valid JSON string. This has been fixed.

v1.10.1

Compare Source

This release contains some fixes to the protoparse package.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• Use of "groups" (an old feature only supported in syntax proto2) in custom options did not work correctly. The parser would incorrectly reject use of groups in some cases. And when using message literals as option values, it did not accept the same input as protoc with regards to the field name used to refer to a group value. These issues have been corrected and behavior is not consistent with protoc.

v1.10.0

Compare Source

This release contains some improvements to the protoparse package and new functionality in the protoprint package.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• A couple of cases were identified where the protoc compiler would reject proto source, but the protoparse package would accept it:
  • The way protoc resolved relative names to fully-qualified names, for the request and response types in a method signature, differed slightly from protoparse. The protoparse package might accept a source program, but protoc might reject it due resolving a type name to something other than a type (such as a service or method name). This package now matches the behavior of protoc and will reject the same kinds of source.
  • The protoparse would accept a program with a range like 1to max, for example, whereas protoc requires a space between the 1 and the keyword to. This was due to a small difference in how tokens are lexed. The lexer in this package has been updated to now match the behavior of protoc and will now issue an error if there is no space between a numeric literal and subsequent identifier/keyword.
• The protoc compiler accepts '\v' (vertical tab) and '\f' (form feed) characters as whitespace, but protoparse would not. This is now fixed.
• The protoparse package could panic (with type assertion failures or with index-out-of-range failures) if the input contained a unicode code point in a particular range (well outside the range of allowed characters in a proto source file, other than inside string literals). This has been fixed.

"github.com/jhump/protoreflect/desc/protoprint"

Additions:

• A new CustomSortFunction field has been added to the Printer type. This can be used to define a custom order in which elements are printed to a file. This also includes a new interface type, protoprint.Element, which custom sort functions can use to inspect the elements for deciding how to order them.

v1.9.0

Compare Source

This release contains numerous improvements to the protoparse package, to more closely match protoc in terms of proto source files that are acceptable. It also contains some fixes in other packages.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• When adding a message to another (to make a nested/enclosed type), the target enclosing message could be incorrectly detached from its parent element. This was the result of a typo in the implementation code and has been fixed.

"github.com/jhump/protoreflect/desc/protoparse"

Additions:

• The protoparse package now issues warnings when it detects that a source file has unused imports. This mirrors the warnings that protoc issues in the same cases. This feature requires the use of a WarningReporter with a parse operation. The concrete type of value provided to the warning reporter will be a protoparse.ErrorUnusedImport.

Changes/fixes:

• The protoc compiler was more strict than protoparse when it comes to resolving relative (vs. full qualified) names. This led to conditions where protoparse would accept a proto source file that protoc would reject. The issue is when the first component of an identifier could match multiple lexical scopes. In such a case protoc only matches the most enclosing scope. But protoparse would fallback to other enclosing scopes if the most enclosing scope could not be used to resolve a symbol. (Hard to describe succinctly, so see the example in this bug report.) This issue is now fixed and protoparse resolves names in the same manner as protoc.
• The protoc compiler uses "C++ enum scoping rules" for protobuf enums. This means that enum values are declared in the namespace of the enclosing enum (as siblings of the enum itself). But protoparse incorrectly treated the enum as the parent scope/namespace. This led to source files that protoparse would accept but that protoc would reject. This issue is now fixed.
• The use of custom options in oneof statements could incorrectly result in error messages about failing to resolve the custom option name, even if the source file and the option reference were valid. This has been fixed.

"github.com/jhump/protoreflect/dynamic/msgregistry"

Additions:

• A new error type, ErrUnexpectedType, was introduced. When a call to FindMessageTypeByUrl or FindEnumTypeByUrl fails because of a type mismatch (expecting a message, got an enum, or vice versa), this can now be determined programmatically by type-asserting the error to the new error type. This provides a proper/robust way to detect this kind of error (previously, callers would have to examine the error text, which is quite brittle).