Support gRPC
complete
Matt Fellows
gRPC is a common microservice framework. Commonly used with Protobufs as the encoding, and HTTP/2 as the transport, it is a highly efficient and type safe architecture.
It is often said that contract testing is not required, due to forwards/backwards compatibility encoded into the schemas.
Arguments for gRPC and Protobuf support in Pact
* > "You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code."
* Whilst this won’t break the “contract”, it may actually not be a plausible situation. There are no guarantees that the actual
RPC service will still work as expected
- The protocol definition itself doesn’t guarantee it can handle all situations the consumers expect to use:
* Proto 3 removes “mandatory fields” - "Making every field optional provides a clearer contract to clients. They are explicitly responsible for checking that every field has been populated with something valid."
* This means specification examples (ala Pact) are very important to ensuring the functionally contract behaviour
* Similar issues to the challenges of "Optional" or "Any" schemas in SOAP SOA architectures
- Backwards guarantee doesn’t tell you _forwards_ compatibility i.e. it doesn’t help you coordinate a release
- OneOf semantics - see https://developers.google.com/protocol-buffers/docs/proto3#backwards-compatibility-issues
- The protocol buffer is separate to the HTTP endpoint serving it. See value prop from above
- It is absolutely possible to break a proto file by modifying numbered tags (field identifiers) or removing fields
Related Resources
* Buf (https://buf.build/) - a useful tool for static protobuf linting, backwards compatiblity checks and introspection
Matt Fellows
complete
The plugin was created and released last year, and after a period of stabilisation I think we can consider this closed: https://github.com/pactflow/pact-protobuf-plugin/
Matt Fellows
The second developer preview of our gRPC/Protobuf support for Pact is now available. Expanding our previous support for Protobufs, this update brings support for using gRPC directly with Pact tests, using a gRPC mock server in consumer tests and the Pact verifier supporting gRPC
for provider verification tests.
Documentation can be found at https://github.com/pactflow/pact-protobuf-plugin#testing-a-grpc-service-method-interaction
gRPC is supported with:
- Pact-JVM 4.4.0-beta.1 and JUnit 5 (https://github.com/pact-foundation/pact-jvm/tree/master/consumer/junit5 and https://github.com/pact-foundation/pact-jvm/tree/master/provider/junit5)
- Pact-Rust consumer crate 0.9.2 (https://crates.io/crates/pact_consumer)
- Pact Verifier CLI 0.9.10 (https://github.com/pact-foundation/pact-reference/releases/tag/pact_verifier_cli-v0.9.10)The Pactflow gRPC/Protobuf plugin can be downloaded from: https://github.com/pactflow/pact-protobuf-plugin/releases
For examples of Pact tests using gRPC:
- https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/consumer-jvm
- https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/consumer-rust
- https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/provider-jvm
What is coming next?
- We want to bring gRPC support to more languages (see https://github.com/pact-foundation/pact-plugins/projects/1).
- We will be updating the Pact Rust FFI with support for plugins, and then updating Pact-Go and Pact-JS with gRPC support.
- We'd welcome a discussion in the #protobufs channel in Slack
Matt Fellows
Blog post released today that summarises our thinking of the deficiencies in protobufs (and similar technologies) and how contract testing can help. Also, a small teaser on the current prototype... https://pactflow.io/blog/the-case-for-contract-testing-protobufs-grpc-avro/
Matt Fellows
in progress
Follow progress here: https://github.com/pact-foundation/pact-plugins/projects/1
Matt Fellows
Just wanted to put a note here about progress on gRPC/Protobufs. My current thinking is that getting it into the core framework via a plugin infrastructure is the path forward, and the proposal for that has been raised here: https://github.com/pact-foundation/pact-specification/issues/83.
R
Ringo De Smet
gRPC and protobuf is probably the first case of supporting other protocols. I assume this asks for a big redesign of Pact and it could be good to think how to support more protocols in the future.
Where I work, we are evaluating various service virtualization tools. Pact is on the list due to extensive language support, but even more because of the broker existence.
Regarding support for multiple protocols, Mountebank is taking a keen approach: http://www.mbtest.org/docs/protocols/custom
You might "borrow" some concepts to get this in Pact as well. ;-)
Our needs:
- AMQP: verifying the message structure used between services
- Redis RESP protocol: to get rid of test data management :-)
Matt Fellows
Ringo De Smet: That MB test looks very similar to at least one prototype that has been done - thanks for sharing!! This could be great inspiration for us.
As for AMQP/Redis, you may be able to do that testing now (except without the SV capabilities that you might want). The "Message Pact" feature of a number of languages (v3 spec) supports this sort of testing now. Jump in to slack.pact.io/https://pact-foundation.slack.com/ and we can chat there.
C
Cody A. Ray
I'm working on providing API evolution and compatibility guidelines to my internal teams now, which use protobuf. I'm finding that a lot of these claims by protobuf are more-or-less irrelevant... particularly the automatic "backward and forward compatibility".
This may true at the wire level but its certainly not true from an application level.
* If you make all fields optional in the spec, service providers just check if they're present and return an error. If you don't say that
string email = 2
must be a valid email in the spec but your server does, again, you get an error. So now we just have a bunch of unspecified behavior implemented by the server but not recorded in the spec. Now you have to have real smoke test to validate breaking changes, because linting the spec (ala uber/prototool) is insufficient.* If you use certain things like google's Any type, even moving a file such that it has a different namespace will break your.
* If you treat the json/yaml encodings as first-class citizens, then a lot more changes including field renames will break your clients.
I'm not sure if pact is the right way forward yet for grpc, or consumer contracts in general. But I'm very interested. :)
Useful resources:
Matt Fellows
planned
Design for this feature is underway.
C
Cody A. Ray
Matt Fellows: Any updates on this feature?
dev-rgupta
Matt Fellows: any update on this feature