posts-go/docs/2022-07-19-migrate-to-buf.html

135 lines
6.6 KiB
HTML

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link
rel="stylesheet"
href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.1.0/github-markdown.min.css"
/>
</head>
<style>
/* https://github.com/sindresorhus/github-markdown-css */
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
@media (max-width: 767px) {
.markdown-body {
padding: 15px;
}
}
</style>
<body class="markdown-body">
<a href="index">Index</a>
<h1><a id="user-content-migrate-to-buf-from-prototool" class="anchor" aria-hidden="true" href="#migrate-to-buf-from-prototool"><span aria-hidden="true" class="octicon octicon-link"></span></a>Migrate to <code>buf</code> from <code>prototool</code>
</h1>
<p>Why? Because <code>prototool</code> is outdated, and can not run on M1 mac.</p>
<p>We need 3 files:</p>
<ul>
<li>
<code>build.go</code>: need to install protoc-gen-* binaries with pin version in <code>go.mod</code>
</li>
<li><code>buf.yaml</code></li>
<li><code>buf.gen.yaml</code></li>
</ul>
<p>FYI, the libs version I use:</p>
<ul>
<li><a href="https://github.com/golang/protobuf/releases/tag/v1.5.2">golang/protobuf v1.5.2</a></li>
<li><a href="https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0">grpc-ecosystem/grpc-gateway v1.16.0</a></li>
<li><a href="github.com/envoyproxy/protoc-gen-validate">envoyproxy/protoc-gen-validate</a></li>
<li><a href="github.com/kei2100/protoc-gen-marshal-zap">kei2100/protoc-gen-marshal-zap</a></li>
</ul>
<p><code>build.go</code>:</p>
<div class="highlight highlight-source-go"><pre><span class="pl-c">//go:build tools</span>
<span class="pl-c">// +build tools</span>
<span class="pl-k">import</span> (
_ <span class="pl-s">"github.com/envoyproxy/protoc-gen-validate"</span>
_ <span class="pl-s">"github.com/golang/protobuf/protoc-gen-go"</span>
_ <span class="pl-s">"github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway"</span>
_ <span class="pl-s">"github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger"</span>
_ <span class="pl-s">"github.com/kei2100/protoc-gen-marshal-zap/plugin/protoc-gen-marshal-zap"</span>
)</pre></div>
<p><code>buf.yaml</code></p>
<div class="highlight highlight-source-yaml"><pre><span class="pl-ent">version</span>: <span class="pl-c1">v1</span>
<span class="pl-ent">deps</span>:
- <span class="pl-s">buf.build/haunt98/googleapis:b38d93f7ade94a698adff9576474ae7c</span>
- <span class="pl-s">buf.build/haunt98/grpc-gateway:ecf4f0f58aa8496f8a76ed303c6e06c7</span>
- <span class="pl-s">buf.build/haunt98/protoc-gen-validate:2686264610fc4ad4a9fcc932647e279d</span>
- <span class="pl-s">buf.build/haunt98/marshal-zap:2a593ca925134680a5820d3f13c1be5a</span>
<span class="pl-ent">breaking</span>:
<span class="pl-ent">use</span>:
- <span class="pl-s">FILE</span>
<span class="pl-ent">lint</span>:
<span class="pl-ent">use</span>:
- <span class="pl-s">DEFAULT</span></pre></div>
<p><code>buf.gen.yaml</code>:</p>
<div class="highlight highlight-source-yaml"><pre><span class="pl-ent">version</span>: <span class="pl-c1">v1</span>
<span class="pl-ent">plugins</span>:
- <span class="pl-ent">name</span>: <span class="pl-s">go</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
<span class="pl-ent">opt</span>:
- <span class="pl-s">plugins=grpc</span>
- <span class="pl-ent">name</span>: <span class="pl-s">grpc-gateway</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
<span class="pl-ent">opt</span>:
- <span class="pl-s">logtostderr=true</span>
- <span class="pl-ent">name</span>: <span class="pl-s">swagger</span>
<span class="pl-ent">out</span>: <span class="pl-s">.</span>
<span class="pl-ent">opt</span>:
- <span class="pl-s">logtostderr=true</span>
- <span class="pl-ent">name</span>: <span class="pl-s">validate</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
<span class="pl-ent">opt</span>:
- <span class="pl-s">lang=go</span>
- <span class="pl-ent">name</span>: <span class="pl-s">marshal-zap</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span></pre></div>
<p>Update <code>Makefile</code>:</p>
<div class="highlight highlight-source-makefile"><pre><span class="pl-en">gen</span>:
go install github.com/golang/protobuf/protoc-gen-go
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
go install github.com/envoyproxy/protoc-gen-validate
go install github.com/kei2100/protoc-gen-marshal-zap/plugin/protoc-gen-marshal-zap
go install github.com/bufbuild/buf/cmd/buf@latest
buf mod update
buf format -w
buf generate</pre></div>
<p>Run <code>make gen</code> to have fun of course.</p>
<h2><a id="user-content-faq" class="anchor" aria-hidden="true" href="#faq"><span aria-hidden="true" class="octicon octicon-link"></span></a>FAQ</h2>
<p>Remember <code>grpc-ecosystem/grpc-gateway</code>, <code>envoyproxy/protoc-gen-validate</code>, <code>kei2100/protoc-gen-marshal-zap</code> is optional, so feel free to delete if you don't use theme.</p>
<p>If use <code>vendor</code>:</p>
<ul>
<li>Replace <code>buf generate</code> with <code>buf generate --exclude-path vendor</code>.</li>
<li>Replace <code>buf format -w</code> with <code>buf format -w --exclude-path vendor</code>.</li>
</ul>
<p>If you use grpc-gateway:</p>
<ul>
<li>Replace <code>import "third_party/googleapis/google/api/annotations.proto";</code> with <code>import "google/api/annotations.proto";</code>
</li>
<li>Delete <code>security_definitions</code>, <code>security</code>, in <code>option (grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger)</code>.</li>
</ul>
<p>The last step is delete <code>prototool.yaml</code>.</p>
<p>If you are not migrate but start from scratch:</p>
<ul>
<li>Add <code>buf lint</code> to make sure your proto is good.</li>
<li>Add <code>buf breaking --against "https://your-grpc-repo-goes-here.git"</code> to make sure each time you update proto, you don't break backward compatibility.</li>
</ul>
<h2><a id="user-content-thanks" class="anchor" aria-hidden="true" href="#thanks"><span aria-hidden="true" class="octicon octicon-link"></span></a>Thanks</h2>
<ul>
<li><a href="https://github.com/uber/prototool">uber/prototool</a></li>
<li><a href="https://github.com/bufbuild/buf">bufbuild/buf</a></li>
</ul>
<a href="mailto:hauvipapro+posts@gmail.com"
>Feel free to ask me via email</a
>
<a rel="me" href="https://hachyderm.io/@haunguyen">Mastodon</a>
</body>
</html>