<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link
rel="stylesheet"
href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.2.0/github-markdown.min.css"
/>
<title>haunt98 posts</title>
</head>
<style>
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
@media (max-width: 767px) {
.markdown-body {
padding: 15px;
}
}
</style>
<body class="markdown-body">
<h2>
<a href="index.html"><code>~</code></a>
</h2>
<h1 id="user-content-migrate-to-buf-from-prototool">
<a class="heading-link" href="#migrate-to-buf-from-prototool"
>Migrate to <code>buf</code> from <code>prototool</code>
<span aria-hidden="true" class="octicon octicon-link"></span
></a>
</h1>
<p>
Why? Because <code>prototool</code> is outdated, and can not run on M1
mac.
</p>
<p>We need 3 files:</p>
<ul>
<li>
<code>build.go</code>: need to install protoc-gen-* binaries with pin
version in
<code>go.mod</code>
</li>
<li><code>buf.yaml</code></li>
<li><code>buf.gen.yaml</code></li>
</ul>
<p>FYI, I use:</p>
<ul>
<li>
<a
href="https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0"
>grpc-ecosystem/grpc-gateway v1.16.0</a
>
</li>
<li>
<a href="github.com/bufbuild/protoc-gen-validate"
>bufbuild/protoc-gen-validate</a
>
</li>
<li>
<a href="github.com/kei2100/protoc-gen-marshal-zap"
>kei2100/protoc-gen-marshal-zap</a
>
</li>
</ul>
<p><code>build.go</code>:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-c">//go:build tools</span>
<span class="pl-c">// +build tools</span>
<span class="pl-k">import</span> (
_ <span class="pl-s">"github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway"</span>
_ <span class="pl-s">"github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger"</span>
_ <span class="pl-s">"github.com/kei2100/protoc-gen-marshal-zap/plugin/protoc-gen-marshal-zap"</span>
)</pre>
</div>
<p><code>buf.yaml</code></p>
<div class="highlight highlight-source-yaml">
<pre><span class="pl-ent">version</span>: <span class="pl-c1">v1</span>
<span class="pl-ent">deps</span>:
- <span class="pl-s">buf.build/envoyproxy/protoc-gen-validate:6607b10f00ed4a3d98f906807131c44a</span>
- <span class="pl-s">buf.build/kei2100/protoc-gen-marshal-zap:081f499bbca4486784773e060c1c1418</span>
- <span class="pl-s">buf.build/haunt98/googleapis:b38d93f7ade94a698adff9576474ae7c</span>
- <span class="pl-s">buf.build/haunt98/grpc-gateway:ecf4f0f58aa8496f8a76ed303c6e06c7</span>
<span class="pl-ent">breaking</span>:
<span class="pl-ent">use</span>:
- <span class="pl-s">PACKAGE</span>
<span class="pl-ent">lint</span>:
<span class="pl-ent">use</span>:
- <span class="pl-s">DEFAULT</span></pre>
</div>
<p><code>buf.gen.yaml</code>:</p>
<div class="highlight highlight-source-yaml">
<pre><span class="pl-ent">version</span>: <span class="pl-c1">v1</span>
<span class="pl-ent">plugins</span>:
- <span class="pl-ent">plugin</span>: <span class="pl-s">buf.build/grpc/go:v1.3.0</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
- <span class="pl-ent">plugin</span>: <span class="pl-s">buf.build/protocolbuffers/go:v1.31.0</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
- <span class="pl-ent">plugin</span>: <span class="pl-s">buf.build/bufbuild/validate-go:v1.0.2</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
<span class="pl-ent">opt</span>:
- <span class="pl-s">lang=go</span>
- <span class="pl-ent">plugin</span>: <span class="pl-s">marshal-zap</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
- <span class="pl-ent">plugin</span>: <span class="pl-s">grpc-gateway</span>
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
<span class="pl-ent">opt</span>:
- <span class="pl-s">logtostderr=true</span>
- <span class="pl-ent">plugin</span>: <span class="pl-s">swagger</span>
<span class="pl-ent">out</span>: <span class="pl-s">.</span>
<span class="pl-ent">opt</span>:
- <span class="pl-s">logtostderr=true</span></pre>
</div>
<p>Update <code>Makefile</code>:</p>
<div class="highlight highlight-source-makefile">
<pre><span class="pl-en">gen</span>:
go install github.com/kei2100/protoc-gen-marshal-zap/plugin/protoc-gen-marshal-zap
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
go install github.com/bufbuild/buf/cmd/buf@latest
buf mod update
buf format -w
buf generate</pre>
</div>
<p>Run <code>make gen</code> to have fun of course.</p>
<p>
If using <code>bufbuild/protoc-gen-validate</code>,
<code>kei2100/protoc-gen-marshal-zap</code>, better make a raw copy of
proto file for other services to integrate:
</p>
<div class="highlight highlight-source-makefile">
<pre><span class="pl-en">raw</span>:
mkdir -p ./raw
cp ./api.proto ./raw/
sed -i "" -e "s/import \"validate\/validate\.proto\";//g" ./raw/api.proto
sed -i "" -e "s/\[(validate\.rules)\.string.min_len = 1\]//g" ./raw/api.proto
sed -i "" -e "s/import \"marshal-zap\.proto\";//g" ./raw/api.proto
sed -i "" -e "s/\[(marshal_zap\.mask) = true]//g" ./raw/api.proto</pre>
</div>
<h2 id="user-content-faq">
<a class="heading-link" href="#faq"
>FAQ<span aria-hidden="true" class="octicon octicon-link"></span
></a>
</h2>
<p>
Remember <code>bufbuild/protoc-gen-validate</code>,
<code>kei2100/protoc-gen-marshal-zap</code>,
<code>grpc-ecosystem/grpc-gateway</code> is optional, so feel free to
delete if you don't use theme.
</p>
<p>If use <code>vendor</code>:</p>
<ul>
<li>
Replace <code>buf generate</code> with
<code>buf generate --exclude-path vendor</code>.
</li>
<li>
Replace <code>buf format -w</code> with
<code>buf format -w --exclude-path vendor</code>.
</li>
</ul>
<p>If you use grpc-gateway:</p>
<ul>
<li>
Replace
<code
>import "third_party/googleapis/google/api/annotations.proto";</code
>
with
<code>import "google/api/annotations.proto";</code>
</li>
<li>
Delete <code>security_definitions</code>, <code>security</code>, in
<code
>option
(grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger)</code
>.
</li>
</ul>
<p>The last step is delete <code>prototool.yaml</code>.</p>
<p>If you are not migrate but start from scratch:</p>
<ul>
<li>Add <code>buf lint</code> to make sure your proto is good.</li>
<li>
Add
<code
>buf breaking --against "https://your-grpc-repo-goes-here.git"</code
>
to make sure each time you update proto, you don't break backward
compatibility.
</li>
</ul>
<h1 id="user-content-tips">
<a class="heading-link" href="#tips"
>Tips<span aria-hidden="true" class="octicon octicon-link"></span
></a>
</h1>
<p>Some experience I got after writing proto files for a living:</p>
<ul>
<li>
Ignore DRY (Do not Repeat Yourself) when handling proto, don't split
proto into many files. Trust me, it saves you from wasting time to debug
how to import Go after generated. Because proto import and Go import is
<a href="https://github.com/golang/protobuf/issues/895">2</a> different
things. If someone already have split proto files, you should use
<code>sed</code> to fix the damn things.
</li>
</ul>
<h2 id="user-content-thanks">
<a class="heading-link" href="#thanks"
>Thanks<span aria-hidden="true" class="octicon octicon-link"></span
></a>
</h2>
<ul>
<li><a href="https://github.com/uber/prototool">uber/prototool</a></li>
<li><a href="https://github.com/bufbuild/buf">bufbuild/buf</a></li>
</ul>
<div>
Feel free to ask me via
<a href="mailto:hauvipapro+posts@gmail.com">email</a> or
<a rel="me" href="https://hachyderm.io/@haunguyen">Mastodon</a>.
<br />Source code is available on
<a href="https://github.com/haunt98/posts-go">GitHub</a>
<a href="https://codeberg.org/yoshie/posts-go">Codeberg</a>
<a href="https://git.sr.ht/~youngyoshie/posts-go">sourcehut</a>
<a href="https://gitea.treehouse.systems/yoshie/posts-go">Treehouse</a>
<a href="https://gitlab.com/youngyoshie/posts-go">GitLab</a>
</div>
</body>
</html>