2023-07-18 18:38:30 +00:00
|
|
|
<!doctype html>
|
2022-12-25 19:13:58 +00:00
|
|
|
<html>
|
|
|
|
|
<head>
|
|
|
|
|
<meta charset="utf-8" />
|
|
|
|
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
|
|
|
<link
|
|
|
|
|
rel="stylesheet"
|
2023-08-06 07:41:38 +00:00
|
|
|
href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.2.0/github-markdown.min.css"
|
2022-12-25 19:13:58 +00:00
|
|
|
/>
|
2023-04-01 03:37:48 +00:00
|
|
|
<title>haunt98 posts</title>
|
2022-12-25 19:13:58 +00:00
|
|
|
</head>
|
|
|
|
|
<style>
|
|
|
|
|
.markdown-body {
|
|
|
|
|
box-sizing: border-box;
|
|
|
|
|
min-width: 200px;
|
|
|
|
|
max-width: 980px;
|
|
|
|
|
margin: 0 auto;
|
|
|
|
|
padding: 45px;
|
|
|
|
|
}
|
2022-07-19 08:55:18 +00:00
|
|
|
|
2022-12-25 19:13:58 +00:00
|
|
|
@media (max-width: 767px) {
|
|
|
|
|
.markdown-body {
|
|
|
|
|
padding: 15px;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
</style>
|
|
|
|
|
<body class="markdown-body">
|
2023-08-09 07:22:58 +00:00
|
|
|
<h2>
|
|
|
|
|
<a href="index.html"><code>~</code></a>
|
|
|
|
|
</h2>
|
2022-12-30 09:26:06 +00:00
|
|
|
<h1>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-migrate-to-buf-from-prototool"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
2023-08-30 03:28:39 +00:00
|
|
|
tabindex="-1"
|
2022-12-30 09:26:06 +00:00
|
|
|
href="#migrate-to-buf-from-prototool"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Migrate to <code>buf</code> from <code>prototool</code>
|
|
|
|
|
</h1>
|
|
|
|
|
<p>
|
|
|
|
|
Why? Because <code>prototool</code> is outdated, and can not run on M1
|
|
|
|
|
mac.
|
|
|
|
|
</p>
|
|
|
|
|
<p>We need 3 files:</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
<code>build.go</code>: need to install protoc-gen-* binaries with pin
|
2023-08-05 19:13:47 +00:00
|
|
|
version in
|
|
|
|
|
<code>go.mod</code>
|
2022-12-30 09:26:06 +00:00
|
|
|
</li>
|
|
|
|
|
<li><code>buf.yaml</code></li>
|
|
|
|
|
<li><code>buf.gen.yaml</code></li>
|
|
|
|
|
</ul>
|
2023-06-11 11:02:42 +00:00
|
|
|
<p>FYI, I use:</p>
|
2022-12-30 09:26:06 +00:00
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.16.0"
|
|
|
|
|
>grpc-ecosystem/grpc-gateway v1.16.0</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
2023-03-20 21:00:21 +00:00
|
|
|
<a href="github.com/bufbuild/protoc-gen-validate"
|
|
|
|
|
>bufbuild/protoc-gen-validate</a
|
2022-12-30 09:26:06 +00:00
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<a href="github.com/kei2100/protoc-gen-marshal-zap"
|
|
|
|
|
>kei2100/protoc-gen-marshal-zap</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p><code>build.go</code>:</p>
|
|
|
|
|
<div class="highlight highlight-source-go">
|
|
|
|
|
<pre><span class="pl-c">//go:build tools</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-c">// +build tools</span>
|
|
|
|
|
|
|
|
|
|
<span class="pl-k">import</span> (
|
|
|
|
|
_ <span class="pl-s">"github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway"</span>
|
|
|
|
|
_ <span class="pl-s">"github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger"</span>
|
|
|
|
|
_ <span class="pl-s">"github.com/kei2100/protoc-gen-marshal-zap/plugin/protoc-gen-marshal-zap"</span>
|
2022-12-30 09:26:06 +00:00
|
|
|
)</pre>
|
|
|
|
|
</div>
|
|
|
|
|
<p><code>buf.yaml</code></p>
|
|
|
|
|
<div class="highlight highlight-source-yaml">
|
|
|
|
|
<pre><span class="pl-ent">version</span>: <span class="pl-c1">v1</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-ent">deps</span>:
|
2023-03-20 21:00:21 +00:00
|
|
|
- <span class="pl-s">buf.build/envoyproxy/protoc-gen-validate:6607b10f00ed4a3d98f906807131c44a</span>
|
|
|
|
|
- <span class="pl-s">buf.build/kei2100/protoc-gen-marshal-zap:081f499bbca4486784773e060c1c1418</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
- <span class="pl-s">buf.build/haunt98/googleapis:b38d93f7ade94a698adff9576474ae7c</span>
|
|
|
|
|
- <span class="pl-s">buf.build/haunt98/grpc-gateway:ecf4f0f58aa8496f8a76ed303c6e06c7</span>
|
|
|
|
|
<span class="pl-ent">breaking</span>:
|
|
|
|
|
<span class="pl-ent">use</span>:
|
2023-06-10 04:13:37 +00:00
|
|
|
- <span class="pl-s">PACKAGE</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-ent">lint</span>:
|
|
|
|
|
<span class="pl-ent">use</span>:
|
2022-12-30 09:26:06 +00:00
|
|
|
- <span class="pl-s">DEFAULT</span></pre>
|
|
|
|
|
</div>
|
|
|
|
|
<p><code>buf.gen.yaml</code>:</p>
|
|
|
|
|
<div class="highlight highlight-source-yaml">
|
|
|
|
|
<pre><span class="pl-ent">version</span>: <span class="pl-c1">v1</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-ent">plugins</span>:
|
2023-06-11 11:02:42 +00:00
|
|
|
- <span class="pl-ent">plugin</span>: <span class="pl-s">buf.build/grpc/go:v1.3.0</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
|
2023-06-30 08:46:42 +00:00
|
|
|
- <span class="pl-ent">plugin</span>: <span class="pl-s">buf.build/protocolbuffers/go:v1.31.0</span>
|
2023-06-10 04:11:44 +00:00
|
|
|
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
|
2023-06-30 08:46:42 +00:00
|
|
|
- <span class="pl-ent">plugin</span>: <span class="pl-s">buf.build/bufbuild/validate-go:v1.0.2</span>
|
2023-03-20 21:00:21 +00:00
|
|
|
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
|
|
|
|
|
<span class="pl-ent">opt</span>:
|
|
|
|
|
- <span class="pl-s">lang=go</span>
|
2023-06-10 04:11:44 +00:00
|
|
|
- <span class="pl-ent">plugin</span>: <span class="pl-s">marshal-zap</span>
|
2023-03-20 21:00:21 +00:00
|
|
|
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
|
2023-06-10 04:11:44 +00:00
|
|
|
- <span class="pl-ent">plugin</span>: <span class="pl-s">grpc-gateway</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-ent">out</span>: <span class="pl-s">pkg</span>
|
|
|
|
|
<span class="pl-ent">opt</span>:
|
|
|
|
|
- <span class="pl-s">logtostderr=true</span>
|
2023-06-10 04:11:44 +00:00
|
|
|
- <span class="pl-ent">plugin</span>: <span class="pl-s">swagger</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-ent">out</span>: <span class="pl-s">.</span>
|
|
|
|
|
<span class="pl-ent">opt</span>:
|
2023-03-20 21:00:21 +00:00
|
|
|
- <span class="pl-s">logtostderr=true</span></pre>
|
2022-12-30 09:26:06 +00:00
|
|
|
</div>
|
|
|
|
|
<p>Update <code>Makefile</code>:</p>
|
|
|
|
|
<div class="highlight highlight-source-makefile">
|
|
|
|
|
<pre><span class="pl-en">gen</span>:
|
2023-03-20 21:00:21 +00:00
|
|
|
go install github.com/kei2100/protoc-gen-marshal-zap/plugin/protoc-gen-marshal-zap
|
2022-10-09 11:03:05 +00:00
|
|
|
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
|
|
|
|
|
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
|
|
|
|
|
go install github.com/bufbuild/buf/cmd/buf@latest
|
|
|
|
|
buf mod update
|
|
|
|
|
buf format -w
|
2022-12-30 09:26:06 +00:00
|
|
|
buf generate</pre>
|
|
|
|
|
</div>
|
|
|
|
|
<p>Run <code>make gen</code> to have fun of course.</p>
|
2023-03-20 21:00:21 +00:00
|
|
|
<p>
|
|
|
|
|
If using <code>bufbuild/protoc-gen-validate</code>,
|
|
|
|
|
<code>kei2100/protoc-gen-marshal-zap</code>, better make a raw copy of
|
|
|
|
|
proto file for other services to integrate:
|
|
|
|
|
</p>
|
|
|
|
|
<div class="highlight highlight-source-makefile">
|
|
|
|
|
<pre><span class="pl-en">raw</span>:
|
2023-06-10 04:11:44 +00:00
|
|
|
mkdir -p ./raw
|
2023-03-20 21:08:31 +00:00
|
|
|
cp ./api.proto ./raw/
|
|
|
|
|
sed -i "" -e "s/import \"validate\/validate\.proto\";//g" ./raw/api.proto
|
|
|
|
|
sed -i "" -e "s/\[(validate\.rules)\.string.min_len = 1\]//g" ./raw/api.proto
|
|
|
|
|
sed -i "" -e "s/import \"marshal-zap\.proto\";//g" ./raw/api.proto
|
|
|
|
|
sed -i "" -e "s/\[(marshal_zap\.mask) = true]//g" ./raw/api.proto</pre>
|
2023-03-20 21:00:21 +00:00
|
|
|
</div>
|
2022-12-30 09:26:06 +00:00
|
|
|
<h2>
|
2023-08-30 03:28:39 +00:00
|
|
|
<a
|
|
|
|
|
id="user-content-faq"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
tabindex="-1"
|
|
|
|
|
href="#faq"
|
2022-12-30 09:26:06 +00:00
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>FAQ
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
2023-03-20 21:00:21 +00:00
|
|
|
Remember <code>bufbuild/protoc-gen-validate</code>,
|
|
|
|
|
<code>kei2100/protoc-gen-marshal-zap</code>,
|
|
|
|
|
<code>grpc-ecosystem/grpc-gateway</code> is optional, so feel free to
|
2022-12-30 09:26:06 +00:00
|
|
|
delete if you don't use theme.
|
|
|
|
|
</p>
|
|
|
|
|
<p>If use <code>vendor</code>:</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
Replace <code>buf generate</code> with
|
|
|
|
|
<code>buf generate --exclude-path vendor</code>.
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
Replace <code>buf format -w</code> with
|
|
|
|
|
<code>buf format -w --exclude-path vendor</code>.
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>If you use grpc-gateway:</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
Replace
|
|
|
|
|
<code
|
|
|
|
|
>import "third_party/googleapis/google/api/annotations.proto";</code
|
|
|
|
|
>
|
2023-08-05 19:13:47 +00:00
|
|
|
with
|
|
|
|
|
<code>import "google/api/annotations.proto";</code>
|
2022-12-30 09:26:06 +00:00
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
Delete <code>security_definitions</code>, <code>security</code>, in
|
|
|
|
|
<code
|
|
|
|
|
>option
|
|
|
|
|
(grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger)</code
|
|
|
|
|
>.
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>The last step is delete <code>prototool.yaml</code>.</p>
|
|
|
|
|
<p>If you are not migrate but start from scratch:</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>Add <code>buf lint</code> to make sure your proto is good.</li>
|
|
|
|
|
<li>
|
|
|
|
|
Add
|
|
|
|
|
<code
|
|
|
|
|
>buf breaking --against "https://your-grpc-repo-goes-here.git"</code
|
|
|
|
|
>
|
|
|
|
|
to make sure each time you update proto, you don't break backward
|
|
|
|
|
compatibility.
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
2023-06-11 11:02:42 +00:00
|
|
|
<h1>
|
2023-08-30 03:28:39 +00:00
|
|
|
<a
|
|
|
|
|
id="user-content-tips"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
tabindex="-1"
|
|
|
|
|
href="#tips"
|
2023-06-11 11:02:42 +00:00
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Tips
|
|
|
|
|
</h1>
|
|
|
|
|
<p>Some experience I got after writing proto files for a living:</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
Ignore DRY (Do not Repeat Yourself) when handling proto, don't split
|
|
|
|
|
proto into many files. Trust me, it saves you from wasting time to debug
|
|
|
|
|
how to import Go after generated. Because proto import and Go import is
|
|
|
|
|
<a href="https://github.com/golang/protobuf/issues/895">2</a> different
|
|
|
|
|
things. If someone already have split proto files, you should use
|
|
|
|
|
<code>sed</code> to fix the damn things.
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
2022-12-30 09:26:06 +00:00
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-thanks"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
2023-08-30 03:28:39 +00:00
|
|
|
tabindex="-1"
|
2022-12-30 09:26:06 +00:00
|
|
|
href="#thanks"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Thanks
|
|
|
|
|
</h2>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><a href="https://github.com/uber/prototool">uber/prototool</a></li>
|
|
|
|
|
<li><a href="https://github.com/bufbuild/buf">bufbuild/buf</a></li>
|
|
|
|
|
</ul>
|
2022-12-25 19:13:58 +00:00
|
|
|
|
2022-12-25 19:22:39 +00:00
|
|
|
<div>
|
|
|
|
|
Feel free to ask me via
|
|
|
|
|
<a href="mailto:hauvipapro+posts@gmail.com">email</a> or
|
2023-08-20 17:29:13 +00:00
|
|
|
<a rel="me" href="https://hachyderm.io/@haunguyen">Mastodon</a>.
|
|
|
|
|
<br />Source code is available on
|
2022-12-30 09:55:38 +00:00
|
|
|
<a href="https://github.com/haunt98/posts-go">GitHub</a>
|
2023-08-20 17:29:13 +00:00
|
|
|
<a href="https://codeberg.org/yoshie/posts-go">Codeberg</a>
|
|
|
|
|
<a href="https://git.sr.ht/~youngyoshie/posts-go">sourcehut</a>
|
|
|
|
|
<a href="https://gitea.treehouse.systems/yoshie/posts-go">Treehouse</a>
|
|
|
|
|
<a href="https://gitlab.com/youngyoshie/posts-go">GitLab</a>
|
2022-12-25 19:22:39 +00:00
|
|
|
</div>
|
2022-12-25 19:13:58 +00:00
|
|
|
</body>
|
|
|
|
|
</html>
|
