posts-go/docs/2022-07-10-bootstrap-go.html

1080 lines
43 KiB
HTML

<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link
rel="stylesheet"
href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.5.1/github-markdown.min.css"
/>
<title>haunt98 posts</title>
</head>
<style>
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
font-family:
Shantell Sans Normal,
Rec Mono Casual,
SF Pro,
Inter,
sans-serif;
font-weight: 500;
}
.markdown-body pre {
font-family:
Berkeley Mono,
IBM Plex Mono,
SF Mono,
Jetbrains Mono,
monospace;
}
@media (max-width: 767px) {
.markdown-body {
padding: 15px;
}
}
</style>
<body class="markdown-body">
<h2>
<a href="index.html"><code>~</code></a>
</h2>
<div class="markdown-heading">
<h1 class="heading-element">Bootstrap Go</h1>
<a
id="user-content-bootstrap-go"
class="anchor"
aria-label="Permalink: Bootstrap Go"
href="#bootstrap-go"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
It is hard to write bootstrap tool to quickly create Go service. So I
write this guide instead. This is a quick checklist for me every damn time
I need to write a Go service from scratch. Also, this is my personal
opinion, so feel free to comment.
</p>
<div class="markdown-heading">
<h2 class="heading-element">Structure</h2>
<a
id="user-content-structure"
class="anchor"
aria-label="Permalink: Structure"
href="#structure"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<div class="highlight highlight-text-adblock">
<pre>
main.go
internal
business
http
handler.go
service.go
models.go
grpc
handler.go
models.go
consumer
handler.go
service.go
models.go
service.go
repository.go
models.go</pre
>
</div>
<p>
All business codes are inside <code>internal</code>. Each business has a
different directory <code>business</code>.
</p>
<p>
Inside each business, there are 2 handlers: <code>http</code>,
<code>grpc</code>:
</p>
<ul>
<li>
<code>http</code> is for public APIs (Android, iOS, ... are clients).
</li>
<li>
<code>grpc</code> is for internal APIs (other services are clients).
</li>
<li>
<code>consumer</code> is for consuming messages from queue (Kafka,
RabbitMQ, ...).
</li>
</ul>
<p>
For each handler, there are usually 3 layers: <code>handler</code>,
<code>service</code>, <code>repository</code>:
</p>
<ul>
<li>
<code>handler</code> interacts directly with gRPC, REST or consumer
using specific codes (cookies, ...) In case gRPC, there are frameworks
outside handle for us so we can write business/logic codes here too. But
remember, gRPC only.
</li>
<li>
<code>service</code> is where we write business/logic codes, and only
business/logic codes is written here.
</li>
<li>
<code>repository</code> is where we write codes which interacts with
database/cache like MySQL, Redis, ...
</li>
<li>
<code>models</code> is where we put all request, response, data models.
</li>
</ul>
<p>Location:</p>
<ul>
<li>
<code>handler</code> must exist inside <code>grpc</code>,
<code>http</code>, <code>consumer</code>.
</li>
<li>
<code>service</code>, <code>models</code> can exist directly inside of
<code>business</code> if both <code>grpc</code>, <code>http</code>,
<code>consumer</code> has same business/logic.
</li>
<li>
<code>repository</code> should be placed directly inside of
<code>business</code>.
</li>
</ul>
<div class="markdown-heading">
<h2 class="heading-element">Do not repeat!</h2>
<a
id="user-content-do-not-repeat"
class="anchor"
aria-label="Permalink: Do not repeat!"
href="#do-not-repeat"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>If we have too many services, some of the logic will be overlapped.</p>
<p>
For example, service A and service B both need to make POST call API to
service C. If service A and service B both have libs to call service C to
do that API, we need to move the libs to some common pkg libs. So in the
future, service D which needs to call C will not need to copy libs to
handle service C api but only need to import from common pkg libs.
</p>
<p>
Another bad practice is adapter service. No need to write a new service if
what we need is just common pkg libs.
</p>
<div class="markdown-heading">
<h2 class="heading-element">Taste on style guide</h2>
<a
id="user-content-taste-on-style-guide"
class="anchor"
aria-label="Permalink: Taste on style guide"
href="#taste-on-style-guide"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<div class="markdown-heading">
<h3 class="heading-element">Stop using global var</h3>
<a
id="user-content-stop-using-global-var"
class="anchor"
aria-label="Permalink: Stop using global var"
href="#stop-using-global-var"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
If I see someone using global var, I swear I will shoot them twice in the
face.
</p>
<p>Why?</p>
<ul>
<li>Can not write unit test.</li>
<li>Is not thread safe.</li>
</ul>
<div class="markdown-heading">
<h3 class="heading-element">
Use functional options, but don't overuse it!
</h3>
<a
id="user-content-use-functional-options-but-dont-overuse-it"
class="anchor"
aria-label="Permalink: Use functional options, but don't overuse it!"
href="#use-functional-options-but-dont-overuse-it"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
For simple struct with 1 or 2 fields, no need to use functional options.
</p>
<p>
<a href="https://go.dev/play/p/0XnOLiHuoz3" rel="nofollow">Example</a>:
</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-k">func</span> <span class="pl-en">main</span>() {
<span class="pl-s1">s</span> <span class="pl-c1">:=</span> <span class="pl-en">NewS</span>(<span class="pl-en">WithA</span>(<span class="pl-c1">1</span>), <span class="pl-en">WithB</span>(<span class="pl-s">"b"</span>))
<span class="pl-s1">fmt</span>.<span class="pl-en">Printf</span>(<span class="pl-s">"%+v<span class="pl-cce">\n</span>"</span>, <span class="pl-s1">s</span>)
}
<span class="pl-k">type</span> <span class="pl-smi">S</span> <span class="pl-k">struct</span> {
<span class="pl-c1">fieldA</span> <span class="pl-smi">int</span>
<span class="pl-c1">fieldB</span> <span class="pl-smi">string</span>
}
<span class="pl-k">type</span> <span class="pl-smi">OptionS</span> <span class="pl-k">func</span>(<span class="pl-s1">s</span> <span class="pl-c1">*</span><span class="pl-smi">S</span>)
<span class="pl-k">func</span> <span class="pl-en">WithA</span>(<span class="pl-s1">a</span> <span class="pl-smi">int</span>) <span class="pl-smi">OptionS</span> {
<span class="pl-k">return</span> <span class="pl-k">func</span>(<span class="pl-s1">s</span> <span class="pl-c1">*</span><span class="pl-smi">S</span>) {
<span class="pl-s1">s</span>.<span class="pl-c1">fieldA</span> <span class="pl-c1">=</span> <span class="pl-s1">a</span>
}
}
<span class="pl-k">func</span> <span class="pl-en">WithB</span>(<span class="pl-s1">b</span> <span class="pl-smi">string</span>) <span class="pl-smi">OptionS</span> {
<span class="pl-k">return</span> <span class="pl-k">func</span>(<span class="pl-s1">s</span> <span class="pl-c1">*</span><span class="pl-smi">S</span>) {
<span class="pl-s1">s</span>.<span class="pl-c1">fieldB</span> <span class="pl-c1">=</span> <span class="pl-s1">b</span>
}
}
<span class="pl-k">func</span> <span class="pl-en">NewS</span>(<span class="pl-s1">opts</span> <span class="pl-c1">...</span><span class="pl-smi">OptionS</span>) <span class="pl-c1">*</span><span class="pl-smi">S</span> {
<span class="pl-s1">s</span> <span class="pl-c1">:=</span> <span class="pl-c1">&amp;</span><span class="pl-smi">S</span>{}
<span class="pl-k">for</span> <span class="pl-s1">_</span>, <span class="pl-s1">opt</span> <span class="pl-c1">:=</span> <span class="pl-k">range</span> <span class="pl-s1">opts</span> {
<span class="pl-en">opt</span>(<span class="pl-s1">s</span>)
}
<span class="pl-k">return</span> <span class="pl-s1">s</span>
}</pre>
</div>
<p>
In above example, I construct <code>s</code> with <code>WithA</code> and
<code>WithB</code> option. No need to pass direct field inside
<code>s</code>.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Use
<a href="https://pkg.go.dev/golang.org/x/sync/errgroup" rel="nofollow"
>errgroup</a
>
as much as possible
</h3>
<a
id="user-content-use-errgroup-as-much-as-possible"
class="anchor"
aria-label="Permalink: Use errgroup as much as possible"
href="#use-errgroup-as-much-as-possible"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
If business logic involves calling too many APIs, but they are not depend
on each other. We can fire them parallel :)
</p>
<p>
Personally, I prefer <code>errgroup</code> to <code>WaitGroup</code> (<a
href="https://pkg.go.dev/sync#WaitGroup"
rel="nofollow"
>https://pkg.go.dev/sync#WaitGroup</a
>). Because I always need deal with error. Be super careful with
<code>egCtx</code>, should use this instead of parent
<code>ctx</code> inside <code>eg.Go</code>.
</p>
<p>Example:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-s1">eg</span>, <span class="pl-s1">egCtx</span> <span class="pl-c1">:=</span> <span class="pl-s1">errgroup</span>.<span class="pl-en">WithContext</span>(<span class="pl-s1">ctx</span>)
<span class="pl-s1">eg</span>.<span class="pl-en">Go</span>(<span class="pl-k">func</span>() <span class="pl-smi">error</span> {
<span class="pl-c">// Do some thing</span>
<span class="pl-k">return</span> <span class="pl-c1">nil</span>
})
<span class="pl-s1">eg</span>.<span class="pl-en">Go</span>(<span class="pl-k">func</span>() <span class="pl-smi">error</span> {
<span class="pl-c">// Do other thing</span>
<span class="pl-k">return</span> <span class="pl-c1">nil</span>
})
<span class="pl-k">if</span> <span class="pl-s1">err</span> <span class="pl-c1">:=</span> <span class="pl-s1">eg</span>.<span class="pl-en">Wait</span>(); <span class="pl-s1">err</span> <span class="pl-c1">!=</span> <span class="pl-c1">nil</span> {
<span class="pl-c">// Handle error</span>
}</pre>
</div>
<div class="markdown-heading">
<h3 class="heading-element">
Use
<a href="https://pkg.go.dev/golang.org/x/sync/semaphore" rel="nofollow"
>semaphore</a
>
when need to implement WorkerPool
</h3>
<a
id="user-content-use-semaphore-when-need-to-implement-workerpool"
class="anchor"
aria-label="Permalink: Use semaphore when need to implement WorkerPool"
href="#use-semaphore-when-need-to-implement-workerpool"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
Please don't use external libs for WorkerPool, I don't want to deal with
dependency hell.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Use
<a href="https://pkg.go.dev/sync#Pool" rel="nofollow">sync.Pool</a> when
need to reuse object, mainly for <code>bytes.Buffer</code>
</h3>
<a
id="user-content-use-syncpool-when-need-to-reuse-object-mainly-for-bytesbuffer"
class="anchor"
aria-label="Permalink: Use sync.Pool when need to reuse object, mainly for bytes.Buffer"
href="#use-syncpool-when-need-to-reuse-object-mainly-for-bytesbuffer"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>Example:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-k">var</span> <span class="pl-s1">bufPool</span> <span class="pl-c1">=</span> sync.<span class="pl-smi">Pool</span>{
<span class="pl-s1">New</span>: <span class="pl-k">func</span>() <span class="pl-smi">any</span> {
<span class="pl-k">return</span> <span class="pl-en">new</span>(bytes.<span class="pl-smi">Buffer</span>)
},
}
<span class="pl-k">func</span> <span class="pl-en">MarshalWithoutEscapeHTML</span>(<span class="pl-s1">v</span> <span class="pl-smi">any</span>) ([]<span class="pl-smi">byte</span>, <span class="pl-smi">error</span>) {
<span class="pl-s1">b</span>, <span class="pl-s1">ok</span> <span class="pl-c1">:=</span> <span class="pl-s1">bufPool</span>.<span class="pl-en">Get</span>().(<span class="pl-c1">*</span>bytes.<span class="pl-smi">Buffer</span>)
<span class="pl-k">if</span> <span class="pl-c1">!</span><span class="pl-s1">ok</span> {
<span class="pl-k">return</span> <span class="pl-c1">nil</span>, <span class="pl-s1">ErrBufPoolNotBytesBuffer</span>
}
<span class="pl-s1">b</span>.<span class="pl-en">Reset</span>()
<span class="pl-k">defer</span> <span class="pl-s1">bufPool</span>.<span class="pl-en">Put</span>(<span class="pl-s1">b</span>)
<span class="pl-s1">encoder</span> <span class="pl-c1">:=</span> <span class="pl-s1">json</span>.<span class="pl-en">NewEncoder</span>(<span class="pl-s1">b</span>)
<span class="pl-s1">encoder</span>.<span class="pl-en">SetEscapeHTML</span>(<span class="pl-c1">false</span>)
<span class="pl-k">if</span> <span class="pl-s1">err</span> <span class="pl-c1">:=</span> <span class="pl-s1">encoder</span>.<span class="pl-en">Encode</span>(<span class="pl-s1">v</span>); <span class="pl-s1">err</span> <span class="pl-c1">!=</span> <span class="pl-c1">nil</span> {
<span class="pl-k">return</span> <span class="pl-c1">nil</span>, <span class="pl-s1">err</span>
}
<span class="pl-s1">result</span> <span class="pl-c1">:=</span> <span class="pl-en">make</span>([]<span class="pl-smi">byte</span>, <span class="pl-s1">b</span>.<span class="pl-en">Len</span>())
<span class="pl-en">copy</span>(<span class="pl-s1">result</span>, <span class="pl-s1">b</span>.<span class="pl-en">Bytes</span>())
<span class="pl-k">return</span> <span class="pl-s1">result</span>, <span class="pl-c1">nil</span>
}</pre>
</div>
<div class="markdown-heading">
<h3 class="heading-element">
<a href="https://go.dev/doc/tutorial/generics" rel="nofollow"
>Generics</a
>
with some tricks
</h3>
<a
id="user-content-generics-with-some-tricks"
class="anchor"
aria-label="Permalink: Generics with some tricks"
href="#generics-with-some-tricks"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
Take value then return pointer, useful with database struct full of
pointers:
</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-c">// Ptr takes in non-pointer and returns a pointer</span>
<span class="pl-k">func</span> <span class="pl-en">Ptr</span>[<span class="pl-s1">T</span> <span class="pl-smi">any</span>](<span class="pl-s1">v</span> <span class="pl-smi">T</span>) <span class="pl-c1">*</span><span class="pl-smi">T</span> {
<span class="pl-k">return</span> <span class="pl-c1">&amp;</span><span class="pl-s1">v</span>
}</pre>
</div>
<p>Return zero value:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-k">func</span> <span class="pl-en">Zero</span>[<span class="pl-s1">T</span> <span class="pl-smi">any</span>]() <span class="pl-smi">T</span> {
<span class="pl-k">var</span> <span class="pl-s1">zero</span> <span class="pl-smi">T</span>
<span class="pl-k">return</span> <span class="pl-s1">zero</span>
}</pre>
</div>
<div class="markdown-heading">
<h2 class="heading-element">External libs</h2>
<a
id="user-content-external-libs"
class="anchor"
aria-label="Permalink: External libs"
href="#external-libs"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<div class="markdown-heading">
<h3 class="heading-element">No need <code>vendor</code></h3>
<a
id="user-content-no-need-vendor"
class="anchor"
aria-label="Permalink: No need vendor"
href="#no-need-vendor"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
Only need if you need something from <code>vendor</code>, to generate mock
or something else.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Use <code>build.go</code> to include build tools in go.mod
</h3>
<a
id="user-content-use-buildgo-to-include-build-tools-in-gomod"
class="anchor"
aria-label="Permalink: Use build.go to include build tools in go.mod"
href="#use-buildgo-to-include-build-tools-in-gomod"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>To easily control version of build tools.</p>
<p>For example <code>build.go</code>:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-c">//go:build tools</span>
<span class="pl-c">// +build tools</span>
<span class="pl-k">package</span> main
<span class="pl-k">import</span> (
_ <span class="pl-s">"github.com/golang/protobuf/protoc-gen-go"</span>
)</pre>
</div>
<p>And then in <code>Makefile</code>:</p>
<div class="highlight highlight-source-makefile">
<pre><span class="pl-en">build</span>:
go install github.com/golang/protobuf/protoc-gen-go</pre>
</div>
<p>
We always get the version of build tools in <code>go.mod</code> each time
we install it. Future contributors will not cry anymore.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Don't use cli libs (<a href="https://github.com/spf13/cobra"
>spf13/cobra</a
>, <a href="https://github.com/urfave/cli">urfave/cli</a>) just for Go
service
</h3>
<a
id="user-content-dont-use-cli-libs-spf13cobra-urfavecli-just-for-go-service"
class="anchor"
aria-label="Permalink: Don't use cli libs (spf13/cobra, urfave/cli) just for Go service"
href="#dont-use-cli-libs-spf13cobra-urfavecli-just-for-go-service"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
What is the point to pass many params (<code>do-it</code>,
<code>--abc</code>, <code>--xyz</code>) when what we only need is start
service?
</p>
<p>
In my case, service starts with only config, and config should be read
from file or environment like
<a href="https://12factor.net/" rel="nofollow">The Twelve Factors</a>
guide.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Don't use
<a href="https://github.com/grpc-ecosystem/grpc-gateway"
>grpc-ecosystem/grpc-gateway</a
>
</h3>
<a
id="user-content-dont-use-grpc-ecosystemgrpc-gateway"
class="anchor"
aria-label="Permalink: Don't use grpc-ecosystem/grpc-gateway"
href="#dont-use-grpc-ecosystemgrpc-gateway"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>Just don't.</p>
<p>
Use
<a href="https://github.com/protocolbuffers/protobuf-go"
>protocolbuffers/protobuf-go</a
>, <a href="https://github.com/grpc/grpc-go">grpc/grpc-go</a> for gRPC.
</p>
<p>
Write 1 for both gRPC, REST sounds good, but in the end, it is not worth
it.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Don't use
<a href="https://github.com/uber/prototool">uber/prototool</a>, use
<a href="https://github.com/bufbuild/buf">bufbuild/buf</a>
</h3>
<a
id="user-content-dont-use-uberprototool-use-bufbuildbuf"
class="anchor"
aria-label="Permalink: Don't use uber/prototool, use bufbuild/buf"
href="#dont-use-uberprototool-use-bufbuildbuf"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
prototool is deprecated, and buf can generate, lint, format as good as
prototool.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Use <a href="https://github.com/gin-gonic/gin">gin-gonic/gin</a> for
REST.
</h3>
<a
id="user-content-use-gin-gonicgin-for-rest"
class="anchor"
aria-label="Permalink: Use gin-gonic/gin for REST."
href="#use-gin-gonicgin-for-rest"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>With <code>c *gin.Context</code>:</p>
<ul>
<li>
Don't use <code>c</code> when passing context, use
<code>c.Request.Context()</code> instead.
</li>
<li>
Don't use <code>c.Request.URL.Path</code>, use
<code>c.FullPath()</code> instead.
</li>
</ul>
<p>Remember to free resources after parse multipart form:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-k">defer</span> <span class="pl-k">func</span>() {
<span class="pl-k">if</span> <span class="pl-s1">err</span> <span class="pl-c1">:=</span> <span class="pl-s1">c</span>.<span class="pl-c1">Request</span>.<span class="pl-c1">MultipartForm</span>.<span class="pl-en">RemoveAll</span>(); <span class="pl-s1">err</span> <span class="pl-c1">!=</span> <span class="pl-c1">nil</span> {
<span class="pl-s1">fmt</span>.<span class="pl-en">Println</span>(<span class="pl-s1">err</span>)
}
}()</pre>
</div>
<div class="markdown-heading">
<h3 class="heading-element">
If you want log, just use
<a href="https://github.com/uber-go/zap">uber-go/zap</a>
</h3>
<a
id="user-content-if-you-want-log-just-use-uber-gozap"
class="anchor"
aria-label="Permalink: If you want log, just use uber-go/zap"
href="#if-you-want-log-just-use-uber-gozap"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>It is fast!</p>
<ul>
<li>
Don't overuse <code>func (*Logger) With</code>. Because if log line is
too long, there is a possibility that we can lost it.
</li>
<li>
Use <code>MarshalLogObject</code> when we need to hide some field of
object when log (field is long or has sensitive value)
</li>
<li>
Don't use <code>Panic</code>. Use <code>Fatal</code> for errors when
start service to check dependencies. If you really need panic level, use
<code>DPanic</code>.
</li>
<li>If doubt, use <code>zap.Any</code>.</li>
<li>
Use <code>contextID</code> or <code>traceID</code> in every log lines
for easily debug.
</li>
</ul>
<div class="markdown-heading">
<h3 class="heading-element">
To read config, use
<a href="https://github.com/spf13/viper">spf13/viper</a>
</h3>
<a
id="user-content-to-read-config-use-spf13viper"
class="anchor"
aria-label="Permalink: To read config, use spf13/viper"
href="#to-read-config-use-spf13viper"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
Only init config in main or cmd layer. Do not use
<code>viper.Get...</code> in business layer or inside business layer.
</p>
<p>Why?</p>
<ul>
<li>Hard to mock and test</li>
<li>Put all config in single place for easily tracking</li>
</ul>
<p>
Also, be careful if config value is empty. You should decide to continue
or stop the service if there is empty config.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Don't overuse ORM libs, no need to handle another layer above SQL.
</h3>
<a
id="user-content-dont-overuse-orm-libs-no-need-to-handle-another-layer-above-sql"
class="anchor"
aria-label="Permalink: Don't overuse ORM libs, no need to handle another layer above SQL."
href="#dont-overuse-orm-libs-no-need-to-handle-another-layer-above-sql"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
Each ORM libs has each different syntax. To learn and use those libs
correctly is time consuming. So just stick to plain SQL. It is easier to
debug when something is wrong.
</p>
<p>
Also please use
<a href="https://go.dev/doc/database/prepared-statements" rel="nofollow"
>prepared statement</a
>
as much as possible. Idealy, we should init all prepared statement when we
init database connection to cached it, not create it every time we need
it.
</p>
<p>
But <code>database/sql</code> has its own limit. For example, it is hard
to get primary key after insert/update. So may be you want to use ORM for
those cases. I hear that
<a href="https://github.com/go-gorm/gorm">go-gorm/gorm</a> is good.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Connect Redis with
<a href="https://github.com/redis/go-redis">redis/go-redis</a>
</h3>
<a
id="user-content-connect-redis-with-redisgo-redis"
class="anchor"
aria-label="Permalink: Connect Redis with redis/go-redis"
href="#connect-redis-with-redisgo-redis"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
Be careful when use
<a href="https://redis.io/commands/hgetall/" rel="nofollow">HGETALL</a>.
If key not found, empty data will be returned not nil error. See
<a href="https://github.com/redis/go-redis/issues/1668"
>redis/go-redis/issues/1668</a
>
</p>
<p>
Use
<a
href="https://redis.uptrace.dev/guide/go-redis-pipelines.html"
rel="nofollow"
>Pipelines</a
>
for:
</p>
<ul>
<li>HSET and EXPIRE in 1 command.</li>
<li>Multiple GET in 1 command.</li>
</ul>
<p>
Prefer to use <code>Pipelined</code> instead of <code>Pipeline</code>.
Inside <code>Pipelined</code>, please return <code>redis.Cmder</code> for
each command.
</p>
<p>Example:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-k">func</span> (<span class="pl-s1">c</span> <span class="pl-c1">*</span><span class="pl-smi">client</span>) <span class="pl-en">HSetWithExpire</span>(<span class="pl-s1">ctx</span> context.<span class="pl-smi">Context</span>, <span class="pl-s1">key</span> <span class="pl-smi">string</span>, <span class="pl-s1">values</span> []<span class="pl-smi">any</span>, <span class="pl-s1">expired</span> time.<span class="pl-smi">Duration</span>) <span class="pl-smi">error</span> {
<span class="pl-s1">cmds</span> <span class="pl-c1">:=</span> <span class="pl-en">make</span>([]redis.<span class="pl-smi">Cmder</span>, <span class="pl-c1">2</span>)
<span class="pl-k">if</span> <span class="pl-s1">_</span>, <span class="pl-s1">err</span> <span class="pl-c1">:=</span> <span class="pl-s1">c</span>.<span class="pl-en">Pipelined</span>(<span class="pl-s1">ctx</span>, <span class="pl-k">func</span>(<span class="pl-s1">pipe</span> redis.<span class="pl-smi">Pipeliner</span>) <span class="pl-smi">error</span> {
<span class="pl-s1">cmds</span>[<span class="pl-c1">0</span>] <span class="pl-c1">=</span> <span class="pl-s1">pipe</span>.<span class="pl-en">HSet</span>(<span class="pl-s1">ctx</span>, <span class="pl-s1">key</span>, <span class="pl-s1">values</span><span class="pl-c1">...</span>)
<span class="pl-k">if</span> <span class="pl-s1">expired</span> <span class="pl-c1">&gt;</span> <span class="pl-c1">0</span> {
<span class="pl-s1">cmds</span>[<span class="pl-c1">1</span>] <span class="pl-c1">=</span> <span class="pl-s1">pipe</span>.<span class="pl-en">Expire</span>(<span class="pl-s1">ctx</span>, <span class="pl-s1">key</span>, <span class="pl-s1">expired</span>)
}
<span class="pl-k">return</span> <span class="pl-c1">nil</span>
}); <span class="pl-s1">err</span> <span class="pl-c1">!=</span> <span class="pl-c1">nil</span> {
<span class="pl-k">return</span> <span class="pl-s1">err</span>
}
<span class="pl-k">for</span> <span class="pl-s1">_</span>, <span class="pl-s1">cmd</span> <span class="pl-c1">:=</span> <span class="pl-k">range</span> <span class="pl-s1">cmds</span> {
<span class="pl-k">if</span> <span class="pl-s1">cmd</span> <span class="pl-c1">==</span> <span class="pl-c1">nil</span> {
<span class="pl-k">continue</span>
}
<span class="pl-k">if</span> <span class="pl-s1">err</span> <span class="pl-c1">:=</span> <span class="pl-s1">cmd</span>.<span class="pl-en">Err</span>(); <span class="pl-s1">err</span> <span class="pl-c1">!=</span> <span class="pl-c1">nil</span> {
<span class="pl-k">return</span> <span class="pl-s1">err</span>
}
}
<span class="pl-k">return</span> <span class="pl-c1">nil</span>
}</pre>
</div>
<p>Remember to config:</p>
<ul>
<li><code>ReadTimeout</code>, <code>WriteTimeout</code></li>
</ul>
<div class="markdown-heading">
<h3 class="heading-element">
Connect MySQL with
<a href="https://github.com/go-sql-driver/mysql">go-sql-driver/mysql</a>
</h3>
<a
id="user-content-connect-mysql-with-go-sql-drivermysql"
class="anchor"
aria-label="Permalink: Connect MySQL with go-sql-driver/mysql"
href="#connect-mysql-with-go-sql-drivermysql"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>Remember to config:</p>
<ul>
<li><code>SetConnMaxLifetime</code></li>
<li><code>SetMaxOpenConns</code></li>
<li><code>SetMaxIdleConns</code></li>
<li><code>ParseTime</code> to true.</li>
<li><code>Loc</code> to <code>time.UTC</code>.</li>
<li><code>CheckConnLiveness</code> to true.</li>
<li><code>ReadTimeout</code>, <code>WriteTimeout</code></li>
</ul>
<div class="markdown-heading">
<h3 class="heading-element">
Connect SQLite with
<a href="https://gitlab.com/cznic/sqlite" rel="nofollow"
>modernc.org/sqlite</a
>
</h3>
<a
id="user-content-connect-sqlite-with-moderncorgsqlite"
class="anchor"
aria-label="Permalink: Connect SQLite with modernc.org/sqlite"
href="#connect-sqlite-with-moderncorgsqlite"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>Remember to config:</p>
<ul>
<li>Write-Ahead Logging: <code>PRAGMA journal_mode=WAL</code></li>
<li>
Disable connections pool with <code>SetMaxOpenConns</code> sets to 1
</li>
</ul>
<p>
Don't use
<a href="https://github.com/mattn/go-sqlite3">mattn/go-sqlite3</a>, it's
required <code>CGO_ENABLED</code>.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Connect Kafka with
<a href="https://github.com/IBM/sarama">IBM/sarama</a>
</h3>
<a
id="user-content-connect-kafka-with-ibmsarama"
class="anchor"
aria-label="Permalink: Connect Kafka with IBM/sarama"
href="#connect-kafka-with-ibmsarama"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
Use <code>sarama.V1_0_0_0</code>, because IBM decide to upgrade default
version.
</p>
<p>
Don't use
<a href="https://github.com/confluentinc/confluent-kafka-go"
>confluentinc/confluent-kafka-go</a
>, it's required <code>CGO_ENABLED</code>.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
If you want test, just use
<a href="https://github.com/stretchr/testify">stretchr/testify</a>.
</h3>
<a
id="user-content-if-you-want-test-just-use-stretchrtestify"
class="anchor"
aria-label="Permalink: If you want test, just use stretchr/testify."
href="#if-you-want-test-just-use-stretchrtestify"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
It is easy to write a suite test, thanks to testify. Also, for mocking,
there are many options out there. Pick 1 then sleep peacefully.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
If need to mock, choose
<a href="https://github.com/matryer/moq">matryer/moq</a> or
<a href="https://github.com/uber/mock">uber/mock</a>
</h3>
<a
id="user-content-if-need-to-mock-choose-matryermoq-or-ubermock"
class="anchor"
aria-label="Permalink: If need to mock, choose matryer/moq or uber/mock"
href="#if-need-to-mock-choose-matryermoq-or-ubermock"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
The first is easy to use but not powerful as the later. If you want to
make sure mock func is called with correct times, use the later.
</p>
<p>Example with <code>matryer/moq</code>:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-c">// Only gen mock if source code file is newer than mock file</span>
<span class="pl-c">// https://jonwillia.ms/2019/12/22/conditional-gomock-mockgen</span>
<span class="pl-c">//go:generate sh -c "test service_mock_generated.go -nt $GOFILE &amp;&amp; exit 0; moq -rm -out service_mock_generated.go . Service"</span></pre>
</div>
<div class="markdown-heading">
<h3 class="heading-element">
Be careful with <a href="https://github.com/spf13/cast">spf13/cast</a>
</h3>
<a
id="user-content-be-careful-with-spf13cast"
class="anchor"
aria-label="Permalink: Be careful with spf13/cast"
href="#be-careful-with-spf13cast"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>Don't cast proto enum:</p>
<div class="highlight highlight-source-go">
<pre><span class="pl-c">// Bad</span>
<span class="pl-s1">a</span> <span class="pl-c1">:=</span> <span class="pl-s1">cast</span>.<span class="pl-en">ToInt32</span>(<span class="pl-s1">servicev1</span>.<span class="pl-c1">ReasonCode_ABC</span>)
<span class="pl-c">// Good</span>
<span class="pl-s1">a</span> <span class="pl-c1">:=</span> <span class="pl-en">int32</span>(<span class="pl-s1">servicev1</span>.<span class="pl-c1">ReasonCode_ABC</span>)</pre>
</div>
<div class="markdown-heading">
<h3 class="heading-element">
Use
<a
href="https://pkg.go.dev/golang.org/x/tools/cmd/stringer"
rel="nofollow"
>stringer</a
>
if you want your type enum can be print as string
</h3>
<a
id="user-content-use-stringer-if-you-want-your-type-enum-can-be-print-as-string"
class="anchor"
aria-label="Permalink: Use stringer if you want your type enum can be print as string"
href="#use-stringer-if-you-want-your-type-enum-can-be-print-as-string"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<div class="highlight highlight-source-go">
<pre><span class="pl-k">type</span> <span class="pl-smi">Drink</span> <span class="pl-smi">int</span>
<span class="pl-k">const</span> (
<span class="pl-s1">Beer</span> <span class="pl-smi">Drink</span> <span class="pl-c1">=</span> <span class="pl-c1">iota</span>
<span class="pl-s1">Water</span>
<span class="pl-s1">OrangeJuice</span>
)</pre>
</div>
<div class="highlight highlight-source-shell">
<pre>go install golang.org/x/tools/cmd/stringer@latest
<span class="pl-c"><span class="pl-c">#</span> Run inside directory which contains Drink</span>
stringer -type=Drink</pre>
</div>
<div class="markdown-heading">
<h3 class="heading-element">
Don't waste your time rewrite rate limiter if your use case is simple,
use
<a href="https://pkg.go.dev/golang.org/x/time/rate" rel="nofollow"
>rate</a
>
or
<a href="https://github.com/go-redis/redis_rate">go-redis/redis_rate</a>
</h3>
<a
id="user-content-dont-waste-your-time-rewrite-rate-limiter-if-your-use-case-is-simple-use-rate-or-go-redisredis_rate"
class="anchor"
aria-label="Permalink: Don't waste your time rewrite rate limiter if your use case is simple, use rate or go-redis/redis_rate"
href="#dont-waste-your-time-rewrite-rate-limiter-if-your-use-case-is-simple-use-rate-or-go-redisredis_rate"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>
<strong>rate</strong> if you want rate limiter locally in your single
instance of service. <strong>redis_rate</strong> if you want rate limiter
distributed across all your instances of service.
</p>
<div class="markdown-heading">
<h3 class="heading-element">
Replace <code>go fmt</code>, <code>goimports</code> with
<a href="https://github.com/mvdan/gofumpt">mvdan/gofumpt</a>.
</h3>
<a
id="user-content-replace-go-fmt-goimports-with-mvdangofumpt"
class="anchor"
aria-label="Permalink: Replace go fmt, goimports with mvdan/gofumpt."
href="#replace-go-fmt-goimports-with-mvdangofumpt"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p><code>gofumpt</code> provides more rules when format Go codes.</p>
<div class="markdown-heading">
<h3 class="heading-element">
Use
<a href="https://github.com/golangci/golangci-lint"
>golangci/golangci-lint</a
>.
</h3>
<a
id="user-content-use-golangcigolangci-lint"
class="anchor"
aria-label="Permalink: Use golangci/golangci-lint."
href="#use-golangcigolangci-lint"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>No need to say more. Lint or get the f out!</p>
<p>
If you get <code>fieldalignment</code> error, use
<a
href="https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment"
rel="nofollow"
>fieldalignment</a
>
to fix them.
</p>
<div class="highlight highlight-source-shell">
<pre><span class="pl-c"><span class="pl-c">#</span> Install</span>
go install golang.org/x/tools/go/analysis/passes/fieldalignment/cmd/fieldalignment@latest
<span class="pl-c"><span class="pl-c">#</span> Fix</span>
fieldalignment -fix ./internal/business/<span class="pl-k">*</span>.go</pre>
</div>
<div class="markdown-heading">
<h2 class="heading-element">Snippet/script</h2>
<a
id="user-content-snippetscript"
class="anchor"
aria-label="Permalink: Snippet/script"
href="#snippetscript"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<p>Change import:</p>
<div class="highlight highlight-source-shell">
<pre>gofmt -w -r <span class="pl-s"><span class="pl-pds">'</span>"github.com/Sirupsen/logrus" -&gt; "github.com/sirupsen/logrus"<span class="pl-pds">'</span></span> <span class="pl-k">*</span>.go</pre>
</div>
<p>Cleanup if storage is full:</p>
<div class="highlight highlight-source-shell">
<pre>go clean -cache -testcache -modcache -fuzzcache -x</pre>
</div>
<div class="markdown-heading">
<h2 class="heading-element">Thanks</h2>
<a
id="user-content-thanks"
class="anchor"
aria-label="Permalink: Thanks"
href="#thanks"
><span aria-hidden="true" class="octicon octicon-link"></span
></a>
</div>
<ul>
<li>
<a
href="https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis"
rel="nofollow"
>Functional options for friendly APIs</a
>
</li>
<li>
<a
href="https://abhinavg.net/2022/12/06/designing-go-libraries/"
rel="nofollow"
>Designing Go Libraries: The Talk: The Article</a
>
</li>
<li>
<a href="https://github.com/uber-go/guide/blob/master/style.md"
>Uber Go Style Guide</a
>
</li>
<li>
<a href="https://google.github.io/styleguide/go/index" rel="nofollow"
>Google Go Style</a
>
</li>
<li>
<a
href="https://github.blog/2020-05-20-three-bugs-in-the-go-mysql-driver/"
rel="nofollow"
>Three bugs in the Go MySQL Driver</a
>
</li>
<li>
<a href="https://mtlynch.io/notes/picoshare-perf/" rel="nofollow"
>Fixing Memory Exhaustion Bugs in My Golang Web App</a
>
</li>
<li>
<a
href="https://www.commonfate.io/blog/prevent-logging-secrets-in-go-by-using-custom-types"
rel="nofollow"
>Prevent Logging Secrets in Go by Using Custom Types</a
>
</li>
<li>
<a
href="https://jonwillia.ms/2019/12/22/conditional-gomock-mockgen"
rel="nofollow"
>Speed Up GoMock with Conditional Generation</a
>
</li>
<li>
<a
href="https://turriate.com/articles/making-sqlite-faster-in-go"
rel="nofollow"
>Making SQLite faster in Go</a
>
</li>
<li>
<a href="https://encore.dev/blog/advanced-go-concurrency" rel="nofollow"
>Advanced Go Concurrency</a
>
</li>
<li>
<a
href="https://danielms.site/zet/2023/go-generic-non-ptr-to-ptr/"
rel="nofollow"
>Go generic: non-ptr to ptr</a
>
</li>
<li>
<a href="https://xeiaso.net/blog/gonads-2022-04-24" rel="nofollow"
>Crimes with Go Generics</a
>
</li>
</ul>
<div>
Feel free to ask me via
<a href="mailto:hauvipapro+posts@gmail.com">email</a> or
<a rel="me" href="https://hachyderm.io/@haunguyen">Mastodon</a>.
<br />Source code is available on
<a href="https://github.com/haunt98/posts-go">GitHub</a>
<a href="https://codeberg.org/yoshie/posts-go">Codeberg</a>
<a href="https://git.sr.ht/~youngyoshie/posts-go">sourcehut</a>
<a href="https://gitea.treehouse.systems/yoshie/posts-go">Treehouse</a>
<a href="https://gitlab.com/youngyoshie/posts-go">GitLab</a>
</div>
</body>
</html>