yoshie
/
posts-go
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

chore: use docs/

main
sudo pacman -Syu 2022-07-10 18:09:42 +07:00
parent baf8ae630d
commit 7260f7ca43
No known key found for this signature in database
GPG Key ID: D6CB5C6C567C47B0
5 changed files with 456 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@ -9,6 +9,3 @@
# VSCode
.vscode
# HTML
generated

136
docs/2022-06-08-backup.html Normal file
View File

@ -0,0 +1,136 @@
<!DOCTYPE html>
<html>
<head>
<title>2022-06-08-backup.md</title>
<meta name="GENERATOR" content="github.com/gomarkdown/markdown markdown processor for Go">
<meta charset="utf-8">
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Recursive:slnt,wght,CASL,CRSV,MONO@-15..0,300..800,0..1,0..1,0..1&display=swap"
rel="stylesheet"
/>
<style>
body {
font-family: "Recursive", sans-serif;
font-variation-settings: "MONO" 0, "CASL" 1;
}
code {
font-family: "Recursive", monospace;
font-variation-settings: "MONO" 1, "CASL" 1;
}
</style>
</head>
<body>
<nav>
<ul>
<li><a href="#toc_0">Backup my way</a>
<ul>
<li><a href="#toc_1">How to backup</a></li>
<li><a href="#toc_2">Recovery strategy</a></li>
<li><a href="#toc_3">The end</a></li>
</ul></li>
</ul>
</nav>
<h1 id="toc_0">Backup my way</h1>
<p>First thing first, I want to list my own devices, which I have through the years:</p>
<ul>
<li>Laptop Samsung NP300E4Z-S06VN (Old laptop which I give to my mom)</li>
<li><a href="https://www.dell.com/support/home/en-vn/product-support/product/inspiron-15-3567-laptop/drivers">Laptop Dell Inspiron 15 3567</a> (My mom bought it for me when I go to college, I give it to my sister afterward)</li>
<li><a href="https://www.acer.com/ac/en/US/content/support-product/8841">Laptop Acer Nitro AN515-45</a> (Gaming laptop which I buy for gaming, of course)</li>
<li>MacBook Pro M1 2020 (My company laptop)</li>
<li>Phone Xiaomi Poco X3 NFC (Primary phone which I use daily)</li>
</ul>
<p>App/Service I use daily:</p>
<ul>
<li><a href="https://bitwarden.com/">Bitwarden</a></li>
<li><a href="https://getaegis.app/">Aegis Authenticator</a></li>
<li><a href="https://rclone.org/">Rclone</a></li>
<li><a href="https://tailscale.com/">Tailscale</a></li>
<li>GitHub / GitLab</li>
<li>Google Keep / Notion</li>
<li>Google Drive (I use 200GB plan)</li>
</ul>
<p>The purpose is that I want my data to be safe, secure, and can be easily recovered if I lost some devices;
or in the worst situation, I lost all.
Because you know, it is hard to guess what is waiting for us in the future.</p>
<p>There are 2 sections which I want to share, the first is <strong>How to backup</strong>, the second is <strong>Recover strategy</strong>.</p>
<h2 id="toc_1">How to backup</h2>
<p>Before I talk about backup, I want to talk about data.
In specifically, which data should I backup?</p>
<p>I use Arch Linux and macOS, primarily work in the terminal so I have too many dotfiles, for example, <code>~/.config/nvim/init.lua</code>.
Each time I reinstall Arch Linux (I like it a lot), I need to reconfigure all the settings, and it is time-consuming.</p>
<p>So for the DE and UI settings, I keep it as default as possible, unless it&rsquo;s getting in my way, I leave the default setting there and forget about it.
The others are dotfiles, which I write my own <a href="https://github.com/haunt98/dotfiles">dotfiles tool</a> to backup and reconfigure easily and quickly.
Also, I know that installing Arch Linux is not easy, despite I install it too many times (Like thousand times since I was in high school).
Not because it is hard, but as life goes on, the <a href="https://wiki.archlinux.org/title/installation_guide">official install guide</a> keeps getting new update and covering too many cases for my own personal use, so I write my own <a href="https://github.com/haunt98/til/blob/main/install-archlinux.md">guide</a> to quickly capture what I need to do.
I back up all my dotfiles inside my dotfiles tool in GitHub and GitLab as I trust them both.
Also as I travel the Internet, I discover <a href="https://codeberg.org/">Codeberg</a> and <a href="https://gitea.treehouse.systems/">Treehouse</a> and use them as another backup for git repo.</p>
<p>So that is my dotfiles, for my regular data, like Wallpaper or Books, Images, I use Google Drive (Actually I pay for it).
But the step: open the webpage, click the upload button and choose files seems boring and time-consuming.
So I use Rclone, it supports Google Drive, One Drive and many providers but I only use Google Drive for now.
The commands are simple:</p>
<pre><code class="language-sh"># Sync from local to remote
rclone sync MyBooks remote:MyBooks -P --exclude .DS_Store
# Sync from remote to local
rclone sync remote:MyBooks MyBooks -P --exclude .DS_Store
</code></pre>
<p>Before you use Rclone to sync to Google Drive, you should read <a href="https://rclone.org/drive/">Google Drive rclone configuration</a> first.</p>
<p>The next data is my passwords and my OTPs.
These are the things which I&rsquo;m scare to lose the most.
First thing first, I enable 2-Step Verification for all of my important accounts, should use both OTP and phone method.</p>
<p>I use Bitwarden for passwords (That is a long story, coming from Google Password manager to Firefox Lockwise and then settle down with Bitwarden) and Aegis for OTPs.
The reason I choose Aegis, not Authy (I use Authy for so long but Aegis is definitely better) is because Aegis allows me to extract all the OTPs to a single file (Can be encrypted), which I use to transfer or backup easily.</p>
<p>As long as Bitwarden provides free passwords stored, I use all of its apps, extensions so that I can easily sync passwords between laptops and phones.
The thing I need to remember is the master password of Bitwarden in my head.</p>
<p>With Aegis, I export the data, then sync it to Google Drive, also store it locally in my phone.
For safety, I also store Aegis data locally on all of my laptops (Encrypted of course).</p>
<p>The main problem here is the OTP, I can not store all of my OTPs in the cloud completely.
Because if I want to access my OTPs in the cloud, I should log in, and then input my OTP, this is a circle, my friends.</p>
<h2 id="toc_2">Recovery strategy</h2>
<p>There are many strategies that I process to react as if something strange is happening to my devices.</p>
<p>If I lost my laptops, single laptop or all, do not panic as long as I have my phones.
The OTPs are in there, the passwords are in Bitwarden cloud, other data is in Google Drive so nothing is lost here.</p>
<p>If I lost my phone, but not my laptops, I use the OTPs which are stored locally in my laptops.</p>
<p>In the worst situation, I lost everything, my laptops, my phone.
The first step is to recover my SIM, then log in to Google account using the password and SMS OTP.
After that, log in to Bitwarden account using the master password and OTP from Gmail, which I open previously.</p>
<h2 id="toc_3">The end</h2>
<p>This guide will be updated regularly I promise.</p>
</body>
</html>

125
docs/2022-06-08-dockerfile-go.html Normal file
View File

@ -0,0 +1,125 @@
<!DOCTYPE html>
<html>
<head>
<title>2022-06-08-dockerfile-go.md</title>
<meta name="GENERATOR" content="github.com/gomarkdown/markdown markdown processor for Go">
<meta charset="utf-8">
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Recursive:slnt,wght,CASL,CRSV,MONO@-15..0,300..800,0..1,0..1,0..1&display=swap"
rel="stylesheet"
/>
<style>
body {
font-family: "Recursive", sans-serif;
font-variation-settings: "MONO" 0, "CASL" 1;
}
code {
font-family: "Recursive", monospace;
font-variation-settings: "MONO" 1, "CASL" 1;
}
</style>
</head>
<body>
<nav>
<ul>
<li><a href="#toc_0">Dockerfile for Go</a></li>
</ul>
</nav>
<h1 id="toc_0">Dockerfile for Go</h1>
<p>Each time I start a new Go project, I repeat many steps.
Like set up <code>.gitignore</code>, CI configs, Dockerfile, &hellip;</p>
<p>So I decide to have a baseline Dockerfile like this:</p>
<pre><code class="language-Dockerfile">FROM golang:1.18-bullseye as builder
RUN go install golang.org/dl/go1.18@latest \
&amp;&amp; go1.18 download
WORKDIR /build
COPY go.mod .
COPY go.sum .
COPY vendor .
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOAMD64=v3 go build -o ./app -tags timetzdata -trimpath .
FROM gcr.io/distroless/base-debian11
COPY --from=builder /build/app /app
ENTRYPOINT [&quot;/app&quot;]
</code></pre>
<p>I use <a href="https://docs.docker.com/develop/develop-images/multistage-build/">multi-stage build</a> to keep my image size small.
First stage is <a href="https://hub.docker.com/_/golang">Go official image</a>,
second stage is <a href="https://github.com/GoogleContainerTools/distroless">Distroless</a>.</p>
<p>Before Distroless, I use <a href="https://hub.docker.com/_/alpine">Alpine official image</a>,
There is a whole discussion on the Internet to choose which is the best base image for Go.
After reading some blogs, I discover Distroless as a small and secure base image.
So I stick with it for a while.</p>
<p>Also, remember to match Distroless Debian version with Go official image Debian version.</p>
<pre><code class="language-Dockerfile">FROM golang:1.18-bullseye as builder
</code></pre>
<p>This is Go image I use as a build stage.
This can be official Go image or custom image is required in some companies.</p>
<pre><code class="language-Dockerfile">RUN go install golang.org/dl/go1.18@latest \
&amp;&amp; go1.18 download
</code></pre>
<p>This is optional.
In my case, my company is slow to update Go image so I use this trick to install latest Go version.</p>
<pre><code class="language-Dockerfile">WORKDIR /build
COPY go.mod .
COPY go.sum .
COPY vendor .
COPY . .
</code></pre>
<p>I use <code>/build</code> to emphasize that I am building something in that directory.</p>
<p>The 4 <code>COPY</code> lines are familiar if you use Go enough.
First is <code>go.mod</code> and <code>go.sum</code> because it defines Go modules.
The second is <code>vendor</code>, this is optional but I use it because I don&rsquo;t want each time I build Dockerfile, I need to redownload Go modules.</p>
<pre><code class="language-Dockerfile">RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOAMD64=v3 go build -o ./app -tags timetzdata -trimpath .
</code></pre>
<p>This is where I build Go program.</p>
<p><code>CGO_ENABLED=0</code> because I don&rsquo;t want to mess with C libraries.
<code>GOOS=linux GOARCH=amd64</code> is easy to explain, Linux with x86-64.
<code>GOAMD64=v3</code> is new since <a href="https://go.dev/doc/go1.18#amd64">Go 1.18</a>,
I use v3 because I read about AMD64 version in <a href="https://gitlab.archlinux.org/archlinux/rfcs/-/blob/master/rfcs/0002-march.rst">Arch Linux rfcs</a>. TLDR&rsquo;s newer computers are already x86-64-v3.</p>
<p><code>-tags timetzdata</code> to embed timezone database incase base image does not have.
<code>-trimpath</code> to support reproduce build.</p>
<pre><code class="language-Dockerfile">FROM gcr.io/distroless/base-debian11
COPY --from=builder /build/app /app
ENTRYPOINT [&quot;/app&quot;]
</code></pre>
<p>Finally, I copy <code>app</code> to Distroless base image.</p>
</body>
</html>

192
docs/2022-07-10-bootstrap-go.html Normal file
View File

@ -0,0 +1,192 @@
<!DOCTYPE html>
<html>
<head>
<title>2022-07-10-bootstrap-go.md</title>
<meta name="GENERATOR" content="github.com/gomarkdown/markdown markdown processor for Go">
<meta charset="utf-8">
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Recursive:slnt,wght,CASL,CRSV,MONO@-15..0,300..800,0..1,0..1,0..1&display=swap"
rel="stylesheet"
/>
<style>
body {
font-family: "Recursive", sans-serif;
font-variation-settings: "MONO" 0, "CASL" 1;
}
code {
font-family: "Recursive", monospace;
font-variation-settings: "MONO" 1, "CASL" 1;
}
</style>
</head>
<body>
<nav>
<ul>
<li><a href="#toc_0">Bootstrap Go</a>
<ul>
<li><a href="#toc_1">Structure</a></li>
<li><a href="#toc_2">Do not repeat!</a></li>
<li><a href="#toc_3">External libs</a>
<ul>
<li><a href="#toc_4">Don&rsquo;t use cli libs (<a href="https://github.com/spf13/cobra">spf13/cobra</a>, <a href="https://github.com/urfave/cli">urfave/cli</a>) just for Go service</a></li>
<li><a href="#toc_5">Don&rsquo;t use <a href="https://github.com/grpc-ecosystem/grpc-gateway">grpc-ecosystem/grpc-gateway</a></a></li>
<li><a href="#toc_6">Don&rsquo;t use <a href="https://github.com/uber/prototool">uber/prototool</a>, use <a href="https://github.com/bufbuild/buf">bufbuild/buf</a></a></li>
<li><a href="#toc_7">Use <a href="https://github.com/gin-gonic/gin">gin-gonic/gin</a> for REST.</a></li>
<li><a href="#toc_8">If you want log, just use <a href="https://github.com/uber-go/zap">uber-go/zap</a></a></li>
<li><a href="#toc_9">Don&rsquo;t overuse ORM libs, no need to handle another layer above SQL.</a></li>
<li><a href="#toc_10">If you want test, just use <a href="https://github.com/stretchr/testify">stretchr/testify</a>.</a></li>
<li><a href="#toc_11">Replace <code>go fmt</code>, <code>goimports</code> with <a href="https://github.com/mvdan/gofumpt">mvdan/gofumpt</a>.</a></li>
<li><a href="#toc_12">Use <a href="https://github.com/golangci/golangci-lint">golangci/golangci-lint</a>.</a></li>
</ul></li>
</ul></li>
</ul>
</nav>
<h1 id="toc_0">Bootstrap Go</h1>
<p>It is hard to write bootstrap tool to quickly create Go service.
So I write this guide instead.
This is a quick checklist for me every damn time I need to write a Go service from scratch.
Also, this is my personal opinion, so feel free to comment.</p>
<h2 id="toc_1">Structure</h2>
<pre><code class="language-txt">main.go
internal
| business_1
| | http
| | | handler.go
| | | service.go
| | | repository.go
| | | models.go
| | grpc
| | | handler.go
| | | service.go
| | | repository.go
| | | models.go
| | service.go
| | repository.go
| | models.go
| business_2
| | grpc
| | | handler.go
| | | service.go
| | | repository.go
| | | models.go
</code></pre>
<p>All business codes are inside <code>internal</code>.
Each business has a different directory (<code>business_1</code>, <code>business_2</code>).</p>
<p>Inside each business, there are 2 handlers: <code>http</code>, <code>grpc</code>:</p>
<ul>
<li><code>http</code> is for public APIs (Android, iOS,&hellip; are clients).</li>
<li><code>grpc</code> is for internal APIs (other services are clients).</li>
</ul>
<p>Inside each handler, there are usually 3 layers: <code>handler</code>, <code>service</code>, <code>repository</code>:</p>
<ul>
<li><code>handler</code> interacts directly with gRPC or REST using specific codes (cookies,&hellip;)</li>
<li><code>service</code> is where we write business/logic codes, and only business/logic codes is written here.</li>
<li><code>repository</code> is where we write codes which interacts with database/cache like MySQL, Redis, &hellip;</li>
</ul>
<p><code>handler</code> must exist inside <code>grpc</code>, <code>http</code>.
But <code>service</code>, <code>repository</code>, <code>models</code> can exist directly inside <code>business</code> if both <code>grpc</code>, <code>http</code> has same business/logic.</p>
<h2 id="toc_2">Do not repeat!</h2>
<p>If we have too many services, some of the logic will be overlapped.</p>
<p>For example, service A and service B both need to make POST call API to service C.
If service A and service B both have libs to call service C to do that API, we need to move the libs to some common pkg libs.
So in the future, service D which needs to call C will not need to copy libs to handle service C api but only need to import from common pkg libs.</p>
<p>Another bad practice is adapter service.
No need to write a new service if what we need is just common pkg libs.</p>
<h2 id="toc_3">External libs</h2>
<h3 id="toc_4">Don&rsquo;t use cli libs (<a href="https://github.com/spf13/cobra">spf13/cobra</a>, <a href="https://github.com/urfave/cli">urfave/cli</a>) just for Go service</h3>
<p>What is the point to pass many params (<code>--abc</code>, <code>--xyz</code>) when what we only need is start service?</p>
<p>In my case, service starts with only config, and config should be read from file or environment like <a href="https://12factor.net/">The Twelve Factors</a> guide.</p>
<h3 id="toc_5">Don&rsquo;t use <a href="https://github.com/grpc-ecosystem/grpc-gateway">grpc-ecosystem/grpc-gateway</a></h3>
<p>Just don&rsquo;t.</p>
<p>Use <a href="https://github.com/protocolbuffers/protobuf-go">protocolbuffers/protobuf-go</a>, <a href="https://github.com/grpc/grpc-go">grpc/grpc-go</a> for gRPC.</p>
<p>Write 1 for both gRPC, REST sounds good, but in the end, it is not worth it.</p>
<h3 id="toc_6">Don&rsquo;t use <a href="https://github.com/uber/prototool">uber/prototool</a>, use <a href="https://github.com/bufbuild/buf">bufbuild/buf</a></h3>
<p>prototool is deprecated, and buf can generate, lint, format as good as prototool.</p>
<h3 id="toc_7">Use <a href="https://github.com/gin-gonic/gin">gin-gonic/gin</a> for REST.</h3>
<p>Don&rsquo;t use <code>gin.Context</code> when pass context from handler layer to service layer, use <code>gin.Context.Request.Context()</code> instead.</p>
<h3 id="toc_8">If you want log, just use <a href="https://github.com/uber-go/zap">uber-go/zap</a></h3>
<p>It is fast!</p>
<ul>
<li><p>Don&rsquo;t overuse <code>func (*Logger) With</code>. Because if log line is too long, there is a possibility that we can lost it.</p></li>
<li><p>Use <code>MarshalLogObject</code> when we need to hide some field of object when log (field has long or sensitive value)</p></li>
<li><p>Don&rsquo;t use <code>Panic</code>. Use <code>Fatal</code> for errors when start service to check dependencies. If you really need panic level, use <code>DPanic</code>.</p></li>
<li><p>Use <code>contextID</code> or <code>traceID</code> in every log lines for easily debug.</p></li>
</ul>
<h3 id="toc_9">Don&rsquo;t overuse ORM libs, no need to handle another layer above SQL.</h3>
<p>Each ORM libs has each different syntax.
To learn and use those libs correctly is time consuming.
So just stick to plain SQL.
It is easier to debug when something is wrong.</p>
<p>But <code>database/sql</code> has its own limit.
For example, it is hard to get primary key after insert/update.
So may be you want to use ORM for those cases.</p>
<h3 id="toc_10">If you want test, just use <a href="https://github.com/stretchr/testify">stretchr/testify</a>.</h3>
<p>It is easy to write a suite test, thanks to testify.
Also, for mocking, there are many options out there.
Pick 1 then sleep peacefully.</p>
<h3 id="toc_11">Replace <code>go fmt</code>, <code>goimports</code> with <a href="https://github.com/mvdan/gofumpt">mvdan/gofumpt</a>.</h3>
<p><code>gofumpt</code> provides more rules when format Go codes.</p>
<h3 id="toc_12">Use <a href="https://github.com/golangci/golangci-lint">golangci/golangci-lint</a>.</h3>
<p>No need to say more.
Lint or get the f out!</p>
</body>
</html>

6
main.go
View File

@ -13,7 +13,7 @@ import (
const (
postsPath = "posts"
headHTMLPath = "custom/head.html"
generatedPath = "generated"
generatedPath = "docs"
htmlExt = ".html"
)
@ -32,7 +32,7 @@ func main() {
log.Fatalln("Failed to remove all", generatedPath, err)
}
if err := os.MkdirAll(generatedPath, 0777); err != nil {
if err := os.MkdirAll(generatedPath, 0o777); err != nil {
log.Fatalln("Failed to mkdir all", generatedPath)
}
@ -60,7 +60,7 @@ func main() {
generatedFileName := strings.TrimSuffix(file.Name(), filepath.Ext(file.Name())) + htmlExt
generatedFilePath := filepath.Join(generatedPath, generatedFileName)
if err := os.WriteFile(generatedFilePath, generatedHTML, 0666); err != nil {
if err := os.WriteFile(generatedFilePath, generatedHTML, 0o666); err != nil {
log.Fatalln("Failed to write file", generatedFilePath, err)
}
}