yoshie
/
posts-go
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

feat: murr sql

main
sudo pacman -Syu 2022-11-20 16:35:49 +07:00
parent faefce6908
commit bb930b0287
No known key found for this signature in database
GPG Key ID: D6CB5C6C567C47B0
4 changed files with 46 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/2022-07-10-bootstrap-go.html
View File

@ -74,10 +74,10 @@ import (
</code></pre><p>We always get the version of build tools in <code>go.mod</code> each time we install it.<br>Future contributors will not cry anymore.<h3>Don't use cli libs (<a href=https://github.com/spf13/cobra>spf13/cobra</a>, <a href=https://github.com/urfave/cli>urfave/cli</a>) just for Go service</h3><p>What is the point to pass many params (<code>do-it</code>, <code>--abc</code>, <code>--xyz</code>) when what we only need is start service?<p>In my case, service starts with only config, and config should be read from file or environment like <a href=https://12factor.net/>The Twelve Factors</a> guide.<h3>Don't use <a href=https://github.com/grpc-ecosystem/grpc-gateway>grpc-ecosystem/grpc-gateway</a></h3><p>Just don't.<p>Use <a href=https://github.com/protocolbuffers/protobuf-go>protocolbuffers/protobuf-go</a>, <a href=https://github.com/grpc/grpc-go>grpc/grpc-go</a> for gRPC.<p>Write 1 for both gRPC, REST sounds good, but in the end, it is not worth it.<h3>Don't use <a href=https://github.com/uber/prototool>uber/prototool</a>, use <a href=https://github.com/bufbuild/buf>bufbuild/buf</a></h3><p>prototool is deprecated, and buf can generate, lint, format as good as prototool.<h3>Use <a href=https://github.com/gin-gonic/gin>gin-gonic/gin</a> for REST.</h3><p>Don't use <code>gin.Context</code> when pass context from handler layer to service layer, use <code>gin.Context.Request.Context()</code> instead.<h3>If you want log, just use <a href=https://github.com/uber-go/zap>uber-go/zap</a></h3><p>It is fast!<ul><li><p>Don't overuse <code>func (*Logger) With</code>. Because if log line is too long, there is a possibility that we can lost it.<li><p>Use <code>MarshalLogObject</code> when we need to hide some field of object when log (field is long or has sensitive value)<li><p>Don't use <code>Panic</code>. Use <code>Fatal</code> for errors when start service to check dependencies. If you really need panic level, use <code>DPanic</code>.<li><p>If doubt, use <code>zap.Any</code>.<li><p>Use <code>contextID</code> or <code>traceID</code> in every log lines for easily debug.</ul><h3>To read config, use <a href=https://github.com/spf13/viper>spf13/viper</a></h3><p>Only init config in main or cmd layer.<br>Do not use <code>viper.Get...</code> in business layer or inside business layer.<p>Why?<ul><li>Hard to mock and test<li>Put all config in single place for easily tracking</ul><h3>Don't overuse ORM libs, no need to handle another layer above SQL.</h3><p>Each ORM libs has each different syntax.<br>To learn and use those libs correctly is time consuming.<br>So just stick to plain SQL.<br>It is easier to debug when something is wrong.<p>But <code>database/sql</code> has its own limit.<br>For example, it is hard to get primary key after insert/update.<br>So may be you want to use ORM for those cases.<br>I hear that <a href=https://github.com/go-gorm/gorm>go-gorm/gorm</a>, <a href=https://github.com/ent/ent>ent/ent</a> is good.<h3>If you want test, just use <a href=https://github.com/stretchr/testify>stretchr/testify</a>.</h3><p>It is easy to write a suite test, thanks to testify.<br>Also, for mocking, there are many options out there.<br>Pick 1 then sleep peacefully.<h3>If need to mock, choose <a href=https://github.com/matryer/moq>matryer/moq</a> or <a href=https://github.com/golang/mock>golang/mock</a></h3><p>The first is easy to use but not powerful as the later.<br>If you want to make sure mock func is called with correct times, use the later.<p>Example with <code>matryer/moq</code>:<pre><code class=language-go>// Only gen mock if source code file is newer than mock file
// https://jonwillia.ms/2019/12/22/conditional-gomock-mockgen
//go:generate sh -c &quot;test service_mock_generated.go -nt $GOFILE &amp;&amp; exit 0; moq -rm -out service_mock_generated.go . Service&quot;
</code></pre><h3>Be careful with <a href=https://github.com/spf13/cast>spf13/cast</a></h3><p>Don't cast proto enum:<pre><code class=language-go>// Don't
</code></pre><h3>Be careful with <a href=https://github.com/spf13/cast>spf13/cast</a></h3><p>Don't cast proto enum:<pre><code class=language-go>// Bad
a := cast.ToInt32(servicev1.ReasonCode_ABC)
// Do
// Good
a := int32(servicev1.ReasonCode_ABC)
</code></pre><h3>Use <a href=https://pkg.go.dev/golang.org/x/tools/cmd/stringer>stringer</a> if you want your type enum can be print as string</h3><pre><code class=language-go>type Drink int

11
docs/2022-07-31-sql.html
View File

@ -1,14 +1,15 @@
<!doctype html><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Recursive:wght,CASL,MONO@300..800,0..1,0..1&display=swap" rel=stylesheet><link href=https://haunt98.github.io/iosevka_webfont/iosevka-term-ss08/iosevka-term-ss08.css rel=stylesheet><link rel=stylesheet href=styles.css><a href=index>Index</a><h1>SQL</h1><p>Previously in my fresher time, I rarely write SQL, I always use ORM to wrap SQL.<br>But time past and too much abstraction bites me.<br>So I decide to only write SQL from now, no more ORM for me.<br>But if there is any cool ORM for Go, I guess I try.<p>This guide is not kind of guide which cover all cases.<br>Just my little tricks when I work with SQL.<h1>Stay away from database unique id</h1><p>Use UUID instead.<h1>Stay away from database timestamp</h1><p>Stay away from all kind of timestamp, timestamp of MySQL or any database.<br>Just use int64 then pass the timestamp in service layer not database layer.<p>Why? Because time and date and location are too much complex to handle.<br>In my business, I use timestamp in milliseconds.<br>Then I save timestamp as int64 value to database.<br>Each time I get timestamp from database, I parse to time struct in Go with location or format I want.<br>No more hassle!<h1>Use index</h1><p>You should use index for faster query, but not too much.<br>Don't create index for every fields in table.<br>Choose wisely!<p>For example, create index in MySQL:<pre><code class=language-sql>CREATE INDEX `idx_timestamp`
<!doctype html><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Recursive:wght,CASL,MONO@300..800,0..1,0..1&display=swap" rel=stylesheet><link href=https://haunt98.github.io/iosevka_webfont/iosevka-term-ss08/iosevka-term-ss08.css rel=stylesheet><link rel=stylesheet href=styles.css><a href=index>Index</a><h1>SQL</h1><p>Previously in my fresher software developer time, I rarely write SQL, I always use ORM to wrap SQL.<br>But time past and too much abstraction bites me.<br>So I decide to only write SQL from now as much as possible, no more ORM for me.<br>But if there is any cool ORM for Go, I guess I try.<p>This guide is not kind of guide which cover all cases.<br>Just my little tricks when I work with SQL.<h2>Stay away from database unique id</h2><p>Use UUID instead.<br>If you can, and you should, choose UUID type which can be sortable.<h2>Stay away from database timestamp</h2><p>Stay away from all kind of database timestamp (MySQL timestmap, SQLite timestamp, ...)<br>Just use int64 then pass the timestamp in service layer not database layer.<p>Why? Because time and date and location are too much complex to handle.<br>In my business, I use timestamp in milliseconds.<br>Then I save timestamp as int64 value to database.<br>Each time I get timestamp from database, I parse to time struct in Go with location or format I want.<br>No more hassle!<p>It looks like this:<pre><code class=language-txt>[Business] time, data -&gt; convert to unix timestamp milliseconds -&gt; [Database] int64
</code></pre><h2>Use index!!!</h2><p>You should use index for faster query, but not too much.<br>Don't create index for every fields in table.<br>Choose wisely!<p>For example, create index in MySQL:<pre><code class=language-sql>CREATE INDEX `idx_timestamp`
ON `user_upload` (`timestamp`);
</code></pre><h1>Be careful with NULL</h1><p>If compare with field which can be NULL, remember to check NULL for safety.<pre><code class=language-sql>-- field_something can be NULL
</code></pre><h2>Be careful with NULL</h2><p>If compare with field which can be NULL, remember to check NULL for safety.<pre><code class=language-sql>-- field_something can be NULL
-- Don't
-- Bad
SELECT *
FROM table
WHERE field_something != 1
-- Do
-- Good
SELECT *
FROM table
WHERE (field_something IS NULL OR field_something != 1)
</code></pre><h1>Thanks</h1><ul><li><a href=https://use-the-index-luke.com/>Use The Index, Luke</a><li><a href=https://www.foxhound.systems/blog/essential-elements-of-high-performance-sql-indexes/>Essential elements of high performance applications: SQL indexes</a><li><a href=https://architecturenotes.co/things-you-should-know-about-databases/>Things You Should Know About Databases</a></ul><a href=mailto:hauvipapro+posts@gmail.com>Feel free to ask me via email</a>
</code></pre><p>Need clarify why this happpen? Idk :(<h2><code>VARCHAR</code> or <code>TEXT</code></h2><p>Prefer <code>VARCHAR</code> if you need to query and of course use index, and make sure size of value will never hit the limit.<br>Prefer <code>TEXT</code> if you don't care, just want to store something.<h2>Be super careful when migrate, update database on production and online!!!</h2><p>Plase read docs about online ddl operations before do anything online (keep database running the same time update it, for example create index, ...)<ul><li><a href=https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-operations.html>For MySQL 5.7</a>, <a href=https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-limitations.html>Limitations</a><li><a href=https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html>For MySQL 8.0</a>, <a href=https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-limitations.html>Limitations</a></ul><h2>Tools</h2><ul><li>Use <a href=https://github.com/sqlfluff/sqlfluff>sqlfluff/sqlfluff</a> to check your SQL.<li>Use <a href=https://github.com/k1LoW/tbls>k1LoW/tbls</a> to grasp your database reality :)</ul><h2>Thanks</h2><ul><li><a href=https://use-the-index-luke.com/>Use The Index, Luke</a><li><a href=https://www.foxhound.systems/blog/essential-elements-of-high-performance-sql-indexes/>Essential elements of high performance applications: SQL indexes</a><li><a href=https://architecturenotes.co/things-you-should-know-about-databases/>Things You Should Know About Databases</a></ul><a href=mailto:hauvipapro+posts@gmail.com>Feel free to ask me via email</a>

4
posts/2022-07-10-bootstrap-go.md
View File

@ -261,10 +261,10 @@ Example with `matryer/moq`:
Don't cast proto enum:
```go
// Don't
// Bad
a := cast.ToInt32(servicev1.ReasonCode_ABC)
// Do
// Good
a := int32(servicev1.ReasonCode_ABC)
```

46
posts/2022-07-31-sql.md
View File

@ -1,20 +1,21 @@
# SQL
Previously in my fresher time, I rarely write SQL, I always use ORM to wrap SQL.
Previously in my fresher software developer time, I rarely write SQL, I always use ORM to wrap SQL.
But time past and too much abstraction bites me.
So I decide to only write SQL from now, no more ORM for me.
So I decide to only write SQL from now as much as possible, no more ORM for me.
But if there is any cool ORM for Go, I guess I try.
This guide is not kind of guide which cover all cases.
Just my little tricks when I work with SQL.
# Stay away from database unique id
## Stay away from database unique id
Use UUID instead.
If you can, and you should, choose UUID type which can be sortable.
# Stay away from database timestamp
## Stay away from database timestamp
Stay away from all kind of timestamp, timestamp of MySQL or any database.
Stay away from all kind of database timestamp (MySQL timestmap, SQLite timestamp, ...)
Just use int64 then pass the timestamp in service layer not database layer.
Why? Because time and date and location are too much complex to handle.
@ -23,7 +24,13 @@ Then I save timestamp as int64 value to database.
Each time I get timestamp from database, I parse to time struct in Go with location or format I want.
No more hassle!
# Use index
It looks like this:
```txt
[Business] time, data -> convert to unix timestamp milliseconds -> [Database] int64
```
## Use index!!!
You should use index for faster query, but not too much.
Don't create index for every fields in table.
@ -36,25 +43,44 @@ CREATE INDEX `idx_timestamp`
ON `user_upload` (`timestamp`);
```
# Be careful with NULL
## Be careful with NULL
If compare with field which can be NULL, remember to check NULL for safety.
```sql
-- field_something can be NULL
-- Don't
-- Bad
SELECT *
FROM table
WHERE field_something != 1
-- Do
-- Good
SELECT *
FROM table
WHERE (field_something IS NULL OR field_something != 1)
```
# Thanks
Need clarify why this happpen? Idk :(
## `VARCHAR` or `TEXT`
Prefer `VARCHAR` if you need to query and of course use index, and make sure size of value will never hit the limit.
Prefer `TEXT` if you don't care, just want to store something.
## Be super careful when migrate, update database on production and online!!!
Plase read docs about online ddl operations before do anything online (keep database running the same time update it, for example create index, ...)
- [For MySQL 5.7](https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-operations.html), [Limitations](https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-limitations.html)
- [For MySQL 8.0](https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html), [Limitations](https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-limitations.html)
## Tools
- Use [sqlfluff/sqlfluff](https://github.com/sqlfluff/sqlfluff) to check your SQL.
- Use [k1LoW/tbls](https://github.com/k1LoW/tbls) to grasp your database reality :)
## Thanks
- [Use The Index, Luke](https://use-the-index-luke.com/)
- [Essential elements of high performance applications: SQL indexes](https://www.foxhound.systems/blog/essential-elements-of-high-performance-sql-indexes/)