2023-07-18 18:38:30 +00:00
|
|
|
<!doctype html>
|
2022-12-25 19:13:58 +00:00
|
|
|
<html>
|
|
|
|
|
<head>
|
|
|
|
|
<meta charset="utf-8" />
|
|
|
|
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
|
|
|
<link
|
|
|
|
|
rel="stylesheet"
|
2023-08-06 07:41:38 +00:00
|
|
|
href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.2.0/github-markdown.min.css"
|
2022-12-25 19:13:58 +00:00
|
|
|
/>
|
2023-04-01 03:37:48 +00:00
|
|
|
<title>haunt98 posts</title>
|
2022-12-25 19:13:58 +00:00
|
|
|
</head>
|
|
|
|
|
<style>
|
|
|
|
|
.markdown-body {
|
|
|
|
|
box-sizing: border-box;
|
|
|
|
|
min-width: 200px;
|
|
|
|
|
max-width: 980px;
|
|
|
|
|
margin: 0 auto;
|
|
|
|
|
padding: 45px;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
@media (max-width: 767px) {
|
|
|
|
|
.markdown-body {
|
|
|
|
|
padding: 15px;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
</style>
|
|
|
|
|
<body class="markdown-body">
|
2023-08-09 07:22:58 +00:00
|
|
|
<h2>
|
|
|
|
|
<a href="index.html"><code>~</code></a>
|
|
|
|
|
</h2>
|
2022-12-30 09:26:06 +00:00
|
|
|
<h1>
|
|
|
|
|
<a id="user-content-sql" class="anchor" aria-hidden="true" href="#sql"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>SQL
|
|
|
|
|
</h1>
|
|
|
|
|
<p>
|
|
|
|
|
Previously in my fresher software developer time, I rarely write SQL, I
|
|
|
|
|
always use ORM to wrap SQL. But time past and too much abstraction bites
|
|
|
|
|
me. So I decide to only write SQL from now as much as possible, no more
|
|
|
|
|
ORM for me. But if there is any cool ORM for Go, I guess I try.
|
|
|
|
|
</p>
|
|
|
|
|
<p>
|
|
|
|
|
This guide is not kind of guide which cover all cases. Just my little
|
|
|
|
|
tricks when I work with SQL.
|
|
|
|
|
</p>
|
|
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-stay-away-from-database-unique-id"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#stay-away-from-database-unique-id"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Stay away from database unique id
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
Use UUID instead. If you can, and you should, choose UUID type which can
|
|
|
|
|
be sortable.
|
|
|
|
|
</p>
|
|
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-stay-away-from-database-timestamp"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#stay-away-from-database-timestamp"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Stay away from database timestamp
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
Stay away from all kind of database timestamp (MySQL timestmap, SQLite
|
|
|
|
|
timestamp, ...) Just use int64 then pass the timestamp in service layer
|
|
|
|
|
not database layer.
|
|
|
|
|
</p>
|
|
|
|
|
<p>
|
|
|
|
|
Why? Because time and date and location are too much complex to handle. In
|
|
|
|
|
my business, I use timestamp in milliseconds. Then I save timestamp as
|
|
|
|
|
int64 value to database. Each time I get timestamp from database, I parse
|
|
|
|
|
to time struct in Go with location or format I want. No more hassle!
|
|
|
|
|
</p>
|
|
|
|
|
<p>It looks like this:</p>
|
|
|
|
|
<div class="highlight highlight-text-adblock">
|
|
|
|
|
<pre>
|
|
|
|
|
[Business] time, data -> convert to unix timestamp milliseconds -> [Database] int64</pre
|
|
|
|
|
>
|
|
|
|
|
</div>
|
2023-04-04 10:25:22 +00:00
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-extra-field-for-extra-things"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#extra-field-for-extra-things"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Extra field for extra things
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
Create new column in database is scary, so I suggest avoid it if you can.
|
|
|
|
|
How to avoid, first design table with extra field. It is black hole, put
|
|
|
|
|
everything in there if you want.
|
|
|
|
|
</p>
|
|
|
|
|
<p>I always use MySQL json data type for extra field.</p>
|
2023-04-08 10:10:10 +00:00
|
|
|
<p>JSON data type also used for dumping request, response data.</p>
|
2022-12-30 09:26:06 +00:00
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-use-index"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#use-index"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Use index!!!
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
You should use index for faster query, but not too much. Don't create
|
|
|
|
|
index for every fields in table. Choose wisely!
|
|
|
|
|
</p>
|
|
|
|
|
<p>For example, create index in MySQL:</p>
|
|
|
|
|
<div class="highlight highlight-source-sql">
|
2023-06-15 19:31:21 +00:00
|
|
|
<pre><span class="pl-k">CREATE</span> <span class="pl-k">INDEX</span> <span class="pl-en">idx_user_id</span>
|
|
|
|
|
<span class="pl-k">ON</span> user_upload (user_id);</pre>
|
|
|
|
|
</div>
|
|
|
|
|
<p>
|
|
|
|
|
If create index inside <code>CREATE TABLE</code>,
|
|
|
|
|
<a href="https://stackoverflow.com/a/1401615" rel="nofollow"
|
|
|
|
|
>prefer <code>INDEX</code> to <code>KEY</code></a
|
|
|
|
|
>:
|
|
|
|
|
</p>
|
|
|
|
|
<div class="highlight highlight-source-sql">
|
|
|
|
|
<pre><span class="pl-k">CREATE</span> <span class="pl-k">TABLE</span> <span class="pl-en">user_upload</span>
|
|
|
|
|
(
|
|
|
|
|
id <span class="pl-k">int</span>(<span class="pl-c1">11</span>) <span class="pl-k">NOT NULL</span>,
|
|
|
|
|
user_id <span class="pl-k">int</span>(<span class="pl-c1">11</span>) <span class="pl-k">NULL</span> DEFAULT <span class="pl-k">NULL</span>,
|
|
|
|
|
<span class="pl-k">PRIMARY KEY</span> (id),
|
|
|
|
|
INDEX idx_user_id (user_id)
|
|
|
|
|
);</pre>
|
2022-12-30 09:26:06 +00:00
|
|
|
</div>
|
2023-04-17 10:14:54 +00:00
|
|
|
<p>Use <code>EXPLAIN</code> to check if index is used or not:</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://dev.mysql.com/doc/refman/5.7/en/explain-output.html"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>For MySQL 5.7</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://dev.mysql.com/doc/refman/8.0/en/explain-output.html"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>For MySQL 8.0</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
2022-12-30 09:26:06 +00:00
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-be-careful-with-null"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#be-careful-with-null"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Be careful with NULL
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
If compare with field which can be NULL, remember to check NULL for
|
|
|
|
|
safety.
|
|
|
|
|
</p>
|
|
|
|
|
<div class="highlight highlight-source-sql">
|
|
|
|
|
<pre><span class="pl-c"><span class="pl-c">--</span> field_something can be NULL</span>
|
2022-12-25 19:13:58 +00:00
|
|
|
|
|
|
|
|
<span class="pl-c"><span class="pl-c">--</span> Bad</span>
|
|
|
|
|
<span class="pl-k">SELECT</span> <span class="pl-k">*</span>
|
|
|
|
|
<span class="pl-k">FROM</span> table
|
|
|
|
|
<span class="pl-k">WHERE</span> field_something <span class="pl-k">!=</span> <span class="pl-c1">1</span>
|
2022-09-06 17:10:02 +00:00
|
|
|
|
2022-12-25 19:13:58 +00:00
|
|
|
<span class="pl-c"><span class="pl-c">--</span> Good</span>
|
|
|
|
|
<span class="pl-k">SELECT</span> <span class="pl-k">*</span>
|
|
|
|
|
<span class="pl-k">FROM</span> table
|
2022-12-30 09:26:06 +00:00
|
|
|
<span class="pl-k">WHERE</span> (field_something IS <span class="pl-k">NULL</span> <span class="pl-k">OR</span> field_something <span class="pl-k">!=</span> <span class="pl-c1">1</span>)</pre>
|
|
|
|
|
</div>
|
|
|
|
|
<p>Need clarify why this happpen? Idk :(</p>
|
|
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-varchar-or-text"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#varchar-or-text"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span
|
|
|
|
|
></a>
|
|
|
|
|
<code>VARCHAR</code> or <code>TEXT</code>
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
Prefer <code>VARCHAR</code> if you need to query and of course use index,
|
|
|
|
|
and make sure size of value will never hit the limit. Prefer
|
|
|
|
|
<code>TEXT</code> if you don't care, just want to store something.
|
|
|
|
|
</p>
|
2023-06-15 19:31:21 +00:00
|
|
|
<h2>
|
|
|
|
|
<a id="user-content-limit" class="anchor" aria-hidden="true" href="#limit"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
><code>LIMIT</code>
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
Prefer <code>LIMIT 10 OFFSET 5</code> to <code>LIMIT 5, 10</code> to avoid
|
|
|
|
|
misunderstanding.
|
|
|
|
|
</p>
|
2022-12-30 09:26:06 +00:00
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-be-super-careful-when-migrate-update-database-on-production-and-online"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#be-super-careful-when-migrate-update-database-on-production-and-online"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Be super careful when migrate, update database on production and
|
|
|
|
|
online!!!
|
|
|
|
|
</h2>
|
|
|
|
|
<p>
|
|
|
|
|
Plase read docs about online ddl operations before do anything online
|
|
|
|
|
(keep database running the same time update it, for example create index,
|
|
|
|
|
...)
|
|
|
|
|
</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-operations.html"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>For MySQL 5.7</a
|
|
|
|
|
>,
|
|
|
|
|
<a
|
|
|
|
|
href="https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-limitations.html"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>Limitations</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>For MySQL 8.0</a
|
|
|
|
|
>,
|
|
|
|
|
<a
|
|
|
|
|
href="https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-limitations.html"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>Limitations</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<h2>
|
|
|
|
|
<a id="user-content-tools" class="anchor" aria-hidden="true" href="#tools"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Tools
|
|
|
|
|
</h2>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
Use
|
|
|
|
|
<a href="https://github.com/sqlfluff/sqlfluff">sqlfluff/sqlfluff</a> to
|
|
|
|
|
check your SQL.
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
Use <a href="https://github.com/k1LoW/tbls">k1LoW/tbls</a> to grasp your
|
|
|
|
|
database reality :)
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<h2>
|
|
|
|
|
<a
|
|
|
|
|
id="user-content-thanks"
|
|
|
|
|
class="anchor"
|
|
|
|
|
aria-hidden="true"
|
|
|
|
|
href="#thanks"
|
|
|
|
|
><span aria-hidden="true" class="octicon octicon-link"></span></a
|
|
|
|
|
>Thanks
|
|
|
|
|
</h2>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
<a href="https://use-the-index-luke.com/" rel="nofollow"
|
|
|
|
|
>Use The Index, Luke</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://www.foxhound.systems/blog/essential-elements-of-high-performance-sql-indexes/"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>Essential elements of high performance applications: SQL indexes</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://architecturenotes.co/things-you-should-know-about-databases/"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>Things You Should Know About Databases</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
2023-04-08 10:10:10 +00:00
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://shekhargulati.com/2022/01/08/when-to-use-json-data-type-in-database-schema-design/"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>When to use JSON data type in database schema design?</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://shekhargulati.com/2022/07/08/my-notes-on-gitlabs-postgres-schema-design/"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>My Notes on GitLab Postgres Schema Design</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
2023-04-17 10:14:54 +00:00
|
|
|
<li>
|
|
|
|
|
<a
|
|
|
|
|
href="https://planetscale.com/blog/how-read-mysql-explains"
|
|
|
|
|
rel="nofollow"
|
|
|
|
|
>How to read MySQL EXPLAINs</a
|
|
|
|
|
>
|
|
|
|
|
</li>
|
2022-12-30 09:26:06 +00:00
|
|
|
</ul>
|
2022-09-06 17:10:02 +00:00
|
|
|
|
2022-12-25 19:22:39 +00:00
|
|
|
<div>
|
|
|
|
|
Feel free to ask me via
|
|
|
|
|
<a href="mailto:hauvipapro+posts@gmail.com">email</a> or
|
2022-12-30 09:55:38 +00:00
|
|
|
<a rel="me" href="https://hachyderm.io/@haunguyen">Mastodon</a>. Source
|
|
|
|
|
code is available on
|
|
|
|
|
<a href="https://github.com/haunt98/posts-go">GitHub</a>
|
2022-12-25 19:22:39 +00:00
|
|
|
</div>
|
2022-12-25 19:13:58 +00:00
|
|
|
</body>
|
|
|
|
|
</html>
|
