<!doctype html> <html> <head> <meta charset="utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1" /> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.5.1/github-markdown.min.css" /> <title>haunt98 posts</title> </head> <style> .markdown-body { box-sizing: border-box; min-width: 200px; max-width: 980px; margin: 0 auto; padding: 45px; font-family: Shantell Sans Normal, Rec Mono Casual, SF Pro, Inter, sans-serif; font-weight: 500; } .markdown-body pre { font-family: Berkeley Mono, IBM Plex Mono, SF Mono, Jetbrains Mono, monospace; } @media (max-width: 767px) { .markdown-body { padding: 15px; } } </style> <body class="markdown-body"> <h2> <a href="index.html"><code>~</code></a> </h2> <div class="markdown-heading"> <h1 class="heading-element">SQL</h1> <a id="user-content-sql" class="anchor" aria-label="Permalink: SQL" href="#sql" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> Previously in my fresher software developer time, I rarely write SQL, I always use ORM to wrap SQL. But time past and too much abstraction bites me. So I decide to only write SQL from now as much as possible, no more ORM for me. But if there is any cool ORM for Go, I guess I try. </p> <p> This guide is not kind of guide which cover all cases. Just my little tricks when I work with SQL. </p> <div class="markdown-heading"> <h2 class="heading-element">Stay away from database unique id</h2> <a id="user-content-stay-away-from-database-unique-id" class="anchor" aria-label="Permalink: Stay away from database unique id" href="#stay-away-from-database-unique-id" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> Use UUID instead. If you can, and you should, choose UUID type which can be sortable. </p> <div class="markdown-heading"> <h2 class="heading-element">Stay away from database timestamp</h2> <a id="user-content-stay-away-from-database-timestamp" class="anchor" aria-label="Permalink: Stay away from database timestamp" href="#stay-away-from-database-timestamp" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> Stay away from all kind of database timestamp (MySQL timestmap, SQLite timestamp, ...) Just use int64 then pass the timestamp in service layer not database layer. </p> <p> Why? Because time and date and location are too much complex to handle. In my business, I use timestamp in milliseconds. Then I save timestamp as int64 value to database. Each time I get timestamp from database, I parse to time struct in Go with location or format I want. No more hassle! </p> <p>It looks like this:</p> <div class="highlight highlight-text-adblock"> <pre> [Business] time, data -> convert to unix timestamp milliseconds -> [Database] int64</pre > </div> <div class="markdown-heading"> <h2 class="heading-element">Extra field for extra things</h2> <a id="user-content-extra-field-for-extra-things" class="anchor" aria-label="Permalink: Extra field for extra things" href="#extra-field-for-extra-things" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> Create new column in database is scary, so I suggest avoid it if you can. How to avoid, first design table with extra field. It is black hole, put everything in there if you want. </p> <p>I always use MySQL json data type for extra field.</p> <p>JSON data type is also useful for dumping request, response data.</p> <ul> <li> <a href="https://dev.mysql.com/doc/refman/5.7/en/json.html" rel="nofollow" >For MySQL 5.7</a > </li> <li> <a href="https://dev.mysql.com/doc/refman/8.0/en/json.html" rel="nofollow" >For MySQL 8.0</a > </li> </ul> <p> Use <code>JSON_EXTRACT(col, '$.key') IS NULL</code> to check json field exist or not. </p> <div class="markdown-heading"> <h2 class="heading-element">Use index!!!</h2> <a id="user-content-use-index" class="anchor" aria-label="Permalink: Use index!!!" href="#use-index" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> You should use index for faster query, but not too much. Don't create index for every fields in table. Choose wisely! </p> <p>For example, create index in MySQL:</p> <div class="highlight highlight-source-sql"> <pre><span class="pl-k">CREATE</span> <span class="pl-k">INDEX</span> <span class="pl-en">idx_user_id</span> <span class="pl-k">ON</span> user_upload (user_id);</pre> </div> <p> If create index inside <code>CREATE TABLE</code>, <a href="https://stackoverflow.com/a/1401615" rel="nofollow" >prefer <code>INDEX</code> to <code>KEY</code></a >: </p> <div class="highlight highlight-source-sql"> <pre><span class="pl-k">CREATE</span> <span class="pl-k">TABLE</span> <span class="pl-en">user_upload</span> ( id <span class="pl-k">int</span>(<span class="pl-c1">11</span>) <span class="pl-k">NOT NULL</span>, user_id <span class="pl-k">int</span>(<span class="pl-c1">11</span>) <span class="pl-k">NULL</span> DEFAULT <span class="pl-k">NULL</span>, <span class="pl-k">PRIMARY KEY</span> (id), INDEX idx_user_id (user_id) );</pre> </div> <p>Use <code>EXPLAIN</code> to check if index is used or not:</p> <ul> <li> <a href="https://dev.mysql.com/doc/refman/5.7/en/explain-output.html" rel="nofollow" >For MySQL 5.7</a > </li> <li> <a href="https://dev.mysql.com/doc/refman/8.0/en/explain-output.html" rel="nofollow" >For MySQL 8.0</a > </li> </ul> <div class="markdown-heading"> <h2 class="heading-element">Be careful with UTF-8</h2> <a id="user-content-be-careful-with-utf-8" class="anchor" aria-label="Permalink: Be careful with UTF-8" href="#be-careful-with-utf-8" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p>TLDR with MySQL:</p> <div class="highlight highlight-source-sql"> <pre><span class="pl-k">CREATE</span> <span class="pl-k">TABLE</span> <span class="pl-en">ekyc_approved</span> ( id <span class="pl-k">varchar</span>(<span class="pl-c1">30</span>) <span class="pl-k">NOT NULL</span>, <span class="pl-k">PRIMARY KEY</span> (id), ) ENGINE <span class="pl-k">=</span> InnoDB DEFAULT CHARSET <span class="pl-k">=</span> utf8mb4;</pre> </div> <div class="markdown-heading"> <h2 class="heading-element">Be careful with NULL</h2> <a id="user-content-be-careful-with-null" class="anchor" aria-label="Permalink: Be careful with NULL" href="#be-careful-with-null" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> If compare with field which can be NULL, remember to check NULL for safety. </p> <div class="highlight highlight-source-sql"> <pre><span class="pl-c"><span class="pl-c">--</span> field_something can be NULL</span> <span class="pl-c"><span class="pl-c">--</span> Bad</span> <span class="pl-k">SELECT</span> <span class="pl-k">*</span> <span class="pl-k">FROM</span> table <span class="pl-k">WHERE</span> field_something <span class="pl-k">!=</span> <span class="pl-c1">1</span> <span class="pl-c"><span class="pl-c">--</span> Good</span> <span class="pl-k">SELECT</span> <span class="pl-k">*</span> <span class="pl-k">FROM</span> table <span class="pl-k">WHERE</span> (field_something IS <span class="pl-k">NULL</span> <span class="pl-k">OR</span> field_something <span class="pl-k">!=</span> <span class="pl-c1">1</span>)</pre> </div> <p>Need clarify why this happpen? Idk :(</p> <div class="markdown-heading"> <h2 class="heading-element"><code>VARCHAR</code> or <code>TEXT</code></h2> <a id="user-content-varchar-or-text" class="anchor" aria-label="Permalink: VARCHAR or TEXT" href="#varchar-or-text" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> Prefer <code>VARCHAR</code> if you need to query and of course use index, and make sure size of value will never hit the limit. Prefer <code>TEXT</code> if you don't care, just want to store something. </p> <p>If you need to store UUID, use <code>VARCHAR(255)</code>.</p> <div class="markdown-heading"> <h2 class="heading-element"><code>LIMIT</code></h2> <a id="user-content-limit" class="anchor" aria-label="Permalink: LIMIT" href="#limit" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> Prefer <code>LIMIT 10 OFFSET 5</code> to <code>LIMIT 5, 10</code> to avoid misunderstanding. </p> <div class="markdown-heading"> <h2 class="heading-element"> Be super careful when migrate, update database on production and online!!! </h2> <a id="user-content-be-super-careful-when-migrate-update-database-on-production-and-online" class="anchor" aria-label="Permalink: Be super careful when migrate, update database on production and online!!!" href="#be-super-careful-when-migrate-update-database-on-production-and-online" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p> Plase read docs about online ddl operations before do anything online (keep database running the same time update it, for example create index, ...) </p> <ul> <li> <a href="https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-operations.html" rel="nofollow" >For MySQL 5.7</a >, <a href="https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-limitations.html" rel="nofollow" >Limitations</a > </li> <li> <a href="https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html" rel="nofollow" >For MySQL 8.0</a >, <a href="https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-limitations.html" rel="nofollow" >Limitations</a > </li> </ul> <div class="markdown-heading"> <h2 class="heading-element">Heathcheck</h2> <a id="user-content-heathcheck" class="anchor" aria-label="Permalink: Heathcheck" href="#heathcheck" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <p>Use <code>SELECT 1</code> to check if database failed yet.</p> <div class="markdown-heading"> <h2 class="heading-element">Tools</h2> <a id="user-content-tools" class="anchor" aria-label="Permalink: Tools" href="#tools" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <ul> <li> Use <a href="https://github.com/sqlfluff/sqlfluff">sqlfluff/sqlfluff</a> to check your SQL. </li> <li> Use <a href="https://github.com/k1LoW/tbls">k1LoW/tbls</a> to grasp your database reality :) </li> </ul> <div class="markdown-heading"> <h2 class="heading-element">Thanks</h2> <a id="user-content-thanks" class="anchor" aria-label="Permalink: Thanks" href="#thanks" ><span aria-hidden="true" class="octicon octicon-link"></span ></a> </div> <ul> <li> <a href="https://use-the-index-luke.com/" rel="nofollow" >Use The Index, Luke</a > </li> <li> <a href="https://www.foxhound.systems/blog/essential-elements-of-high-performance-sql-indexes/" rel="nofollow" >Essential elements of high performance applications: SQL indexes</a > </li> <li> <a href="https://architecturenotes.co/things-you-should-know-about-databases/" rel="nofollow" >Things You Should Know About Databases</a > </li> <li> <a href="https://shekhargulati.com/2022/01/08/when-to-use-json-data-type-in-database-schema-design/" rel="nofollow" >When to use JSON data type in database schema design?</a > </li> <li> <a href="https://shekhargulati.com/2022/07/08/my-notes-on-gitlabs-postgres-schema-design/" rel="nofollow" >My Notes on GitLab Postgres Schema Design</a > </li> <li> <a href="https://planetscale.com/blog/how-read-mysql-explains" rel="nofollow" >How to read MySQL EXPLAINs</a > </li> <li> <a href="https://brandur.org/fragments/database-health-check" rel="nofollow" >Honest health checks that hit the database</a > </li> <li> <a href="https://www.grouparoo.com/blog/varchar-191" rel="nofollow" >Why are database columns 191 characters?</a > </li> <li> <a href="https://stackoverflow.com/a/43056611" rel="nofollow" >Store UUID v4 in MySQL</a > </li> </ul> <div> Feel free to ask me via <a href="mailto:hauvipapro+posts@gmail.com">email</a> or <a rel="me" href="https://hachyderm.io/@haunguyen">Mastodon</a>. <br />Source code is available on <a href="https://github.com/haunt98/posts-go">GitHub</a> <a href="https://codeberg.org/yoshie/posts-go">Codeberg</a> <a href="https://git.sr.ht/~youngyoshie/posts-go">sourcehut</a> <a href="https://gitea.treehouse.systems/yoshie/posts-go">Treehouse</a> <a href="https://gitlab.com/youngyoshie/posts-go">GitLab</a> </div> </body> </html>