posts-go/posts/2023-03-05-incident.md

42 lines
1.4 KiB
Markdown
Raw Permalink Normal View History

2023-03-05 08:48:43 +00:00
# Incident come and go
This is collect of all incidents I created in the past :(
## Service starts with empty port
Because all configs is read from file.
2023-08-05 18:56:25 +00:00
But the port config is empty -> So when service inits, it use that empty port
somehow.
2023-03-05 08:48:43 +00:00
**Solution**: For some configs, make sure to failed first if it's empty.
2023-04-09 04:53:17 +00:00
## Race condition of series of APIs
For example I have 2 APIs:
- API upload: allow user to upload image
- API submit: submit data to server
2023-08-05 18:56:25 +00:00
API upload is slow, it takes 10s to finish. API submit is fast, only takes 2s.
2023-04-09 04:53:17 +00:00
2023-08-05 18:56:25 +00:00
The problem is submit use data from upload too. When user calls API upload,
image is stored in cache. When user calls API submit, it use whatever image is
stored in cache.
2023-04-09 04:53:17 +00:00
It's when the fun begins.
2023-08-05 18:56:25 +00:00
Imagine user Trong already upload image. So he is ready to submit. But for the
same time, he re-call API upload to upload another image too.
2023-04-09 04:53:17 +00:00
2023-08-05 18:56:25 +00:00
So if API upload is finished first, which is kinda impossible (U know upload
file is not fast right?), everything right. But for most cases, API submit is
finished first. It means Trong's data is submitted with the old image. Then API
upload is finished, it will replace the old image with the new one.So the old
one, aka image in submitted data, is gone.
2023-04-09 04:53:17 +00:00
Chaos right there!
2023-08-05 18:56:25 +00:00
**Solution**: Use a lock, if user enter API upload, lock it to prevent user call
other APIs. Rememeber to unlock after finished