Commit Graph

772 Commits (e46321e63d4dd59102e28d3f54813efbb0227eb8)

Author SHA1 Message Date
Takeshi Umeda 0ce0baa9b5 Add parallelization to `tootctl search deploy` (#12051)
* Add parallel gem

* Modify parallel option in tootctl search deploy

* Add paralell option to tootctl search deploy

* Change 1 to false

* Clean up

* Rename --parallel to --processes
2019-10-02 21:50:43 +02:00
Eugen Rochko c7adf80827
Change `tootctl domains purge` to accept multiple domains at once (#12046) 2019-10-02 04:49:53 +02:00
Thibaut Girka 16ff7c5627 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- Gemfile.lock
- app/controllers/about_controller.rb
- app/controllers/auth/sessions_controller.rb
2019-09-30 12:23:57 +02:00
Eugen Rochko 13b06d4b3b
Bump version to 3.0.0rc2 (#11999) 2019-09-29 18:50:16 +02:00
Eugen Rochko e87bcaa107
Bump version to 3.0.0rc1 (#11900) 2019-09-21 20:02:14 +02:00
Eugen Rochko 3c8372fa81
Bump version to 2.9.3 (#11899) 2019-09-20 22:59:29 +02:00
Thibaut Girka 74c5b2bd08 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
  Conflict because we changed the number of default results to be
  configurable
- app/lib/settings/scoped_settings.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
2019-09-13 18:13:43 +02:00
Eugen Rochko c5d37f18cb
Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Eugen Rochko 031ca25014
Add retry for failed media downloads and `tootctl media refresh` (#11775) 2019-09-10 15:29:12 +02:00
Eugen Rochko 8674814825
Change tootctl to use inline parallelization instead of Sidekiq (#11776)
- Remove --background option
- Add --concurrency(=5) option
- Add progress bars
2019-09-10 13:48:48 +02:00
Thibaut Girka 1488be7d96 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/home_controller.rb
- app/controllers/shares_controller.rb
- app/javascript/packs/public.js
- app/models/status.rb
- app/serializers/initial_state_serializer.rb
- app/views/home/index.html.haml
- app/views/layouts/public.html.haml
- app/views/public_timelines/show.html.haml
- app/views/shares/show.html.haml
- app/views/tags/show.html.haml
- config/initializers/content_security_policy.rb
- config/locales/en.yml
- config/webpack/shared.js
- package.json
2019-08-19 21:49:35 +02:00
Eugen Rochko 96702e7f67
Add `tootctl cache recount` command (#11597) 2019-08-18 14:55:03 +02:00
Stanislas 3a77090d01 Limit "tootctl accounts follow" to local accounts (#11592)
To (somewhat) limit mass remote follow. Fix #11360
2019-08-17 22:59:40 +02:00
Thibaut Girka e50554391a Merge branch 'master' into glitch-soc/merge-upstream 2019-08-05 13:13:28 +02:00
Darius Kazemi f96f45ef12 Add option to exclude suspended domains/subdomains from tootctl domains crawl (#11454)
* Add "--exclude-suspended" to tootctl domains crawl

This new option ignores any instances suspended server-wide as
well as their associated subdomains. This queries all domain
blocks up front, then runs a regexp on each domain. This improves
performance over what may be the obvious implementation, which is
to ask `DomainBlocks.blocked?(domain)` for each domain -- this
hits the DB many times, slowing things down considerably.

* cleaning up code style

* Compiling regex

* Removing ternary operator
2019-08-03 19:11:09 +02:00
Thibaut Girka 24968d20a0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/directories_controller.rb
- package.json
- yarn.lock
2019-07-30 12:22:33 +02:00
Eugen Rochko 24552b5160
Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Thibaut Girka bca3825c17 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-28 16:28:05 +02:00
mayaeh 0d80f686d8 Add `tootctl preview_cards remove` (#11320)
* Add `tootctl preview_cards remove`

* fix code style

* Remove `Scheduler::PreviewCardsCleanupScheduler` file

* fix code style again
Add exclude case where image_file_name is blank

* Added a function to output confirmation if the specified number of days is less than 2 weeks
2019-07-28 13:48:19 +02:00
Clar Fon a6b4440115 Remove pre from version, add extra suffix variable (#11407) 2019-07-26 07:57:27 +02:00
Thomas Citharel e3a551cbd1 Fix tootctl statuses remove
A ending parenthesis was missing in the bookmark special line.
2019-07-24 15:55:39 +02:00
Thibaut Girka 20520e6158 When cleaning up remote statuses, keep bookmarked ones 2019-07-18 13:34:54 +02:00
Thibaut Girka 3922b518f7 Merge branch 'master' into glitch-soc/merge-upstream 2019-06-28 21:36:50 +02:00
Eugen Rochko e64e6a03dd
Add categories for custom emojis (#11196)
Fix #7940
2019-06-28 15:54:10 +02:00
Thibaut Girka ddd875ad99 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream added audio attachment support
- app/serializers/initial_state_serializer.rb
  Upstream added audio attachment support and how mimetypes are returned
- app/serializers/rest/instance_serializer.rb
  Upstream added a few fields
- config/application.rb
  Upstream added a different paperclip transcoder
2019-06-24 15:02:59 +02:00
Eugen Rochko b5c772c3d4
Bump version to 2.9.2 (#11152) 2019-06-22 17:28:26 +02:00
Eugen Rochko 84f945d64c
Bump version to 2.9.1 (#11143) 2019-06-22 01:51:27 +02:00
Thibaut Girka 60adda7e59 Merge branch 'master' into glitch-soc/merge-upstream 2019-06-13 22:23:20 +02:00
Eugen Rochko c9eeb2e832
Bump version to 2.9.0 (#11074) 2019-06-13 20:19:21 +02:00
Eugen Rochko 275f09ccab
Bump version to 2.9.0rc2 (#11070) 2019-06-13 00:43:59 +02:00
Thibaut Girka 1b0ff4cd69 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/notifications_controller.rb
- app/javascript/packs/public.js
- app/views/settings/preferences/show.html.haml
- app/views/stream_entries/_simple_status.html.haml
- config/locales/simple_form.en.yml
- config/locales/simple_form.pl.yml
- config/navigation.rb
- config/routes.rb
2019-06-10 18:59:53 +02:00
Eugen Rochko 0949c43ab3
Bump version to 2.9.0rc1 (#11004) 2019-06-09 15:53:08 +02:00
Darius Kazemi 4431ce52a6 Specify gzip required in tootctl emoji help (#11000) 2019-06-08 12:43:11 -04:00
Thibaut Girka 20d01a954e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/preferences_controller.rb
- app/lib/user_settings_decorator.rb
- app/models/user.rb

Conflicts due to the addition of a new preference upstream,
“advanced layout”.
2019-05-26 15:41:40 +02:00
Eugen Rochko 5cdb4c483f
Forward port version bumps to 2.8.3 and 2.8.4 (#10819)
* Bump version to 2.8.3

* Bump version to 2.8.4
2019-05-24 15:57:31 +02:00
Thibaut Girka 6badf2d252 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-15 17:11:40 +02:00
ThibG 14f6ce2885 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
Thibaut Girka 68629f2773 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
  minor conflict because of glitch-soc's theming system
- app/controllers/stream_entries_controller.rb
  minor conflict because of glitch-soc's theming system
2019-05-10 17:09:12 +02:00
Jeong Arm 03801fcaa5 Remove custom emojis on "tootctl domains purge" (#10721)
* Remove custom emojis on domains purge

* Change message "Removing" to "Removed"
2019-05-09 22:03:20 +02:00
Eugen Rochko 2508370f44
Bump version to 2.8.2 (#10703) 2019-05-05 17:14:15 +02:00
Thibaut Girka 52a4284fe5 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-04 19:15:50 +02:00
Ushitora Anqou 56880fa76a Add SOURCE_TAG to show source repository's tag (#10698) 2019-05-04 17:39:17 +02:00
Thibaut Girka 33c80e0783 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
2019-05-04 16:37:26 +02:00
Eugen Rochko 8025a41a1f
Add `tootctl cache clear` (#10689) 2019-05-04 01:02:57 +02:00
Eugen Rochko 5f9f610a23
Bump version to 2.8.1 (#10687) 2019-05-04 00:31:06 +02:00
Eugen Rochko d77ee3f276
Fix accounts created through tootctl not being always pre-approved (#10684)
Add `--approve` option to `tootctl accounts modify`
2019-05-03 20:49:27 +02:00
Thibaut Girka a9eaa780f5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/javascript/mastodon/features/compose/components/compose_form.js
  Upstream cleaned up a bit, including on lines in which
  we replaced the hardcoded 500 character limit with a maxChar
  constant. Applied the changes while keeping maxChar instead of 500.
- app/javascript/packs/public.js
  Moved upstream's new animated avatar hover handling in
  app/javascript/core/public.js
- app/javascript/styles/fonts/montserrat.scss
  Upstream fixed local font name, applied those changes.
- app/javascript/styles/fonts/roboto.scss
  Upstream fixed local font name, applied those changes.
- lib/mastodon/version.rb
  Upstream made repo URL configurable, did the same, but
  default to glitch-soc
2019-04-22 20:40:04 +02:00
Sho Kusano 3e6c7f3617 Configrationable repository url (#10600)
* config: Add GITHUB_REPOSITORY for repository name

* config: Add SOURCE_BASE_URL for repository url

* Show source_url and repository name on getting started
2019-04-21 04:41:34 +02:00
Thibaut Girka f5b4e9a54a Merge branch 'master' into glitch-soc/merge-upstream 2019-04-11 15:13:52 +02:00
Eugen Rochko 6afab2587d
Bump version to 2.8.0 (#10550) 2019-04-10 23:32:08 +02:00
Thibaut Girka 5085559b15 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/admin/pending_accounts/index.html.haml
  No real conflict, upstream changes too close to glitch-specific
  theming system changes.
- config/navigation.rb
  Upstream redesigned the settings navigation, took those changes
  and re-inserted the flavours menu.
2019-04-10 09:16:08 +02:00
Eugen Rochko 830c2a9ccb
Bump version to 2.8.0rc3 (#10535) 2019-04-10 04:25:09 +02:00
Eugen Rochko c03f926cf3
Bump version to 2.8.0rc2 (#10501) 2019-04-09 05:04:11 +02:00
Thibaut Girka f5f6d23d55 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/pl.yml
  Conflict caused by new upstream string too close to glitch-specific
  “flavour” string. Took both strings.
2019-04-08 15:57:56 +02:00
Takeshi Umeda 08ba69b538 Add `tootctl accounts reset-relationships` (#10483)
* Add `tootctl accounts reset`

* Rename reset to reset-relationships

* Improve command description
2019-04-08 07:46:55 +02:00
Eugen Rochko 20d301c383
Fix missing long description on `tootctl statuses remove` (#10482) 2019-04-06 04:47:26 +02:00
Eugen Rochko 6689e572f3
Add `tootctl emoji purge` (#10481)
Fix #10441
2019-04-06 04:47:16 +02:00
Eugen Rochko e007c7a99b
Add `tootctl accounts approve` (#10480) 2019-04-06 04:47:05 +02:00
Ben Lubar e6814a332c Fix `tootctl accounts cull` (#10460)
* List the actual accounts that would have been culled during a dry run.

Otherwise, the dry run mode is basically useless.

* Prevent unreachable domains from inheriting the previous status code.

* Update CHANGELOG.md for #10460.
2019-04-04 16:46:27 +02:00
Thibaut Girka 12dae9d583 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/form/admin_settings.rb
- config/locales/ja.yml
2019-04-01 21:28:31 +02:00
Jeong Arm 67eb47e25f Simplify culling and fix race condition (#10440) 2019-04-01 19:17:21 +02:00
Thibaut Girka 4444d8f9b3 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-30 12:36:24 +01:00
Eugen Rochko ac650d7a13
Bump version to 2.8.0rc1 (#10340)
Update AUTHORS.md
Update CHANGELOG.md
2019-03-30 02:13:42 +01:00
Thibaut Girka 050efbc126 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/workers/activitypub/distribute_poll_update_worker.rb
- config/locales/pl.yml
2019-03-28 13:01:33 +01:00
Eugen Rochko 5d41532d94
Add `tootctl search deploy` to avoid ugly rake task syntax (#10403) 2019-03-28 02:16:11 +01:00
Thibaut Girka 07eb31eeac Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/admin/settings/edit.html.haml:
  Conflict due to how the theming settings change.
2019-03-14 17:17:20 +01:00
Eugen Rochko 51e154f5e8
Admission-based registrations mode (#10250)
Fix #6856
Fix #6951
2019-03-14 05:28:30 +01:00
Thibaut Girka f3e7634156 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-11 17:31:02 +01:00
Eugen Rochko 67665023c3
Add `tootctl statuses remove` to sweep unreferenced statuses (#10063)
Query by @tateisu

Fix #1554
2019-03-11 13:45:17 +01:00
Thibaut Girka f513317ba2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- db/schema.rb

Both conflicts are caused by us having extra database columns.
2019-03-05 19:23:16 +01:00
Eugen Rochko 7d5e2dda78
Bump version to 2.7.4 (#10179) 2019-03-05 18:22:46 +01:00
Thibaut Girka 0158937fc2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Dockerfile
  Took upstream version and reverted f13ebd02c9.
  Hopefuly it's not needed anymore.
- app/controllers/api/v1/search_controller.rb
  The conflict was due to us raising the number of results returned.
  Upstream raised it further, so took it.
- config/locales/de.yml
  Took upstream changes to theme translation strings.
- config/locales/gl.yml
  Took upstream changes to theme translation strings.
- config/locales/nl.yml
  Took upstream changes to theme translation strings.
- config/locales/sk.yml
  Took upstream changes to theme translation strings.
2019-02-26 17:04:44 +01:00
Eugen Rochko 96db7191d8
Bump version to 2.7.3 (#10102) 2019-02-23 22:20:20 +01:00
Thibaut Girka ff0576cc2b Merge branch 'master' into glitch-soc/merge-upstream 2019-02-19 20:18:40 +01:00
Eugen Rochko 7853be1f06
Bump version to 2.7.2 (#10067) 2019-02-17 20:24:12 +01:00
Thibaut Girka 42713b9710 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md: kept our version
2019-01-28 21:14:47 +01:00
Eugen Rochko 28866d329b
Bump version to 2.7.1 (#9932) 2019-01-28 04:24:12 +01:00
Stanislas 01bc2f84a9 Fix SUPERUSER postgres command (#9877) 2019-01-27 23:56:07 +01:00
Thibaut Girka 9dfbcf941e Merge branch 'master' into glitch-soc/merge-upstream 2019-01-20 16:49:23 +01:00
Eugen Rochko e9cd3636c6
Fix `tootctl accounts delete` not deleting user record as well (#9874) 2019-01-20 12:30:25 +01:00
Eugen Rochko 7cf85c0133
Bump version to 2.7.0 2019-01-20 00:30:53 +01:00
Thibaut Girka 5e0cf92fd1 Merge branch 'master' into glitch-soc/merge-upstream
No conflicts.
2019-01-19 18:28:37 +01:00
Eugen Rochko 57e79eb1c2
Bump to 2.7.0rc3 2019-01-19 00:01:27 +01:00
Eugen Rochko b59818d4ab
Bump version to 2.7.0rc2 (#9836) 2019-01-18 22:55:01 +01:00
Eugen Rochko 59e3a7f87b
Fix `tootctl domains crawl` with JSON format output crash (#9820)
Fix #9817
2019-01-16 15:47:55 +01:00
Thibaut Girka dbe311f0e9 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/simple_form.pl.yml
2019-01-16 14:20:07 +01:00
Eugen Rochko ee5e24807f
Add `tootctl domains crawl` (#9809) 2019-01-15 09:24:35 +01:00
Thibaut Girka a2a64ecd3e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .eslintrc.yml
  Removed, as upstream removed it.
- app/controllers/admin/statuses_controller.rb
  Minor code cleanup when porting one of our features.
- app/models/account.rb
  Note length validation has changed upstream.
  We now use upstream's validation (dropped legacy glitch-soc
  account metadata stuff) but with configurable limit.
- app/services/post_status_service.rb
  Upstream has added support for scheduled toots, refactoring
  the code a bit. Adapted our changes to this refactoring.
- app/views/stream_entries/_detailed_status.html.haml
  Not a real conflict, changes too close.
- app/views/stream_entries/_simple_status.html.haml
  Not a real conflict, changes too close.
2019-01-10 21:00:30 +01:00
Eugen Rochko 15aed1ee6c
Bump version to 2.7.0rc1 (#9718) 2019-01-08 14:10:54 +01:00
Thibaut Girka 571d219bb9 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts manually resolved:
- app/services/post_status_service.rb
- config/locales/simple_form.pl.yml
- config/routes.rb
- config/webpack/loaders/sass.js
- config/webpack/shared.js
- package.json
- yarn.lock
2019-01-02 15:36:53 +01:00
Eugen Rochko ed12619985
Add tootctl accounts follow ACCT (#9414)
Fix #9369
2019-01-01 16:24:26 +01:00
Eugen Rochko 5d2fc6de32
Add REST API for creating an account (#9572)
* Add REST API for creating an account

The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.

The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.

The method is rate-limited by IP to 5 requests per 30 minutes.

* Redirect users back to app from confirmation if they were created with an app

* Add tests

* Return 403 on the method if registrations are not open

* Require agreement param to be true in the API when creating an account
2018-12-24 19:12:38 +01:00
Thibaut Girka 034ffc079e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/base_controller.rb
  Some refactoring made upstream, no real conflict.
- app/javascript/mastodon/features/compose/components/compose_form.js
  Updated using upstream's code but using maxChars instead of the
  hardcoded length of 500 characters per toot.
- app/javascript/styles/mastodon/components.scss
  Upstream redesigned the onboarding modal. Not sure why we had a
  conflict there.
2018-12-18 16:55:15 +01:00
Eugen Rochko 857e8eb312
Fix tootctl accounts rotate not updating public keys (#9556)
This allowed you to brick your system when running that command, because the accounts would continue to advertise the old public key, but sign things with the new one
2018-12-18 01:22:29 +01:00
Thibaut Girka 1df392819d Merge branch 'master' into glitch-soc/merge-upstream 2018-12-06 12:44:38 +01:00
Eugen Rochko 6ddf0432e7
Improve account suspension speed and completeness (#9290)
- Some associations were missing from the clean-up
- Some attributes were not reset on suspension
- Skip federation and streaming deletes when purging a dead domain
- Move account association definitions to concern
2018-12-03 01:32:08 +01:00
Thibaut Girka d32d33f408 Merge branch 'master' into glitch-soc/merge-upstream 2018-12-02 20:29:14 +01:00
Eugen Rochko 2df5ef18ae
Bump version to 2.6.5 (#9413) 2018-12-02 17:41:38 +01:00
Thibaut Girka a07e37cc17 Merge branch 'master' into glitch-soc/merge-upstream 2018-12-01 21:00:16 +01:00
Eugen Rochko c480da375b
Bump version to 2.6.4 (#9402) 2018-11-30 21:38:09 +01:00
Thibaut Girka bc8411ec87 Merge branch 'master' into glitch-soc/merge-upstream 2018-11-30 20:16:26 +01:00
Eugen Rochko d037e88277
Bump version to 2.6.3 (#9391) 2018-11-30 14:20:15 +01:00
Thibaut Girka 55edfd6e0e Merge branch 'master' into glitch-soc/merge-upstream 2018-11-27 15:10:58 +01:00
Eugen Rochko 637d0ed838
Bump version to 2.6.2 (#9370) 2018-11-27 13:56:47 +01:00
Thibaut Girka f8e07ca5cd Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb

Resolved by taking both changes (not a real conflict, just changes too close
to each other).
2018-11-27 13:23:02 +01:00
Renato "Lond" Cerqueira c66739b418 Touch account on successful response, change char shown when culled (#9293)
Just the color is not enough change since not everyone uses colored
terminals.
Touching the account makes it so that the account is not in the
threshold window in case of running again
2018-11-20 22:25:32 +01:00
mayaeh 8f7e3d3772 Fix "tootctl media remove" can't count the file size (#9288)
* Fixed an issue where "tootctl media remove" can not count the file size.

* Fixed the problem pointed out by codeclimate.
2018-11-16 09:47:40 +01:00
Thibaut Girka 9201398507 Merge branch 'master' into glitch-soc/merge-upstream 2018-11-09 14:56:31 +01:00
Eugen Rochko 16a16f62c7
Fix nil error regression from #9229 in tootctl media remove (#9239)
Fix #9237
2018-11-08 21:55:59 +01:00
Eugen Rochko 3ddb460449
Display amount of freed disk space in tootctl media remove (#9229)
* Display amount of freed disk space in tootctl media remove

Fix #9213

* Fix code style issue
2018-11-08 21:06:26 +01:00
Thibaut Girka 75b48dc177 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-31 10:08:36 +01:00
Eugen Rochko ba06a5f485
Bump version to 2.6.1 (#9172) 2018-10-31 01:04:45 +01:00
Eugen Rochko 50ce347ef9
Bump version to 2.6.0 (#9149)
* Bump version to 2.6.0

* Update CHANGELOG.md
2018-10-30 22:06:59 +01:00
Thibaut Girka 1c448d6474 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb

Upstream reverted something we partially reverted already.
Reverted the rest to match upstream.
2018-10-30 17:52:08 +01:00
Eugen Rochko 66019b0ec4
Bump version to 2.6.0rc4 (#9164)
* Bump version to 2.6.0rc4

* Update CHANGELOG.md
2018-10-30 17:37:49 +01:00
Thibaut Girka ee5132a00e Merge branch 'master' into glitch-soc/merge-upstream 2018-10-29 14:16:35 +01:00
Eugen Rochko e84da282f6
Bump version to 2.6.0rc3 (#9142)
* Bump version to 2.6.0rc3

* Update CHANGELOG.md
2018-10-29 14:15:54 +01:00
Thibaut Girka b00f60f1d3 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-28 08:37:49 +01:00
Eugen Rochko 6f78500d4f
Do not remove "dead" domains in tootctl accounts cull (#9108)
Leave `tootctl accounts cull` to simply check removed accounts from
live domains, and skip temporarily unavailable domains, while listing
them in the final output for further action.

Add `tootctl domains purge DOMAIN` to be able to purge a domain from
that list manually
2018-10-27 22:56:16 +02:00
Thibaut Girka 5bb8563f6c Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/base_controller.rb
- app/controllers/filters_controller.rb
- app/controllers/invites_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/exports_controller.rb
- app/controllers/settings/follower_domains_controller.rb
- app/controllers/settings/migrations_controller.rb
- app/controllers/settings/notifications_controller.rb
- app/controllers/settings/preferences_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
- app/javascript/packs/public.js
- app/views/settings/profiles/show.html.haml

Conflicts were mostly due to the addition of body classes to the settings page,
this was caused by rejecting upstream changes for most of those files and
modifying Settings::BaseController instead.

Another cause of conflicts was the deletion of client-side checking of
display name / bio length, this was modified in app/javascript/core/settings.js
instead.
2018-10-26 20:41:43 +02:00
Sascha b9d7021c1b cli: set exit_on_failure for all CLI classes (#9094) 2018-10-25 16:05:33 +02:00
Eugen Rochko 2f0797bdbd
Bump version to 2.6.0rc2 (#9087)
* Bump version to 2.6.0rc2

* Update CHANGELOG.md
2018-10-25 04:42:25 +02:00
Thibaut Girka dcded13a99 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .github/ISSUE_TEMPLATE/bug_report.md
  Took our version.
- CONTRIBUTING.md
  Updated the embedded copy of upstream's version.
- README.md
  Took our version.
- app/policies/status_policy.rb
  Not a real conflict, took code from both.
- app/views/layouts/embedded.html.haml
  Added upstream's changes (dns-prefetch) and fixed
  `%body.embed`
- app/views/settings/preferences/show.html.haml
  Reverted some of upstream changes, as we have a
  page dedicated for flavours and skins.
- config/initializers/content_security_policy.rb
  Kept our version of the CSP.
- config/initializers/doorkeeper.rb
  Not a real conflict, took code from both.
2018-10-22 17:51:38 +02:00
Eugen Rochko f5b8bd4392
Fix cull tripping on nil in last_webfingered_at (#9051)
Fix #8741
2018-10-22 16:58:08 +02:00
Jeong Arm 2e18ad74dc Fix tootctl cull on dead servers (#9041)
* Delete first 9 accounts on dead servers

* Clean up code by moving dead server culling to the end
2018-10-21 22:52:27 +02:00
Jeong Arm 68f0e4d54e Handle if username is not found on tootctl feeds build (#9040) 2018-10-21 16:42:22 +02:00
Eugen Rochko f468bfb830
Bump version to 2.6.0rc1 (#9025)
* Bump version to 2.6.0rc1

* Update AUTHORS.md

* Update CHANGELOG.md
2018-10-21 00:49:36 +02:00
Eugen Rochko ef43f1d2ca
Bump version to 2.5.2 (#8960) 2018-10-12 00:24:09 +02:00
Eugen Rochko e6c01171de
Bump version to 2.5.1 (#8953) 2018-10-11 02:29:32 +02:00
Thibaut Girka d17844e6d1 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-08 13:51:33 +02:00
ashleyhull-versent f194857ac9 rubocop issues - Cleaning up (#8912)
* cleanup pass

* undo mistakes

* fixed.

* revert
2018-10-08 04:50:11 +02:00
Thibaut Girka 00fcdebed7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	db/migrate/20170716191202_add_hide_notifications_to_mute.rb
	spec/controllers/application_controller_spec.rb

Took our version, upstream changes were only minor style linting.
2018-10-05 15:23:57 +02:00
Eugen Rochko 186024a058
Add tootctl settings registrations open (#8829) 2018-10-04 16:05:54 +02:00
Thibaut Girka c91d9b7389 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/lib/user_settings_decorator.rb
	app/models/user.rb
	app/serializers/initial_state_serializer.rb
	app/views/stream_entries/_simple_status.html.haml
	config/locales/simple_form.en.yml
	config/locales/simple_form.ja.yml
	config/locales/simple_form.pl.yml
	config/routes.rb
2018-10-01 12:43:20 +02:00
Eugen Rochko ac07bfb018 Fix tootctl accounts reattaching not unsuspending deleted account (#8812) 2018-09-29 18:40:40 +09:00
Eugen Rochko 38f5f9cf5b
Add tootctl accounts backup (#8811) 2018-09-28 03:34:24 +02:00
Thibaut Girka 3b05eb8002 Add -glitch suffix to version string
Fixes #715
2018-09-26 11:11:30 +02:00
Thibaut Girka 91bef4759f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	Vagrantfile
	app/javascript/packs/public.js
	app/views/admin/settings/edit.html.haml
	app/views/settings/preferences/show.html.haml
	app/views/settings/profiles/show.html.haml
	config/locales/es.yml
	config/locales/simple_form.en.yml
	config/webpack/configuration.js
	config/webpack/loaders/babel.js
	package.json
	yarn.lock

Split new additions to app/javascript/packs/public.js to
app/javascript/core/settings.js
2018-09-19 21:46:01 +02:00
Eugen Rochko 6a3f9b7e53
Move more tasks to tootctl (#8675)
* Move more tasks to tootctl

- tootctl feeds build
- tootctl feeds clear
- tootctl accounts refresh

Clean up exit codes and help messages

* Move user modifying to tootctl

* Improve user modification through CLI, rename commands

add -> create
mod -> modify
del -> delete

To remove ambiguity

* Fix code style issues

* Fix not being able to unset admin/mod role
2018-09-14 17:42:22 +02:00
luzpaz 40dd19be37 Misc. typos (#8694)
Found via `codespell -q 3 --skip="./app/javascript/mastodon/locales,./config/locales"`
2018-09-14 00:53:09 +02:00
Thibaut Girka cd99255698 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/controllers/oauth/authorizations_controller.rb

Just two changes being too close to one another.
Took both.
2018-09-11 16:51:26 +02:00
Eugen Rochko 47ea318479
tootctl accounts [add|del|cull] (#8642)
* CLI interface for creating/deleting local users

- tootctl accounts add USERNAME
- tootctl accounts del USERNAME

* Add CLI interface for culling remote users that no longer exist

- tootctl accounts cull
2018-09-09 13:33:36 +02:00
Thibaut Girka 6c233a04a0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	README.md

Discarded upstream changes to the README, kept our version.
2018-09-03 14:33:06 +02:00
Eugen Rochko b65a9b7243
Bump version to 2.5.0 2018-09-02 22:32:27 +02:00
Thibaut Girka a74a7d5fe3 Merge branch 'master' into glitch-soc/merge-upstream 2018-09-01 17:23:46 +02:00
Eugen Rochko 08721170da
Bump version to 2.5.0rc2 2018-09-01 03:37:43 +02:00
Thibaut Girka 6f75a9001f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/stream_entries/_simple_status.html.haml
- config/locales/nl.yml

Deleted unused translation strings (themes) and adapted minor changes to
_simple_status.html.haml
2018-08-31 19:10:57 +02:00
mayaeh 18eb565755 Add --verbose and --dry-run option to tootctl media remove (#8519)
* Add --verbose and --dry-run options to CLI interface for removing remote media.

* Fix coding style problem.
2018-08-31 03:46:13 +02:00
mayaeh cc26fd71ac Fix CLI interface for removing remote media (#8506)
* Fix that can't delete media files even if "tootctl media remove" execute when "--background" not attached.

* Revert

This reverts commit 5aa7e09645b27bae38a26030148b23e553ee2662.

* Change to obtain and pass all columns when "--background" option is false.
2018-08-29 18:35:09 +02:00
Thibaut Girka 53b7bb9d04 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/controllers/api/v1/mutes_controller.rb
	config/locales/simple_form.pl.yml
2018-08-27 18:09:28 +02:00
Eugen Rochko cabdbb7f9c
Add CLI task for rotating keys (#8466)
* If an Update is signed with known key, skip re-following procedure

Because it means the remote actor did *not* lose their database

* Add CLI method for rotating keys

    bin/tootctl accounts rotate [USERNAME]

Generates a new RSA key per account and sends out an Update activity
signed with the old key.

* Key rotation: Space out Update fan-outs every 5 minutes per 1000 accounts

* Skip suspended accounts in key rotation
2018-08-26 20:21:03 +02:00
Thibaut Girka be20eb4b2f Merge branch 'master' into glitch-soc/merge-upstream 2018-08-26 18:21:33 +02:00
Eugen Rochko b378b4c5c7
Add CLI interface for importing custom emoji (#8437)
bin/tootctl emoji import PATH_TO_TAR

Fix #8435
2018-08-26 16:53:06 +02:00
Thibaut Girka 36393e1d2b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/views/layouts/application.html.haml

Edited:
        app/helpers/application_helper.rb
        app/views/admin/domain_blocks/new.html.haml

Conflict wasn't really one, just two changes too close to one another.
Edition was to adapt the class names for themes to class names for
skins and flavours.

Also edited app/views/admin/domain_blocks/new.html.haml to strip the
duplicate admin pack inclusion thing.
2018-08-26 14:23:24 +02:00
Eugen Rochko 793eea2982
Add improved CLI interface for removing remote media (#8411)
./bin/tootctl media remove --days 7 --background

Make the old rake task point to it
2018-08-25 13:25:39 +02:00
Thibaut Girka 246c313d45 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/controllers/application_controller.rb

Changed instance theme selection by instance flavour selection.
2018-08-24 15:10:34 +02:00
Eugen Rochko 23752639b2
Bump version to 2.5.0rc1 (#8356) 2018-08-23 23:30:57 +02:00
Thibaut Girka c1c514ca70 Merge branch 'master' into glitch-soc/merge-upstream
Manually-resolved conflicts:
	.circleci/config.yml
	app/controllers/accounts_controller.rb
	app/controllers/auth/passwords_controller.rb
	app/controllers/statuses_controller.rb
	app/javascript/packs/public.js
	app/models/media_attachment.rb
	app/views/stream_entries/_content_spoiler.html.haml
	app/views/stream_entries/_media.html.haml
	config/locales/en.yml
	config/locales/ja.yml
	config/locales/pl.yml
	lib/mastodon/version.rb

Some content from app/javascript/packs/public.js has been split to
app/javascript/core/settings.js.

Translation strings for glitch-soc's keyword mutes were dropped.

Everything else was mostly “take both”.
2018-08-01 00:18:13 +02:00
Eugen Rochko bb71538bb5
Redesign public profiles and toots (#8068) 2018-07-28 19:25:33 +02:00
Thibaut Girka 6f0f583abe Merge branch 'master' into glitch-soc/merge-upstream 2018-07-11 17:57:08 +02:00
Eugen Rochko 7ac5151b74
Bump version to 2.4.3 2018-07-11 11:21:27 +02:00
Eugen Rochko 89dc0c98ee
Bump version to 2.4.3rc3 2018-07-11 01:31:23 +02:00
Thibaut Girka d392020da6 Merge branch 'master' into glitch-soc/tentative-merge
Conflicts:
	README.md
	app/controllers/statuses_controller.rb
	app/lib/feed_manager.rb
	config/navigation.rb
	spec/lib/feed_manager_spec.rb

Conflicts were resolved by taking both versions for each change.
This means the two filter systems (glitch-soc's keyword mutes and tootsuite's
custom filters) are in place, which will be changed in a follow-up commit.
2018-07-09 07:13:59 +02:00
Eugen Rochko 451e585b97
Bump version to 2.4.3rc2 2018-07-07 22:23:52 +02:00
Eugen Rochko cb36ab9a10
Bump version to 2.4.3rc1 2018-07-05 21:08:03 +02:00
Thibaut Girka ab5f450700 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/models/user.rb

Resolved by adding :default_language to user settings fields
2018-06-21 20:49:57 +02:00
Eugen Rochko d017f240f9
Bump version to 2.4.2 2018-06-19 01:45:00 +02:00
Eugen Rochko 86cce466b6
Bump version to 2.4.2rc3 2018-06-19 00:19:54 +02:00
Eugen Rochko a04dca36a3
Bump version to 2.4.2rc2 2018-06-18 23:01:52 +02:00
Eugen Rochko 3b8023f9f9
Bump version to 2.4.2rc1 2018-06-15 20:38:41 +02:00
Jenkins 4d8d1fa129 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-06-12 13:17:21 +00:00
Eugen Rochko 0979d4275a
Bump version to 2.4.1 2018-06-10 17:20:56 +02:00
Eugen Rochko 35ae960122
Bump version to 2.4.1rc4 2018-06-10 00:39:56 +02:00
David Yip 8142bd2553
Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
 	app/models/status.rb

The conflict in the Status model was due to
5bf5003384.
It was resolved by accepting tootsuite's changes.
2018-06-07 05:13:49 -05:00
Eugen Rochko 158cd7ee74
Bump version to 2.4.1rc3 2018-06-06 21:27:06 +02:00
Eugen Rochko fab65fb7e5
Bump version to 2.4.1rc2 2018-06-05 11:53:30 +00:00
Jenkins 165b5dc7f5 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-05-31 01:17:25 +00:00
Eugen Rochko a29f196f95
Bump version to 2.4.1rc1 2018-05-30 19:16:41 +02:00
Thibaut Girka e0ef89c73f Merge branch 'master' into glitch-soc/merge-upstream 2018-05-23 16:05:19 +02:00
Eugen Rochko 7745a22ec7
Bump version to 2.4.0 2018-05-22 19:39:35 +02:00
Thibaut Girka a4c9bda771 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/javascript/styles/mastodon-light.scss
	config/locales/en.yml
	config/locales/fr.yml
	config/locales/simple_form.pl.yml
	config/themes.yml

Conflicts resolved by deleting config/themes.yml,
marking app/javascript/styles/mastodon-light.scss as added,
and taking all new translation strings, not removing anything from
them.
2018-05-21 16:40:02 +02:00
Eugen Rochko 292c987522
Bump version to 2.4.0rc5 2018-05-21 16:05:31 +02:00
Thibaut Girka b481e4fac1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	config/locales/simple_form.ja.yml
2018-05-19 22:41:10 +02:00
Eugen Rochko bcbb6aa46f
Bump version to 2.4.0rc4 2018-05-19 18:38:10 +02:00
Jenkins 4c0770d6c8 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-05-18 00:17:23 +00:00
MIYAGI Hikaru 919eef3098 User agent for WebFinger (#7531)
* User agent for WebFinger

* local_domain → web_domain

* 'http' is away accidentally...
2018-05-18 01:47:22 +02:00
Thibaut Girka ba7ee67498 Merge branch 'master' into glitch-soc/merge 2018-05-14 20:51:50 +02:00
Eugen Rochko 8756fd1e82
Bump version to 2.4.0rc3 2018-05-14 17:46:44 +02:00
Eugen Rochko d75b63e4fb
Bump version to 2.4.0rc2 2018-05-14 14:49:38 +02:00
Thibaut Girka 1c9c0167b7 Merge branch 'master' into glitch-soc/master
Conflicts:
	app/models/account.rb
	app/views/accounts/_header.html.haml
2018-05-10 00:03:28 +02:00
Eugen Rochko 50ed1e83ac
Bump version to 2.4.0rc1 (#7283) 2018-05-06 11:50:00 +02:00
David Yip da70208b45
Merge remote-tracking branch 'origin/master' into merge-upstream 2018-05-04 03:23:57 -05:00
David Yip c816701550
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	.travis.yml
 	Gemfile.lock
 	README.md
 	app/controllers/settings/follower_domains_controller.rb
 	app/controllers/statuses_controller.rb
 	app/javascript/mastodon/locales/ja.json
 	app/lib/feed_manager.rb
 	app/models/media_attachment.rb
 	app/models/mute.rb
 	app/models/status.rb
 	app/services/mute_service.rb
 	app/views/home/index.html.haml
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/ca.yml
 	config/locales/en.yml
 	config/locales/es.yml
 	config/locales/fr.yml
 	config/locales/nl.yml
 	config/locales/pl.yml
 	config/locales/pt-BR.yml
 	config/themes.yml
2018-05-03 17:23:44 -05:00
ThibG a24605961a Fixes/do not override timestamps (#7336)
* Revert "Fixes/do not override timestamps (#7331)"

This reverts commit 581a5c9d29.

* Document Snowflake ID corner-case a bit more

Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.

Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.

* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
2018-05-03 23:02:46 +02:00
Eugen Rochko 63553c6b5c
Add support for separate Redis for cache (#7272)
* Add support for separate Redis for cache

CACHE_REDIS_URL to allow using a different Redis server for cache
purposes, with cache-specific configuration such as key eviction

* Fix code style issues
2018-04-27 01:37:59 +02:00
Yamagishi Kazutoshi 084cf0babf Add extract_foreign_key_action to Mastodon::MigrationHelpers (#7195) 2018-04-20 12:21:28 +02:00
Thibaut Girka 88bcfaeb18 Point to glitch-soc source and not tootsuite on the public page
(hello AGPLv3)
2018-04-08 15:43:26 +02:00
Eugen Rochko 2a90da1837
Fix UniqueUsernameValidator comparison (#6926)
Comparison was downcasing only one side, therefore if previously
existing account had a non-lowercase spelling, it would be ignored
when checking for duplicates.

New rake task `mastodon:maintenance:find_duplicate_usernames` will
help find constraint violations that might have occured from the
presence of this bug.

Bump version to 2.3.3
2018-03-27 04:33:57 +02:00
Eugen Rochko 6cc432bbc4
Bump version to 2.3.2 2018-03-22 14:13:46 +01:00
Eugen Rochko dafae9818d
Bump version to 2.3.2rc5 2018-03-22 11:31:52 +01:00
Eugen Rochko f64af6473f
Bump version to 2.3.2rc4 2018-03-20 23:49:24 +01:00
Eugen Rochko 74c39fada0
Bump version to 2.3.2rc3 2018-03-19 12:20:57 +01:00
Eugen Rochko 092f1df9d0
Bump version to 2.3.2rc2 2018-03-17 15:28:52 +01:00
Eugen Rochko cdf8b92fea
Bump version to 2.3.2rc1 2018-03-17 14:07:00 +01:00
Eugen Rochko 2ef1ce1182
Bump version to 2.3.1 2018-03-13 15:50:15 +01:00
Eugen Rochko 79d3a8553f
Bump version to 2.3.1rc3 2018-03-13 09:21:21 +01:00
Eugen Rochko f89ff65ec7
Bump version to 2.3.1rc2 2018-03-12 12:48:49 +01:00
Eugen Rochko ab27dccba5
Bump version to 2.3.1rc1 2018-03-11 15:13:13 +01:00
Eugen Rochko 6e309f5e45
Bump version to 2.3.0 2018-03-09 12:59:58 +01:00
Eugen Rochko bd077ad7d9
Bump version to 2.3.0rc3 2018-03-08 11:19:02 +01:00
Eugen Rochko 4ca60c665e
Bump version to 2.3.0rc2 2018-03-07 12:06:23 +01:00
Eugen Rochko 64f2ada5d4
Bump version to 2.3.0rc1 2018-03-01 20:50:23 +01:00
Eugen Rochko e14c20582f
Bump version to 2.2.0 2018-01-29 01:12:36 +01:00
Eugen Rochko 97dcfb0f50
ASSET_HOST is wrong env variable. Fix to CDN_HOST (#6372) 2018-01-29 01:06:39 +01:00
Eugen Rochko e6db3427b7
Bump to 2.2.0rc2 2018-01-24 04:16:30 +01:00
Eugen Rochko b1daa71da5
Fix #6311: Replace relative URLs in CSS only for Premailer (#6335) 2018-01-24 02:57:14 +01:00
Eugen Rochko 0b7a0d15c7
Bump version to 2.2.0rc1 2018-01-20 16:27:19 +01:00
Eugen Rochko 872a0d5bd8
Improve HTML e-mails based on Litmus tests (#6301)
* Use PNG images in HTML e-mails

* Make webpack use URLs with host so fonts load inside HTML e-mails

Convert this back to a relative URL in the premailer CSS loader
since local requests are quicker

* Improve responsive design

* Add missing PNG icon
2018-01-20 01:32:21 +01:00
Eugen Rochko 5276c0a090
HTML e-mails for UserMailer (#6256)
- premailer gem to turn CSS into inline styles automatically
- rework UserMailer templates
- reword UserMailer templates
2018-01-16 03:29:11 +01:00
Eugen Rochko ff6ca8bdc6
Bump version to 2.1.3 2018-01-08 19:15:31 +01:00
Eugen Rochko e9822a4e4e
Bump version to 2.1.2 2018-01-05 04:52:06 +01:00
Eugen Rochko 7571c37c99
Bump version to 2.1.1 (#6164) 2018-01-04 16:40:26 +01:00
Daigo 3 Dango a434d9c0cc Remove period from the version number (#6039)
2.1.0. -> 2.1.0
2017-12-15 21:38:25 +01:00
Eugen Rochko a29432f0cd
Bump version to 2.1.0 🎆 2017-12-15 19:14:57 +01:00
Eugen Rochko 098c7d27fe
Bump version to 2.1.0rc6 2017-12-15 02:00:28 +01:00
Eugen Rochko 7cad926401
Bump version to 2.1.0rc5 2017-12-13 20:53:39 +01:00
Eugen Rochko 57fcc21a86
Bump version to 2.1.0rc4 2017-12-13 12:45:12 +01:00
Eugen Rochko 2526ef10c2
Bump version to 2.1.0rc3 2017-12-09 02:42:59 +01:00
Eugen Rochko 369f40bb9f
Bump version to 2.1.0rc2 2017-12-08 01:47:08 +01:00
Eugen Rochko f9a0d8f2b9
Bump version to 2.1.0rc1 (#5834) 2017-12-06 16:13:32 +01:00
Yamagishi Kazutoshi b21db9bbde Using double splat operator (#5859) 2017-12-06 11:41:57 +01:00
ysksn 9dd5e329ab Remove empty strings (#5732) 2017-11-17 10:52:30 +09:00
Eugen Rochko 3810d98cd8 Bump version to 2.0.0 🐘 2017-10-18 13:53:56 +02:00
Eugen Rochko 637ea3bb5b Bump version to 2.0.0rc4 2017-10-17 23:16:35 +02:00
Eugen Rochko bc89995f65 Bump version to 2.0.0rc3 2017-10-16 16:29:00 +02:00
aschmitz e5d8166a12 Fix #5329 (#5332)
This fixes #5329, which occurred when using MigrationHelpers on a
table that Postgres estimated to be non-empty when it was actually
empty.
2017-10-11 21:20:39 +02:00
Eugen Rochko 7c33da45f0 Bump version to 2.0.0rc2 2017-10-10 20:48:26 +02:00
Eugen Rochko 0aa810f9c8 Bump version to 2.0.0rc1 (#5209) 2017-10-08 22:03:44 +02:00
Eugen Rochko 0717d9b3e6 Set snowflake IDs for backdated statuses (#5260)
- Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity
- Skip for statuses coming from inbox, aka delivered in real-time
- Skip for statuses that claim to be from the future
2017-10-08 17:34:34 +02:00
Eugen Rochko eb5ac23434 Clean up code style of Mastodon::TimestampId module (#5232)
* Clean up code style of Mastodon::TimestampId module

* Update brakeman config
2017-10-06 03:42:21 +02:00
aschmitz 468523f4ad Non-Serial ("Snowflake") IDs (#4801)
* Use non-serial IDs

This change makes a number of nontrivial tweaks to the data model in
Mastodon:

* All IDs are now 8 byte integers (rather than mixed 4- and 8-byte)
* IDs are now assigned as:
  * Top 6 bytes: millisecond-resolution time from epoch
  * Bottom 2 bytes: serial (within the millisecond) sequence number
  * See /lib/tasks/db.rake's `define_timestamp_id` for details, but
    note that the purpose of these changes is to make it difficult to
    determine the number of objects in a table from the ID of any
    object.
* The Redis sorted set used for the feed will have values used to look
  up toots, rather than scores. This is almost always the same as the
  existing behavior, except in the case of boosted toots. This change
  was made because Redis stores scores as double-precision floats,
  which cannot store the new ID format exactly. Note that this doesn't
  cause problems with sorting/pagination, because ZREVRANGEBYSCORE
  sorts lexicographically when scores are tied. (This will still cause
  sorting issues when the ID gains a new significant digit, but that's
  extraordinarily uncommon.)

Note a couple of tradeoffs have been made in this commit:

* lib/tasks/db.rake is used to enforce many/most column constraints,
  because this commit seems likely to take a while to bring upstream.
  Enforcing a post-migrate hook is an easier way to maintain the code
  in the interim.
* Boosted toots will appear in the timeline as many times as they have
  been boosted. This is a tradeoff due to the way the feed is saved in
  Redis at the moment, but will be handled by a future commit.

This would effectively close Mastodon's #1059, as it is a
snowflake-like system of generating IDs. However, given how involved
the changes were simply within Mastodon, it may have unexpected
interactions with some clients, if they store IDs as doubles
(or as 4-byte integers). This was a problem that Twitter ran into with
their "snowflake" transition, particularly in JavaScript clients that
treated IDs as JS integers, rather than strings. It therefore would be
useful to test these changes at least in the web interface and popular
clients before pushing them to all users.

* Fix JavaScript interface with long IDs

Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.

BREAKING CHANGE:

The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)

Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.

1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html

* Restructure feed pushes/unpushes

This was necessary because the previous behavior used Redis zset scores
to identify statuses, but those are IEEE double-precision floats, so we
can't actually use them to identify all 64-bit IDs. However, it leaves
the code in a much better state for refactoring reblog handling /
coalescing.

Feed-management code has been consolidated in FeedManager, including:

* BatchedRemoveStatusService no longer directly manipulates feed zsets
* RemoveStatusService no longer directly manipulates feed zsets
* PrecomputeFeedService has moved its logic to FeedManager#populate_feed

(PrecomputeFeedService largely made lots of calls to FeedManager, but
didn't follow the normal adding-to-feed process.)

This has the effect of unifying all of the feed push/unpush logic in
FeedManager, making it much more tractable to update it in the future.

Due to some additional checks that must be made during, for example,
batch status removals, some Redis pipelining has been removed. It does
not appear that this should cause significantly increased load, but if
necessary, some optimizations are possible in batch cases. These were
omitted in the pursuit of simplicity, but a batch_push and batch_unpush
would be possible in the future.

Tests were added to verify that pushes happen under expected conditions,
and to verify reblog behavior (both on pushing and unpushing). In the
case of unpushing, this includes testing behavior that currently leads
to confusion such as Mastodon's #2817, but this codifies that the
behavior is currently expected.

* Rubocop fixes

I could swear I made these changes already, but I must have lost them
somewhere along the line.

* Address review comments

This addresses the first two comments from review of this feature:

https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336735
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336931

This adds an optional argument to FeedManager#key, the subtype of feed
key to generate. It also tests to ensure that FeedManager's settings are
such that reblogs won't be tracked forever.

* Hardcode IdToBigints migration columns

This addresses a comment during review:
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139337452

This means we'll need to make sure that all _id columns going forward
are bigints, but that should happen automatically in most cases.

* Additional fixes for stringified IDs in JSON

These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)

Adding the following to .eslintrc.yml will identify casts to numbers:

~~~
  no-restricted-syntax:
  - warn
  - selector: UnaryExpression[operator='+'] > :not(Literal)
    message: Avoid the use of unary +
  - selector: CallExpression[callee.name='Number']
    message: Casting with Number() may coerce string IDs to numbers
~~~

The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.

* Only implement timestamp IDs for Status IDs

Per discussion in #4801, this is only being merged in for Status IDs at
this point. We do this in a migration, as there is no longer use for
a post-migration hook. We keep the initialization of the timestamp_id
function as a Rake task, as it is also needed after db:schema:load (as
db/schema.rb doesn't store Postgres functions).

* Change internal streaming payloads to stringified IDs as well

This is equivalent to 591a9af356faf2d5c7e66e3ec715502796c875cd from
#5019, with an extra change for the addition to FeedManager#unpush.

* Ensure we have a status_id_seq sequence

Apparently this is not a given when specifying a custom ID function,
so now we ensure it gets created. This uses the generic version of this
function to more easily support adding additional tables with timestamp
IDs in the future, although it would be possible to cut this down to a
less generic version if necessary. It is only run during db:schema:load
or the relevant migration, so the overhead is extraordinarily minimal.

* Transition reblogs to new Redis format

This provides a one-way migration to transition old Redis reblog entries
into the new format, with a separate tracking entry for reblogs.

It is not invertible because doing so could (if timestamp IDs are used)
require a database query for each status in each users' feed, which is
likely to be a significant toll on major instances.

* Address review comments from @akihikodaki

No functional changes.

* Additional review changes

* Heredoc cleanup

* Run db:schema:load hooks for test in development

This matches the behavior in Rails'
ActiveRecord::Tasks::DatabaseTasks.each_current_configuration, which
would otherwise break `rake db:setup` in development.

It also moves some functionality out to a library, which will be a good
place to put additional related functionality in the near future.
2017-10-04 09:56:37 +02:00
aschmitz 97c02c3389 Make IdsToBigints (mostly!) non-blocking (#5088)
* Make IdsToBigints (mostly!) non-blocking

This pulls in GitLab's MigrationHelpers, which include code to make
column changes in ways that Postgres can do without locking. In general,
this involves creating a new column, adding an index and any foreign
keys as appropriate, adding a trigger to keep it populated alongside
the old column, and then progressively copying data over to the new
column, before removing the old column and replacing it with the new
one.

A few changes to GitLab's MigrationHelpers were necessary:

* Some changes were made to remove dependencies on other GitLab code.
* We explicitly wait for index creation before forging ahead on column
  replacements.
* We use different temporary column names, to avoid running into index
  name length limits.
* We rename the generated indices back to what they "should" be after
  replacing columns.
* We rename the generated foreign keys to use the new column names when
  we had to create them. (This allows the migration to be rolled back
  without incident.)

# Big Scary Warning

There are two things here that may trip up large instances:

1. The change for tables' "id" columns is not concurrent. In
   particular, the stream_entries table may be big, and does not
   concurrently migrate its id column. (On the other hand, x_id type
   columns are all concurrent.)
2. This migration will take a long time to run, *but it should not
   lock tables during that time* (with the exception of the "id"
   columns as described above). That means this should probably be run
   in `screen` or some other session that can be run for a long time.
   Notably, the migration will take *longer* than it would without
   these changes, but the website will still be responsive during that
   time.

These changes were tested on a relatively large statuses table (256k
entries), and the service remained responsive during the migration.
Migrations both forward and backward were tested.

* Rubocop fixes

* MigrationHelpers: Support ID columns in some cases

This doesn't work in cases where the ID column is referred to as a
foreign key by another table.

* MigrationHelpers: support foreign keys for ID cols

Note that this does not yet support foreign keys on non-primary-key
columns, but Mastodon also doesn't yet have any that we've needed to
migrate.

This means we can perform fully "concurrent" migrations to change ID
column types, and the IdsToBigints migration can happen with effectively
no downtime. (A few operations require a transaction, such as renaming
columns or deleting them, but these transactions should not block for
noticeable amounts of time.)

The algorithm for generating foreign key names has changed with this,
and therefore all of those changed in schema.rb.

* Provide status, allow for interruptions

The MigrationHelpers now allow restarting the rename of a column if it
was interrupted, by removing the old "new column" and re-starting the
process.

Along with this, they now provide status updates on the changes which
are happening, as well as indications about when the changes can be
safely interrupted (when there are at least 10 seconds estimated to be
left before copying data is complete).

The IdsToBigints migration now also sorts the columns it migrates by
size, starting with the largest tables. This should provide
administrators a worst-case scenario estimate for the length of
migrations: each successive change will get faster, giving admins a
chance to abort early on if they need to run the migration later. The
idea is that this does not force them to try to time interruptions
between smaller migrations.

* Fix column sorting in IdsToBigints

Not a significant change, but it impacts the order of columns in the
database and db/schema.rb.

* Actually pause before IdsToBigints
2017-10-02 21:28:59 +02:00
Eugen Rochko efec507230 Bump to 1.6.1 2017-09-16 03:08:29 +02:00
abcang 1aad015bbb Revert unique retry job (#4937)
* Revert "Enable UniqueRetryJobMiddleware even when called from sidekiq worker (#4836)"

This reverts commit 6859d4c028.

* Revert "Do not execute the job with the same arguments as the retry job (#4814)"

This reverts commit be7ffa2d75.
2017-09-14 15:12:43 +02:00
Eugen Rochko ab71cf4593 Bump to 1.6.0 2017-09-10 15:10:03 +02:00
Eugen Rochko 4819e2913d Bump version to 1.6.0rc5 2017-09-10 10:26:51 +02:00
Eugen Rochko 2ff7146b6d Bump version to 1.6.0rc4 2017-09-09 14:53:49 +02:00
Eugen Rochko dd5cb5085c Bump version to 1.6.0rc2 2017-09-06 19:02:03 +02:00
abcang be7ffa2d75 Do not execute the job with the same arguments as the retry job (#4814) 2017-09-05 20:56:20 +02:00
Eugen Rochko 9dd8dff683 Bump version to 1.6.0rc1 (#4768) 2017-09-03 01:12:05 +02:00
Daigo 3 Dango 696c2c6f2f Add Mastodon::Source.url (#4643)
* Add Mastodon::Source.url

* Update spec

* Refactor

Move things frmo Mastodon::Source to Mastodon::Version
2017-08-22 22:54:19 +02:00
Eugen Rochko 41c3389d76 Bump to 1.5.1 2017-08-06 23:53:25 +02:00
Eugen Rochko 559fd08845 Bump to 1.5.0 2017-08-01 15:12:07 +02:00
Eugen Rochko 9b247c3d88 Bump to 1.5.0rc3 2017-07-31 15:28:36 +02:00
Eugen Rochko fdb65dcbee Bump to 1.5.0rc2 2017-07-28 17:22:41 +02:00
Eugen Rochko a549d1ae6b Bump to 1.5.0rc1 (#4318) 2017-07-24 16:21:08 +02:00
Eugen Rochko b6a19e7b89 Bump version to 1.4.7 2017-06-28 17:44:17 +02:00
Eugen Rochko 7b13e6efc2 Bump version to 1.4.6 2017-06-23 17:02:14 +02:00
Eugen Rochko 1585b0c6cc Bump version to 1.4.4 2017-06-20 21:32:37 +02:00
Eugen Rochko 947887f261 Bump version to 1.4.3 2017-06-15 03:03:42 +02:00
Eugen Rochko 380b20eed6 Bump version to 1.4.2 2017-06-08 15:30:43 +02:00
Eugen Rochko 8963f8c3c2 Bump version to 1.4.1 2017-05-28 19:30:38 +02:00
Eugen Rochko 7b23f79d41 Bump version to 1.4.0.6 2017-05-28 16:32:53 +02:00
Eugen Rochko e0e12b0fee Bump version to 1.4.0.5 2017-05-27 16:56:47 +02:00
Eugen Rochko 2d97c898f2 Bump version to 1.4.0.4 2017-05-26 14:16:04 +02:00
Eugen Rochko b51398d0dd Bump version 2017-05-22 21:38:19 +02:00
Clworld df92f010ad Set config.cache_store in environments file. (#3219)
* Set config.cache_store in application.rb

* Set config.cache_store in environments.

* fix code format.
2017-05-22 15:01:02 +02:00
Eugen Rochko 756db8103a Bump version to 1.4.0.2 (#3190) 2017-05-20 23:30:20 +02:00
Eugen Rochko 01e011bc90 Bump version to 1.3.2 (#2623) 2017-04-29 19:26:32 +02:00
Ash Furrow 5eef9dab80 Update version to 1.3.1 (#2571) 2017-04-28 09:07:48 -04:00
Eugen Rochko b48c9013aa Bump version, improve how version is stored for better commit history (#2526) 2017-04-27 15:22:19 +02:00
Ash Furrow a0ed88a99b Adds version to about/more and API (#2181)
* Adds version.

* Cleans up code.

* Removes standalone endpoint and adds version to instance endpoint.

* Addresses feedback from #2181.
2017-04-21 03:30:59 +02:00