* refactor(statuses_vacuum): remove dead code - unused
Method is not called inside class and private.
Clean up dead code.
* refactor(statuses_vacuum): make retention_period present test explicit
This private method only hides functionality.
It is best practice to be as explicit as possible.
* refactor(statuses_vacuum): improve query performance
- fix statuses_scope having sub-select for Account.remote scope by
`joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
because it is inexplicit, bad practice and even slower than normal
`.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
because in_batches delivers a full blown active record query result,
in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send
* refactor(status): simplify unlink_from_conversations
- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?
* refactor(status): searchable_by way less sub selects
These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.
This is how it was:
```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
Status Load (1.6ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]]
Mention Load (1.5ms) SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
QUERY PLAN
------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.15..23.08 rows=1 width=41)
-> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8)
Filter: (domain IS NULL)
-> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41)
Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
Filter: (NOT silent)
(6 rows)
```
This is how it is with this change:
```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
Status Load (1.7ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]]
Mention Load (0.7ms) SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
QUERY PLAN
------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.15..23.08 rows=1 width=41)
-> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8)
Filter: (domain IS NULL)
-> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41)
Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
Filter: (NOT silent)
(6 rows)
```
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
glitch-soc change because of having changed the default number of
available poll options.
Applied upstream's changes while keeping glitch-soc's default number of
poll options.
- `public/oops.png`:
We had a minor graphics change, probably not worth diverging from upstream.
Took upstream version.
In the absence of an opt-in to multiple specific languages in the
preferences, it makes more sense to filter by the user's presumed
language only (interface language or `lang` override)
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
Upstream dropping `digest` from notifications emails while we have more
notification emails settings.
Removed `digest` from our list while keeping our extra settings.
- `app/javascript/packs/admin.js`:
Conflicts caused by glitch-soc's theming system.
Applied the changes to `app/javascript/core/admin.js`.
- `app/views/settings/preferences/other/show.html.haml`:
Upstream removed a setting close to a glitch-soc-only setting.
Applied upstream's change.
* Remove duplicate in_chosen_languages definition
* Use `DEFAULT_FIELDS_SIZE` instead of `MAX_FIELDS` to reduce code differences with upstream
* Remove duplicate annotation
* Fix incorrect cross-flavor imports
* Remove deprecated `hide_network` setting (replaced by account column)
* Remove unused KeywordMutesHelper
* Remove trailing spaces
* Remove commit_hash from InstancePresenter, as it has been unused since mid-2017
Conflicts:
- `app/javascript/packs/admin.js`:
Conflicts due to glitch-soc's theming system.
Upstream changes have been ported to `app/javascript/core/admin.js`
- `app/models/trends/statuses.rb`:
Minor conflict due to glitch-soc's option to allow CWed toots in trends.
Ported upstream changes.
Conflicts:
- `app/lib/formatter.rb`:
Upstream completely refactored the formatting code and removed that file,
while glitch-soc had code for Markdown and HTML toots.
Took upstream code, glitch-soc changes will be re-implemented on top of the
refactored classes in a later commit.
- `app/models/status.rb`:
Upstream refactored status edit handling and moved code to
`app/models/concerns/status_snapshot_concern.rb`.
Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
Not really a conflict, just a line added too close to one modified by
glitch-soc.
Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
Not really a conflict, upstream modified a line adjacent to one added by
glitch-soc.
Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
Upstream refactored formatting, glitch-soc changed the markup slightly.
Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
Upstream completely refactored the formatting code and removed that file,
while glitch-soc had code for Markdown and HTML toots.
Took upstream code, glitch-soc changes will be re-implemented on top of the
refactored classes in a later commit.
* Fix edits with no actual changes being allowed locally
* Fix edits with no actual changes being allowed through ActivityPub
* Fix false positive changes caused by description processing in model
* Fix not recording poll expiration update
* Fix test
* Revert changes to ProcessStatusUpdateService
* Various fixes and improvements
* Fix code style issues
* Various changes and improvements
* Add guard clause
Conflicts:
- `app/models/status.rb`:
Upstream updated media and edit-related code textually close to glitch-soc
additions (local-only and content-type).
Ported upstream changes.
- `app/models/status_edit.rb`:
Upstream changes textually close to glitch-soc additions (content-type).
Ported upstream changes.
- `app/serializers/activitypub/note_serializer.rb`:
Upstream changed how media attachments are handled. Not really a conflict,
but textually close to glitch-soc additions (directMessage attribute).
Ported upstream changes.
- `app/services/remove_status_service.rb`:
Upstream changed how media attachments are handled. Not really a conflict,
but textually close to glitch-soc additions (DM timeline).
Ported upstream changes.
- `app/services/update_status_service.rb`:
Upstream fixed an issue with language selection. Not really a conflict,
but textually close to glitch-soc additions (content-type).
Ported upstream changes.
- `db/schema.rb`:
Upstream added columns to the `status_edits` table, the conflict is because
of an additional column (`content-type`) in glitch-soc.
Ported upstream changes.
- `package.json`:
Upstream dependency (express) textually adjacent to a glitch-soc-specific one
(favico.js) got updated.
Updated it as well.
* Change how changes to media attachments are stored for edits
Fix not being able to re-order media attachments
* Fix not broadcasting updates when polls/media is changed through ActivityPub
* Various fixes and improvements
* Update app/models/report.rb
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Add tracking of media attachment description changes
* Change poll in status edit to have a structure closer to the real one
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
Conflicts due to us having more user settings and upstream dropping
`hide_network` (to replace it with an account attribute, properly migrated).
Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
Conflicts due to us having more user settings and upstream dropping
`hide_network` (to replace it with an account attribute, properly migrated).
Dropped `hide_network` like upstream.
- `app/models/status.rb`:
Conflict because of slight change in how glitch-soc handles the scope to
filter out local-only posts for anonymous viewers.
Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
Conflicts due to us having more user settings and upstream dropping
`hide_network` (to replace it with an account attribute, properly migrated).
Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
Conflict because upstream redesigned that page while glitch-soc had a minor
change to support hiding the number of followers.
Ported glitch-soc's change on top of upstream's redesign.
Additional changes:
- `app/models/account_statuses_filter.rb`:
See change to `app/models/status.rb`.
* Fix performance of account timelines
* Various fixes and improvements
* Fix duplicate results being returned
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix grouping for pinned statuses scope
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/models/account.rb`:
Not a real conflict, just upstream getting rid of unused constants too close
to glitch-soc-specific contents.
Removed unused constants like upstream did.
- `app/models/trends.rb`:
Conflict because glitch-soc disabled email notifications for trending links.
Upstream has refactored this quite a bit and added trending posts.
Took upstream code, but disabling the extra trending stuff will come in
another commit.
- `app/views/admin/trends/links/index.html.haml`:
Conflict due to glitch-soc's theming system.
Ported upstream changes accordingly.
* Add trending statuses
* Fix dangling items with stale scores in localized sets
* Various fixes and improvements
- Change approve_all/reject_all to approve_accounts/reject_accounts
- Change Trends::Query methods to not mutate the original query
- Change Trends::Query#skip to offset
- Change follow recommendations to be refreshed in a transaction
* Add tests for trending statuses filtering behaviour
* Fix not applying filtering scope in controller
* Add notifications when a reblogged status has been updated
* Change wording to say "edit" instead of "update" and add missing controls
* Replace previous update notifications with the most up-to-date one
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
Upstream moved things around in a place where glitch-soc had support for
an extra parameter (`content_type`).
Follow upstream but reintroduce `content_type`.
* Add editing for published statuses
* Fix change of multiple-choice boolean in poll not resetting votes
* Remove the ability to update existing media attachments for now
* Fix error-prone SQL queries in Account search
While this code seems to not present an actual vulnerability, one could
easily be introduced by mistake due to how the query is built.
This PR parameterises the `to_tsquery` input to make the query more robust.
* Harden code for Status#tagged_with_all and Status#tagged_with_none
Those two scopes aren't used in a way that could be vulnerable to an SQL
injection, but keeping them unchanged might be a hazard.
* Remove unneeded spaces surrounding tsquery term
* Please CodeClimate
* Move advanced_search_for SQL template to its own function
This avoids one level of indentation while making clearer that the SQL template
isn't build from all the dynamic parameters of advanced_search_for.
* Add tests covering tagged_with, tagged_with_all and tagged_with_none
* Rewrite tagged_with_none to avoid multiple joins and make it more robust
* Remove obsolete brakeman warnings
* Revert "Remove unneeded spaces surrounding tsquery term"
The two queries are not strictly equivalent.
This reverts commit 86f16c537e06c6ba4a8b250f25dcce9f049023ff.
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
Upstream refactored how `Create` activities are handled and how values are
extracted from `Create`d objects. This conflicted with how glitch-soc
supported the `directMessage` flag to explicitly distinguish between
limited and direct messages.
Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
Upstream largely refactored that file and changed some of the logic.
This conflicted with glitch-soc's handling of the direct timeline and
the options to allow replies and boosts in public feeds.
Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored to move mention-related ActivityPub deliveries to
`ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
to not send local-only toots to remote mentioned users.
Took upstream's version, as the check is not needed anymore, since it is
performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
Upstream added support for `update` toot events, while glitch-soc had
support for an extra timeline support, `direct`.
Ported upstream changes and extended them to the `direct` timeline.
Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
Added code to handle the `directMessage` flag and take it into account
to compute visibility.
- `app/lib/feed_manager.rb`:
Extended upstream's support of `update` toot events to glitch-soc's
`direct` timeline.
* Add support for editing for published statuses
* Fix references to stripped-out code
* Various fixes and improvements
* Further fixes and improvements
* Fix updates being potentially sent to unauthorized recipients
* Various fixes and improvements
* Fix wrong words in test
* Fix notifying accounts that were tagged but were not in the audience
* Fix mistake
Conflicts:
- `app/controllers/admin/dashboard_controller.rb`:
Upstream completely redesigned the admin dashboard.
glitch-soc tracked extra features, but that list is
gone.
Followed upstram.
- `app/views/admin/dashboard/index.html.haml`
Upstream completely redesigned the admin dashboard.
glitch-soc tracked extra features, but that list is
gone.
Followed upstram.
- `app/views/statuses/_simple_status.html.haml`:
Small markup change in glitch-soc, on a line that has been modified by
upstream. Ported upstream changes.
Claire