Commit Graph

1742 Commits (aec61a703f4c1f4724bbb5b912504e19b3303e9d)

Author SHA1 Message Date
Jeong Arm df27953f96
Support audio/vnd.wave (#18737)
See: https://datatracker.ietf.org/doc/html/rfc2361
And Misskey uses this mime type for wav file.
2022-06-28 19:49:35 +02:00
Claire fe5f6bc7ed Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Fix erroneous deletion in a previous merge.
- `Gemfile`:
  Conflict caused by glitch-soc-only hCaptcha dependency
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/controllers/filters_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/serializers/rest/status_serializer.rb`:
  Minor conflict due to glitch-soc having an extra `local_only` property
2022-06-28 11:11:18 +02:00
Claire 02851848e9
Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Eugen Rochko 2936f42a14
Add notifications for new reports (#18697) 2022-06-27 09:30:15 +02:00
Claire 35588d09e2
Add /api/v1/admin/domain_allows (#18668)
- `GET /api/v1/admin/domain_allows` lists allowed domains
- `GET /api/v1/admin/domain_allows/:id` shows one by ID
- `DELETE /api/v1/admin/domain_allows/:id` deletes a given domain from the list
  of allowed domains
- `POST /api/v1/admin/domain_allows` to allow a new domain:
  if that domain is already allowed, the existing DomainAllow will be returned
2022-06-23 23:12:01 +02:00
Claire edaaf6a5e9 Merge branch 'main' into glitch-soc/merge-upstream 2022-06-10 11:51:43 +02:00
Eugen Rochko a2871cd747
Add administrative webhooks (#18510)
* Add administrative webhooks

* Fix error when webhook is deleted before delivery worker runs
2022-06-09 21:57:36 +02:00
Claire 25a46d9e59 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/styles/fonts/montserrat.scss`:
  Code style changes upstream, path changes in glitch-soc.
  Applied upstream's code style changes.
- `app/javascript/styles/fonts/roboto-mono.scss`:
  Code style changes upstream, path changes in glitch-soc.
  Applied upstream's code style changes.
- `app/javascript/styles/fonts/roboto.scss`:
  Code style changes upstream, path changes in glitch-soc.
  Applied upstream's code style changes.
- `app/models/account.rb`:
  Textual conflict only caused by glitch-soc using a different value
  for character limits in a nearby line.
  Applied upstream's changes.
- `app/views/statuses/_simple_status.html.haml`:
  Attribute added to a tag modified by glitch-soc.
  Added upstream's attributes.
- `yarn.lock`:
  Upstream added/updated dependencies close to glitch-soc-only ones.
  Updated/added upstream dependencies.
2022-06-02 09:49:38 +02:00
Claire 9d4861b498
Remove dependency on running Redis server for db:setup (#18560) 2022-06-01 19:23:31 +02:00
Claire 28329ba62f
Add /api/v1/admin/domain_blocks (#18247)
* Add /api/v1/admin/domain_blocks

Fixes #18140

- `GET /api/v1/admin/domain_blocks` lists domain blocks
- `GET /api/v1/admin/domain_blocks/:id` shows one by ID
- `DELETE /api/v1/admin/domain_blocks/:id` deletes a given domain block
- `POST /api/v1/admin/domain_blocks` to create a new domain block:
  if it conflicts with an existing one, returns an error with
  an attribute `existing_domain_block` with the rendered domain block

* Simplify conflict handling as suggested in review
2022-06-01 17:31:36 +02:00
Alexander Ivanov 379a7a7ca7
Add support for webp uploads (#18506) 2022-05-27 20:06:40 +02:00
Claire e34dd3644c
Remove unused `filtered_languages` column (#18533)
* Remove unused `filtered_languages` column

Fixes #18522

* Fix tests
2022-05-27 20:05:22 +02:00
Claire 7c6fff34d0 Merge branch 'main' into glitch-soc/merge-upstream 2022-05-26 22:49:47 +02:00
Eugen Rochko c4d2c39a75
Fix being able to report otherwise inaccessible statuses (#18528) 2022-05-26 22:08:02 +02:00
Eugen Rochko 3e0e7a1cfb
Fix follower and other counters being able to go negative (#18517) 2022-05-26 20:32:48 +02:00
Claire f13d910f3e Merge branch 'main' into glitch-soc/merge-upstream 2022-05-26 18:34:27 +02:00
Eugen Rochko 088dc0ec5a
Fix regression in `tootctl search deploy` caused by unloaded attribute (#18514) 2022-05-26 18:05:47 +02:00
Claire 440eb71310
Change unapproved and unconfirmed account to not be accessible in the REST API (#17530)
* Change unapproved and unconfirmed account to not be accessible in the REST API

* Change Account#searchable? to reject unconfirmed and unapproved users

* Disable search for unapproved and unconfirmed users in Account.search_for

* Disable search for unapproved and unconfirmed users in Account.advanced_search_for

* Remove unconfirmed and unapproved accounts from Account.searchable scope

* Prevent mentions to unapproved/unconfirmed accounts

* Fix some old tests for Account.advanced_search_for

* Add some Account.advanced_search_for tests for existing behaviors

* Add some tests for Account.search_for

* Add Account.advanced_search_for tests unconfirmed and unapproved accounts

* Add Account.searchable tests

* Fix Account.without_unapproved scope potentially messing with previously-applied scopes

* Allow lookup of unconfirmed/unapproved accounts through /api/v1/accounts/lookup

This is so that the API can still be used to check whether an username is free
to use.
2022-05-26 15:50:33 +02:00
Claire 25dda3061e
Fix unnecessary query on status creation (#17901) 2022-05-26 00:20:30 +02:00
Claire 3fd2aadb23
Various code cleanup (#1782)
* Remove duplicate in_chosen_languages definition

* Use `DEFAULT_FIELDS_SIZE` instead of `MAX_FIELDS` to reduce code differences with upstream

* Remove duplicate annotation

* Fix incorrect cross-flavor imports

* Remove deprecated `hide_network` setting (replaced by account column)

* Remove unused KeywordMutesHelper

* Remove trailing spaces

* Remove commit_hash from InstancePresenter, as it has been unused since mid-2017
2022-05-25 12:27:11 +02:00
Claire 22111914bf Merge branch 'main' into glitch-soc/merge-upstream 2022-05-24 13:59:21 +02:00
Claire e5997a1956
Fix warning an account outside of a report closing all reports for that account (#18387)
* Fix warning an account outside of a report closing all reports for that account

* Make it clear what actions solve other reports

* Revert "Make it clear what actions solve other reports"

This reverts commit ad006de821f72e75480701298d13f0945b509059.
2022-05-23 20:38:29 +02:00
Eugen Rochko a9b64b24d6
Change algorithm of `tootctl search deploy` to improve performance (#18463) 2022-05-22 22:16:43 +02:00
Claire 4415005de7 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
  Conflict caused because of minor code style change upstream, while glitch-soc
  has different code for handling variable maximum chars.
  Ported the change.
- `app/serializers/initial_state_serializer.rb`:
  Conflict because glitch-soc had two extra attributes where upstream added
  one.
  Added upstream's attribute.
2022-05-16 22:29:05 +02:00
Eugen Rochko 6c699b1723
Fix preferred posting language returning unusable value in REST API (#18428) 2022-05-16 19:13:36 +02:00
Claire b91196f4b7
Add confirmation page when importing blocked domains (#1773)
* Move glitch-soc-specific strings to glitch-soc-specific locale files

* Add confirmation page when importing blocked domains
2022-05-16 18:26:49 +02:00
Claire 040b7d37a4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/services/remove_status_service.rb`:
  Conflict due to glitch-soc having extra code for a proper direct visibility
  timeline, in a part of the code upstream refactored.
  Restored glitch-soc's extra code in the refactored bit.
2022-05-16 09:42:32 +02:00
Levi Bard 94e98864e3
Allow import/export of instance-level domain blocks/allows (#1754)
* Allow import/export of instance-level domain blocks/allows.
Fixes #15095

* Pacify circleci

* Address simple code review feedback

* Add headers to exported CSV

* Extract common import/export functionality to
AdminExportControllerConcern

* Add additional fields to instance-blocked domain export

* Address review feedback

* Split instance domain block/allow import/export into separate pages/controllers

* Address code review feedback

* Pacify DeepSource

* Work around Paperclip::HasAttachmentFile for Rails 6

* Fix deprecated API warning in export tests

* Remove after_commit workaround
2022-05-16 09:29:01 +02:00
Claire aa08399e6f
Fix setting for trending tags and trending links notifications being swapped (#1771) 2022-05-13 22:57:47 +02:00
Eugen Rochko 6cf57c6765
Refactor how Redis locks are created (#18400)
* Refactor how Redis locks are created

* Fix autorelease duration on account deletion lock
2022-05-13 00:02:35 +02:00
Claire 22080786df Merge branch 'main' into glitch-soc/merge-upstream 2022-05-08 21:00:49 +02:00
Claire a53003c6f1
Fix account warnings not being recorded in audit log (#18338)
* Fix account warnings not being recorded in audit log

Fixes #18334

* Only record warnings if they are not associated to another action
2022-05-06 21:40:20 +02:00
Claire 392ca0472a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/admin.js`:
  Conflicts due to glitch-soc's theming system.
  Upstream changes have been ported to `app/javascript/core/admin.js`
- `app/models/trends/statuses.rb`:
  Minor conflict due to glitch-soc's option to allow CWed toots in trends.
  Ported upstream changes.
2022-05-01 16:52:27 +02:00
Claire 6e4d932da5
Fix possible crash when a post references an invalid media attachment (#18211) 2022-05-01 00:55:26 +02:00
Eugen Rochko f6d35ed57d
Remove IP matching from e-mail domain blocks (#18190)
Clear out e-mail domain blocks created from automatically resolved DNS records
2022-04-29 23:27:03 +02:00
Eugen Rochko 7b0fe4aef9
Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Eugen Rochko 6476f7e4da
Change trending statuses to only show one status from each account (#18181)
Calculate trends in temporary sets to avoid having to manage items
that go below the decay threshold while not having any moments
where a half-processed set is accessible to end-users
2022-04-29 22:42:42 +02:00
Eugen Rochko 6726d2933a
Change half-life of trending status scores from 6 hours to 2 hours (#18182) 2022-04-29 11:46:59 +02:00
Claire f23f784f18 Merge branch 'main' into glitch-soc/merge-upstream 2022-04-28 18:16:42 +02:00
Eugen Rochko 3917353645
Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire fc1dd81ea5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not a real conflict, upstream dependency updated textually too close to a
  glitch-soc-only dependency.
  Updated the upstream dependency.
2022-04-27 08:48:21 +02:00
Claire 33cd80d69c
Fix instance actor being incorrectly created when running migrations (#18109)
* Add migration test about instance actor key

* Fix old migration

* Work around incorrect database state
2022-04-26 21:22:09 +02:00
Claire 002d2729fb Merge branch 'main' into glitch-soc/merge-upstream 2022-04-25 17:09:32 +02:00
Jeong Arm ea38327816
Let votes statuses are also searchable (#18070) 2022-04-23 21:47:27 +02:00
Claire 5c808ee0de Revert support from loading Home timeline from database
Unfortunately, the database query could turn out very inefficient and I did not
manage to find a way to improve that. Furthermore, there were still behavior
inconsistencies between fetching the timeline from Redis and fetching it from
Postgres.
2022-04-19 15:06:38 +02:00
Claire ec4a8d81b1 Revert DM support in HomeFeed#from_database
Fixes #1746

Queries could get prohibitively expensive.
2022-04-16 21:28:46 +02:00
Claire 5db3a14388
Fix loading Home TL from database not respecting `min_id` and not including DMs (#1744)
* Rework tests

* Add tests

* Fix HomeFeed#get with min_id fetching from database

* Minor code cleanup and optimizations

* Add tests

* Take DMs into account when fetching home TL from database

* Fix not listing own DMs in Home timeline

* Add tests

* Please CodeClimate
2022-04-14 17:44:21 +02:00
Jeong Arm d353bb5ee3
Implement infinity home timeline (#1610)
* Implement infinity home timeline

* Fix test for infinite home timeline

* Fix infinity home timeline with min_id

* Fix infinite home timeline duplicated statuses

* Codeclimate for infinite home timeline

* Refactor code as reviewed

* Fix redis sufficient check

* Fix typo on variable name
2022-04-12 17:52:14 +02:00
Claire f9fbff8539 Merge branch 'main' into glitch-soc/merge-upstream 2022-04-11 16:16:33 +02:00
Claire dc50347247
Fix crash in alias settings page (#18004) 2022-04-09 20:11:06 +02:00
Claire ff73d43b35 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, just an upstream dependency udpated
  textually too close to a glitch-soc-only dependency.
  Updated dependencies like upstream.
- `app/controllers/settings/preferences_controller.rb`:
  Upstream added settings where we had extra glitch-soc-specific settings.
  Added upstream's new settings.
- `app/models/user.rb`:
  Upstream added settings where we had extra glitch-soc-specific settings.
  Added upstream's new settings.
- `config/i18n-tasks.yml`:
  Not a real conflict, just a new upstream line too textually close to
  a glitch-soc-only line.
  Ported upstream's change.
2022-04-08 19:53:32 +02:00
Eugen Rochko 68273a7c6d
Fix dangling language-specific trends (#17997)
- Change score half-life for trending statuses from 2 to 6 hours
- Change score threshold for trimming old items from 1 to 0.3
2022-04-08 19:35:31 +02:00
Eugen Rochko 8e20e16cf0
Change e-mail notifications to only be sent when recipient is offline (#17984)
* Change e-mail notifications to only be sent when recipient is offline

Change the default for follow and mention notifications back on

* Add preference to always send e-mail notifications

* Change wording
2022-04-08 18:03:31 +02:00
Eugen Rochko fd9a9b07c2
Fix trends returning less results per page when filtered in REST API (#17996)
- Change filtering and pagination to occur in SQL instead of Redis
- Change rank/score displayed on trends in admin UI to be locale-specific
2022-04-08 17:10:53 +02:00
Eugen Rochko 465ee7792f
Fix pagination header on empty trends responses in REST API (#17986) 2022-04-07 18:06:15 +02:00
Claire b368c75029 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Upstream removed this file, while glitch-soc had changes to deal with
  its theming system.
  Removed the file like upstream did.
2022-04-06 21:10:23 +02:00
Eugen Rochko 6221b36b27
Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Claire 6bdec0aec9 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-31 10:36:23 +02:00
Claire ef196c913c
Fix error MethodError in Chewy::Strategy::Sidekiq::Worker (#17912)
Also refactor a bit to reduce code duplication.
2022-03-31 00:49:24 +02:00
Claire fc5dd0c538 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-30 15:02:56 +02:00
Eugen Rochko 5554ff2a1d
Fix being able to bypass e-mail restrictions (#17909) 2022-03-30 14:45:52 +02:00
Claire 51a297e128 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-30 13:40:10 +02:00
Claire 894956e20c
Fix /api/v1/admin/accounts (#17887)
* Fix /api/v1/admin/accounts

Compatibility was broken since #17009 which changed the underlying filter class
without changing the controller.

This commits restore support for the old parameters.

* Add /api/v2/admin/accounts with the new parameters

* Add tests

* Add missing filter for `silenced` status

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-03-28 23:57:38 +02:00
Claire aaa9ec340b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/formatter.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
- `app/models/status.rb`:
  Upstream refactored status edit handling and moved code to
  `app/models/concerns/status_snapshot_concern.rb`.
  Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
  Not really a conflict, just a line added too close to one modified by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
  Not really a conflict, upstream modified a line adjacent to one added by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream refactored formatting, glitch-soc changed the markup slightly.
  Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
2022-03-26 19:18:55 +01:00
Eugen Rochko 2dd30804b6
Change how unconfirmed accounts are displayed in admin UI (#17874)
Fix #17815
2022-03-26 02:53:13 +01:00
Eugen Rochko 71f2b95106
Fix edits with no actual changes being allowed (#17843)
* Fix edits with no actual changes being allowed locally

* Fix edits with no actual changes being allowed through ActivityPub

* Fix false positive changes caused by description processing in model

* Fix not recording poll expiration update

* Fix test

* Revert changes to ProcessStatusUpdateService

* Various fixes and improvements

* Fix code style issues

* Various changes and improvements

* Add guard clause
2022-03-26 00:38:44 +01:00
Eugen Rochko e3a2203061
Add offset pagination to trends in REST API (#17872) 2022-03-26 00:26:50 +01:00
Claire c79a03b319 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream changed their README, we have a completely different one.
  Kept our README.
2022-03-15 20:45:27 +01:00
Eugen Rochko a794117679
Fix individually approved/rejected statuses/links showing as pending review (#17787) 2022-03-15 07:51:55 +01:00
Eugen Rochko e6ffbfb5e7
Add `types` param to `GET /api/v1/notifications` in REST API (#17767)
* Add `types` param to `GET /api/v1/notifications` in REST API

* Improve tests
2022-03-15 04:11:29 +01:00
Eugen Rochko eebafe24a8
Fix statuses not being referenced in strike when category is spam (#17786) 2022-03-15 04:11:13 +01:00
Claire a945182b56 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-14 16:44:35 +01:00
Eugen Rochko 5db1f377ea
Fix nil error when viewing suspended domain in admin UI (#17765) 2022-03-14 05:27:37 +01:00
Claire bb9a6fb5c2 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-12 11:51:25 +01:00
Claire 642528f455
Update fix-duplicates maintenance task (#17731)
* Update fix-duplicates task to 2022_02_10_153119

Also add support for Appeal to AccountMerging#merge_with!

* Update fix-duplicates task to 2022_03_07_094650

* Update fix-duplicates task to 2022_03_09_213005

* Update fix-duplicates task to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060626

* Update fix-duplicates script to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060706

* Update fix-duplicates task to 2022_03_10_060959

* Silence CodeClimate
2022-03-12 08:33:11 +01:00
Eugen Rochko ee8d4dd056
Fix moderator account leak in status edit history (#17746) 2022-03-12 08:23:03 +01:00
Jeong Arm 18e27e0488
Enable domain purge button on suspended domains too (#17741)
Related: #17209
2022-03-11 04:58:09 +01:00
Claire 24e83246f9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/status.rb`:
  Upstream updated media and edit-related code textually close to glitch-soc
  additions (local-only and content-type).
  Ported upstream changes.
- `app/models/status_edit.rb`:
  Upstream changes textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `app/serializers/activitypub/note_serializer.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (directMessage attribute).
  Ported upstream changes.
- `app/services/remove_status_service.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (DM timeline).
  Ported upstream changes.
- `app/services/update_status_service.rb`:
  Upstream fixed an issue with language selection. Not really a conflict,
  but textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `db/schema.rb`:
  Upstream added columns to the `status_edits` table, the conflict is because
  of an additional column (`content-type`) in glitch-soc.
  Ported upstream changes.
- `package.json`:
  Upstream dependency (express) textually adjacent to a glitch-soc-specific one
  (favico.js) got updated.
  Updated it as well.
2022-03-10 09:52:45 +01:00
Takeshi Umeda d7fab238a8
Fix a type error in domain_block policies (#17735) 2022-03-10 04:10:20 +01:00
Eugen Rochko 9f2791eb64
Add polls and media attachments to edit comparison modal in web UI (#17727) 2022-03-09 21:15:24 +01:00
Claire 2a56a890da
Fix rare race condition when rebloged status is deleted (#17693)
* Fix rare race condition when rebloged status is deleted

* Use INSERT INTO … SELECT
2022-03-09 20:49:14 +01:00
Eugen Rochko b2cd34474b
Add rate limit for editing (#17728) 2022-03-09 20:06:51 +01:00
chandrn7 a6ed6845c9
Allow login through OpenID Connect (#16221)
* added OpenID Connect as an SSO option

* minor fixes

* added comments, removed an option that shouldn't be set

* fixed Gemfile.lock

* added newline to end of Gemfile.lock

* removed tab from Gemfile.lock

* remove chomp

* codeclimate changes and small name change to make function's purpose clearer

* codeclimate fix

* added SSO buttons to /about page

* minor refactor

* minor style change

* removed spurious change

* removed unecessary conditional from ensure_valid_username and added support for auth.info.name in user_params_from_auth

* minor changes
2022-03-09 12:07:35 +01:00
Eugen Rochko d17fb70131
Change how changes to media attachments are stored for edits (#17696)
* Change how changes to media attachments are stored for edits

Fix not being able to re-order media attachments

* Fix not broadcasting updates when polls/media is changed through ActivityPub

* Various fixes and improvements

* Update app/models/report.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add tracking of media attachment description changes

* Change poll in status edit to have a structure closer to the real one

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-09 09:06:17 +01:00
Eugen Rochko bd53dd5210
Change design of federation pages in admin UI (#17704)
* Change design of federation pages in admin UI

* Fix query performance in instance media attachments measure

* Fix reblogs being included in instance languages dimension
2022-03-09 08:52:32 +01:00
Eugen Rochko 318d34d528
Fix data integrity of featured tags (#17712) 2022-03-09 08:51:12 +01:00
Claire 1fbd1fa5c4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/models/status.rb`:
  Conflict because of slight change in how glitch-soc handles the scope to
  filter out local-only posts for anonymous viewers.
  Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
  Conflict because upstream redesigned that page while glitch-soc had a minor
  change to support hiding the number of followers.
  Ported glitch-soc's change on top of upstream's redesign.

Additional changes:
- `app/models/account_statuses_filter.rb`:
  See change to `app/models/status.rb`.
2022-03-08 20:22:54 +01:00
Eugen Rochko 8f6c67bfde
Fix performance of account timelines (#17709)
* Fix performance of account timelines

* Various fixes and improvements

* Fix duplicate results being returned

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Fix grouping for pinned statuses scope

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-08 09:14:39 +01:00
Eugen Rochko 292c75aa31
Fix nil error when submitting report without `rule_ids` (#17713) 2022-03-07 11:21:27 +01:00
Eugen Rochko edf09ec747
Add `/api/v1/accounts/familiar_followers` to REST API (#17700)
* Add `/api/v1/accounts/familiar_followers` to REST API

* Change hide network preference to be stored consistently for local and remote accounts

* Add dummy classes to migration

* Apply suggestions from code review

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-07 09:36:47 +01:00
Claire d9c3109bb6 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-04 01:07:49 +01:00
Claire 02dbc59c91
Add option to automatically delete toots after one week (#17691)
Fixes #17658
2022-03-03 16:18:58 +01:00
Claire 425a6c90c4 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-02 18:02:48 +01:00
Eugen Rochko 25d3dc4373
Add ability to mark statuses as sensitive from reports in admin UI (#17668)
* Add ability to mark statuses as sensitive from reports in admin UI

* Allow mark as sensitive action on statuses with preview cards
2022-03-01 22:20:29 +01:00
Claire 14919fe11e
Change old moderation strikes to be displayed in a separate page (#17566)
* Change old moderation strikes to be displayed in a separate page

Fixes #17552

This changes the moderation strikes displayed on `/auth/edit` to be those from
the past 3 months, and make all moderation strikes targeting the current user
available in `/disputes`.

* Add short description of what the strikes page is for

* Move link to list of strikes to “Account status” instead of navigation item

* Normalize i18n file

* Fix layout and styling of strikes link

* Revert highlights_on regexp

* Reintroduce account status summary

- this way, “Account status” is never empty
- account status is not necessarily bound to strikes, or recent strikes
2022-03-01 19:37:47 +01:00
Claire 756f1b6615 Add option (on by default) to allow toots with content warnings to trend 2022-02-26 10:46:26 +01:00
Claire c6df6686af Disable notifications for trending links and trending statuses by default 2022-02-26 10:46:26 +01:00
Claire be493b6c0d Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/account.rb`:
  Not a real conflict, just upstream getting rid of unused constants too close
  to glitch-soc-specific contents.
  Removed unused constants like upstream did.
- `app/models/trends.rb`:
  Conflict because glitch-soc disabled email notifications for trending links.
  Upstream has refactored this quite a bit and added trending posts.
  Took upstream code, but disabling the extra trending stuff will come in
  another commit.
- `app/views/admin/trends/links/index.html.haml`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes accordingly.
2022-02-26 09:29:23 +01:00
Eugen Rochko 27965ce5ed
Add trending statuses (#17431)
* Add trending statuses

* Fix dangling items with stale scores in localized sets

* Various fixes and improvements

- Change approve_all/reject_all to approve_accounts/reject_accounts
- Change Trends::Query methods to not mutate the original query
- Change Trends::Query#skip to offset
- Change follow recommendations to be refreshed in a transaction

* Add tests for trending statuses filtering behaviour

* Fix not applying filtering scope in controller
2022-02-25 00:34:14 +01:00
Eugen Rochko a29a982eaa
Change e-mail domain blocks to block IPs dynamically (#17635)
* Change e-mail domain blocks to block IPs dynamically

* Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

* Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-24 17:28:23 +01:00
Claire 3d25fe0426 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream changed the workflow a bit.
  Conflict comes from us pushing to ghcr while upstream pushes to dockerhub.
  Ported the upstream changes while still pushing to ghcr.
2022-02-23 18:13:12 +01:00
Eugen Rochko 7b816eb5ae
Add notifications for new sign-ups (#16953) 2022-02-23 16:45:22 +01:00
Claire 954e052b07 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/media_attachment.rb`:
  Upstream moved hardcoded values around, while in glitch-soc they are
  configurable.
  Moved them like upstream, but keeping glitch-soc's ability to configure
  them through env vars.
2022-02-22 18:06:29 +01:00
Claire 166f6e4b50
Fix some media attachments being converted with too high framerates (#17619)
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).

This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.

This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
2022-02-22 17:11:22 +01:00
Claire f224237862 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `db/schema.rb`:
  Conflict due to glitch-soc adding the `content_type` column on status edits
  and thus having a different schema version number.
  Solved by taking upstream's schema version number, as it is higher than
  glitch-soc's.
2022-02-17 10:58:44 +01:00
Claire ac99f586bb
Fix issues when attempting to appeal an old strike (#17554)
* Display an error when an appeal could not be submitted

* Do not offer users to appeal old strikes

* Fix 500 error when trying to appeal a strike that is too old

* Avoid using an extra translatable string
2022-02-16 22:29:48 +01:00
Eugen Rochko 564efd0651
Add appeals (#17364)
* Add appeals

* Add ability to reject appeals and ability to browse pending appeals in admin UI

* Add strikes to account page in settings

* Various fixes and improvements

- Add separate notification setting for appeals, separate from reports
- Fix style of links in report/strike header
- Change approving an appeal to not restore statuses (due to federation complexities)
- Change style of successfully appealed strikes on account settings page
- Change account settings page to only show unappealed or recently appealed strikes

* Change appealed_at to overruled_at

* Fix missing method error
2022-02-14 21:27:53 +01:00
Jeong Arm d39df35441
Fix admin statuses page order with media (#17538) 2022-02-14 16:07:04 +01:00
Eugen Rochko 6240466866
Fix duplicate accounts when searching by IP range in admin UI (#17524) 2022-02-13 01:58:26 +01:00
Claire d9379f5331 Merge branch 'main' into glitch-soc/merge-upstream 2022-02-12 11:12:41 +01:00
Eugen Rochko 8f03b7a2fb
Add notifications when a reblogged status has been updated (#17404)
* Add notifications when a reblogged status has been updated

* Change wording to say "edit" instead of "update" and add missing controls

* Replace previous update notifications with the most up-to-date one
2022-02-11 22:20:19 +01:00
Claire 51cad4f7fb Merge branch 'main' into glitch-soc/merge-upstream 2022-02-11 21:53:20 +01:00
Eugen Rochko d0fcf07436
Change actions in reports to require only one click (#17487) 2022-02-11 21:51:57 +01:00
Eugen Rochko a27729ee48
Change reblogs to not count towards hashtag trends anymore (#17501) 2022-02-11 21:16:23 +01:00
Claire abd113167b Add ability to change content-type when editing a toot
Content-type defaults to edited toot's content-type to avoid surprising
behaviors when using clients that do not support this feature.
2022-02-10 19:10:14 +01:00
Claire f1a6f9062e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
  Upstream moved things around in a place where glitch-soc had support for
  an extra parameter (`content_type`).
  Follow upstream but reintroduce `content_type`.
2022-02-10 19:09:27 +01:00
Claire 63854bee6c
Fix poll votes not being properly reset on poll change (#17498)
* Fix poll votes not being properly reset on poll change

* Fix and add tests

* Fix poll update handling when the number of options changes
2022-02-10 14:26:54 +01:00
Eugen Rochko 63002cde03
Add editing for published statuses (#17320)
* Add editing for published statuses

* Fix change of multiple-choice boolean in poll not resetting votes

* Remove the ability to update existing media attachments for now
2022-02-10 00:15:30 +01:00
Eugen Rochko 2f8159baad
Add `category` and `rule_ids` params to `POST /api/v1/reports` (#17492) 2022-02-10 00:10:16 +01:00
Claire d90da7d080 Add content_type to status_edits 2022-02-09 18:02:31 +01:00
Claire 322e907e04 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Upstream renamed some helper functions that were used in a part of the
  settings page which glitch-soc slightly changed the layout of.
  Ported the change.
2022-02-09 17:28:33 +01:00
Eugen Rochko fd3a45e348
Add edit history to web UI (#17390)
* Add edit history to web UI

* Change history reducer to store items per status

* Fix missing loading prop
2022-02-09 01:17:07 +01:00
Eugen Rochko 2adcad04ff
Fix error in suggestions API due to typo (#17486)
Regression from #17479
2022-02-08 22:23:04 +01:00
Claire 692963d43b Merge branch 'main' into glitch-soc/merge-upstream 2022-02-08 18:23:53 +01:00
Eugen Rochko b6d7726ecb
Remove language detection through cld3 (#17478)
* Remove language detection through cld3

* Update app/helpers/languages_helper.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-08 02:41:17 +01:00
Eugen Rochko 35850f8195
Fix localization of cold-start follow recommendations (#17479) 2022-02-08 01:53:49 +01:00
Claire 098f2bc1e1 Merge branch 'main' into glitch-soc/merge-upstream 2022-02-01 20:59:28 +01:00
Claire 987d88ea56
Fix requiring an extra restart after recent post-deployment migrations (#17422)
Follow-up to #16409
2022-02-01 20:57:39 +01:00
Claire b2915613fb Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Upstream-updated lib textually too close to glitch-soc-only dep.
  Updated like upstream.
2022-01-28 08:58:32 +01:00
Claire 03d59340da
Fix Sidekiq warnings about JSON serialization (#17381)
* Fix Sidekiq warnings about JSON serialization

This occurs on every symbol argument we pass, and every symbol key in hashes,
because Sidekiq expects strings instead.

See https://github.com/mperham/sidekiq/pull/5071

We do not need to change how workers parse their arguments because this has
not changed and we were already converting to symbols adequately or using
`with_indifferent_access`.

* Set Sidekiq to raise on unsafe arguments in test mode

In order to more easily catch issues that would produce warnings in production
code.
2022-01-28 00:43:56 +01:00
Claire b7cf3941b3 Change CAPTCHA handling to be only on email verification
This simplifies the implementation considerably, and while not providing
ideal UX, it's the most flexible approach.
2022-01-25 23:56:57 +01:00
Claire 0fb907441c Add ability to set hCaptcha either on registration form or on e-mail validation
Upshot of CAPTCHA on e-mail validation is it does not need to break the in-band
registration API.
2022-01-25 23:09:48 +01:00
Claire 1b493c9fee Add optional hCaptcha support
Fixes #1649

This requires setting `HCAPTCHA_SECRET_KEY` and `HCAPTCHA_SITE_KEY`, then
enabling the admin setting at
`/admin/settings/edit#form_admin_settings_captcha_enabled`

Subsequently, a hCaptcha widget will be displayed on `/about` and
`/auth/sign_up` unless:
- the user is already signed-up already
- the user has used an invite link
- the user has already solved the captcha (and registration failed for another
  reason)

The Content-Security-Policy headers are altered automatically to allow the
third-party hCaptcha scripts on `/about` and `/auth/sign_up` following the same
rules as above.
2022-01-24 21:22:13 +01:00
Claire 61ef81c548 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `spec/models/status_spec.rb`:
  Upstream added tests too close to glitch-soc-specific tests.
  Kept both tests.
2022-01-23 18:24:01 +01:00
Claire 0a120d86d2
Fix error-prone SQL queries (#15828)
* Fix error-prone SQL queries in Account search

While this code seems to not present an actual vulnerability, one could
easily be introduced by mistake due to how the query is built.

This PR parameterises the `to_tsquery` input to make the query more robust.

* Harden code for Status#tagged_with_all and Status#tagged_with_none

Those two scopes aren't used in a way that could be vulnerable to an SQL
injection, but keeping them unchanged might be a hazard.

* Remove unneeded spaces surrounding tsquery term

* Please CodeClimate

* Move advanced_search_for SQL template to its own function

This avoids one level of indentation while making clearer that the SQL template
isn't build from all the dynamic parameters of advanced_search_for.

* Add tests covering tagged_with, tagged_with_all and tagged_with_none

* Rewrite tagged_with_none to avoid multiple joins and make it more robust

* Remove obsolete brakeman warnings

* Revert "Remove unneeded spaces surrounding tsquery term"

The two queries are not strictly equivalent.

This reverts commit 86f16c537e06c6ba4a8b250f25dcce9f049023ff.
2022-01-23 18:10:10 +01:00
Claire 8a07ecd377
Remove leftover database columns from Devise::Models::Rememberable (#17191)
* Remove leftover database columns from Devise::Models::Rememberable

* Update fix-duplication maintenance script

* Improve errors/warnings in the fix-duplicates maintenance script
2022-01-23 15:46:30 +01:00
Claire fe89554a54 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
  Upstream refactored how `Create` activities are handled and how values are
  extracted from `Create`d objects. This conflicted with how glitch-soc
  supported the `directMessage` flag to explicitly distinguish between
  limited and direct messages.
  Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Upstream largely refactored that file and changed some of the logic.
  This conflicted with glitch-soc's handling of the direct timeline and
  the options to allow replies and boosts in public feeds.
  Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored to move mention-related ActivityPub deliveries to
  `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
  to not send local-only toots to remote mentioned users.
  Took upstream's version, as the check is not needed anymore, since it is
  performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
  Upstream added support for `update` toot events, while glitch-soc had
  support for an extra timeline support, `direct`.
  Ported upstream changes and extended them to the `direct` timeline.

Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
  Added code to handle the `directMessage` flag and take it into account
  to compute visibility.
- `app/lib/feed_manager.rb`:
  Extended upstream's support of `update` toot events to glitch-soc's
  `direct` timeline.
2022-01-19 23:52:48 +01:00
Eugen Rochko 1060666c58
Add support for editing for published statuses (#16697)
* Add support for editing for published statuses

* Fix references to stripped-out code

* Various fixes and improvements

* Further fixes and improvements

* Fix updates being potentially sent to unauthorized recipients

* Various fixes and improvements

* Fix wrong words in test

* Fix notifying accounts that were tagged but were not in the audience

* Fix mistake
2022-01-19 22:37:27 +01:00
Claire b3bf32a21e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/reports/show.html.haml`:
  Conflicts due to glitch-soc's theming system.
2022-01-17 10:45:25 +01:00
Eugen Rochko 14f436c457
Add notifications for statuses deleted by moderators (#17204) 2022-01-17 09:41:33 +01:00
Claire b61c3ddff8 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream added some text, but our README is completely different.
  Kept our README unchanged.
2022-01-16 18:11:20 +01:00
Eugen Rochko 8e84ebf0cb
Remove IP tracking columns from users table (#16409) 2022-01-16 13:23:50 +01:00
Claire 09c61e1376 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  We have completely different contents. Kept our version.
- `package.json`:
  Not a real conflict, just an upstream dependency udpated
  textually too close to a glitch-soc-only dependency.
  Updated dependencies like upstream.
- `streaming/index.js`:
  Conflict due to code style changes on parts that were
  modified in glitch-soc to handle local-only toots.
  Changed style according to upstream.
2021-12-27 15:16:18 +01:00
Jeong Arm 720e8ab0f5
Fix duplicate record on admin/accounts when searching with IP (#17150) 2021-12-21 00:17:14 +01:00
Claire d4f315f304 Merge branch 'main' into glitch-soc/merge-upstream 2021-12-18 11:44:54 +01:00
Claire 76761d5fc0
Add ability for admins to delete canonical email blocks (#16644)
* Add admin option to remove canonical email blocks from a deleted account

* Add tootctl canonical_email_blocks to inspect and remove canonical email blocks
2021-12-17 23:02:14 +01:00
Claire 7f803c41e2
Add ability to purge undeliverable domains from admin interface (#16686)
* Add ability to purge undeliverable domains from admin interface

* Add tests
2021-12-17 23:01:21 +01:00
Claire b2526316f5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/pending_accounts/index.html.haml`:
  Removed upstream, while it had glitch-soc-specific changes to accomodate
  for glitch-soc's theming system.
  Removed the file.

Additional changes:
- `app/views/admin/accounts/index.html.haml':
  Accomodate for glitch-soc's theming system.
2021-12-16 16:20:44 +01:00
Eugen Rochko fe45184b36
Change trending hashtags threshold back from 15 to 5 (#17122) 2021-12-13 05:32:29 +01:00
Eugen Rochko 0fb9536d38
Add batch suspend for accounts in admin UI (#17009) 2021-12-05 21:48:39 +01:00