Commit Graph

1903 Commits (351f0cf51a41b66d2ceb639cec6fdeac766d9c4e)

Author SHA1 Message Date
luzpaz 898fe2fa8e
Fix typo in source `setted`->`set` (#18385)
Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,ro`
2022-05-10 04:58:04 +02:00
Eugen Rochko 2b8dc58b7f
Change RSS feeds (#18356)
* Change RSS feeds

- Use date and time for titles instead of ellipsized text
- Use full content in body, even when there is a content warning
- Use media extensions

* Change feed icons and add width and height attributes to custom emojis

* Fix custom emoji animate on hover breaking

* Fix tests
2022-05-09 07:43:08 +02:00
Claire 3869daef09 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, upstream updated a dependency textually adjacent to a
  glitch-soc-only one.
  Updated the dependency as upstream did.
2022-05-03 10:16:49 +02:00
Claire 71d02ffcf3
Fix compatibility with Friendica regarding pinned posts (#18254)
* Fix multiple database queries when fetching pinned posts for remote account

* Fix compatibility with Friendica regarding pinned posts

Fixes #18066

* Add tests
2022-05-02 17:41:01 +02:00
Claire 392ca0472a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/admin.js`:
  Conflicts due to glitch-soc's theming system.
  Upstream changes have been ported to `app/javascript/core/admin.js`
- `app/models/trends/statuses.rb`:
  Minor conflict due to glitch-soc's option to allow CWed toots in trends.
  Ported upstream changes.
2022-05-01 16:52:27 +02:00
Eugen Rochko f6d35ed57d
Remove IP matching from e-mail domain blocks (#18190)
Clear out e-mail domain blocks created from automatically resolved DNS records
2022-04-29 23:27:03 +02:00
Eugen Rochko 7b0fe4aef9
Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Claire 01266e4d7f Merge branch 'main' into glitch-soc/merge-upstream 2022-04-28 23:41:53 +02:00
Claire 84d991988e
Fix temporary network/remote server error prevent from interactions with remote accounts (#18161)
* Fix temporary network/remote server error prevent from interactions with remote accounts

* Fix and add tests
2022-04-28 20:19:10 +02:00
Claire f23f784f18 Merge branch 'main' into glitch-soc/merge-upstream 2022-04-28 18:16:42 +02:00
Eugen Rochko 3917353645
Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire fc1dd81ea5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not a real conflict, upstream dependency updated textually too close to a
  glitch-soc-only dependency.
  Updated the upstream dependency.
2022-04-27 08:48:21 +02:00
Claire 0360135d4d
Fix PeerTube videos appearing with an erroneous “Edited at” marker (#18100)
* Fix PeerTube videos appearing with an erroneous “Edited at” marker

PeerTube videos have an `updated` field equal to `published`.
When processing an incoming activity that has the same value for `updated` and
`published`, assume this doesn't represent an actual edit.

* Please CodeClimate
2022-04-26 21:25:26 +02:00
Claire 5c808ee0de Revert support from loading Home timeline from database
Unfortunately, the database query could turn out very inefficient and I did not
manage to find a way to improve that. Furthermore, there were still behavior
inconsistencies between fetching the timeline from Redis and fetching it from
Postgres.
2022-04-19 15:06:38 +02:00
Claire ec4a8d81b1 Revert DM support in HomeFeed#from_database
Fixes #1746

Queries could get prohibitively expensive.
2022-04-16 21:28:46 +02:00
Claire 5db3a14388
Fix loading Home TL from database not respecting `min_id` and not including DMs (#1744)
* Rework tests

* Add tests

* Fix HomeFeed#get with min_id fetching from database

* Minor code cleanup and optimizations

* Add tests

* Take DMs into account when fetching home TL from database

* Fix not listing own DMs in Home timeline

* Add tests

* Please CodeClimate
2022-04-14 17:44:21 +02:00
Jeong Arm d353bb5ee3
Implement infinity home timeline (#1610)
* Implement infinity home timeline

* Fix test for infinite home timeline

* Fix infinity home timeline with min_id

* Fix infinite home timeline duplicated statuses

* Codeclimate for infinite home timeline

* Refactor code as reviewed

* Fix redis sufficient check

* Fix typo on variable name
2022-04-12 17:52:14 +02:00
Claire f2b2614d0a Fix link sanitization for outgoing text/html and text/markdown toots
Fixes #1739
2022-04-11 09:06:25 +02:00
Claire 489f250877 Merge branch 'main' into glitch-soc/merge-upstream 2022-04-07 15:03:54 +02:00
Claire ce9dcbea32
Fix failure when sending warning emails with custom text (#17983)
* Add tests

* Fix failure when sending warning emails with custom text
2022-04-07 14:47:30 +02:00
Claire b368c75029 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Upstream removed this file, while glitch-soc had changes to deal with
  its theming system.
  Removed the file like upstream did.
2022-04-06 21:10:23 +02:00
Claire 8f91e304a5
Fix spurious edits and require incoming edits to be explicitly marked as such (#17918)
* Change post text edit to not be considered significant if it's identical after reformatting

* We don't need to clear previous change information anymore

* Require status edits to be explicit, except for poll tallies

* Fix tests

* Add some tests

* Add poll-related tests

* Add HTML-formatting related tests
2022-04-06 21:01:02 +02:00
Eugen Rochko 6221b36b27
Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Claire fc5dd0c538 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-30 15:02:56 +02:00
Eugen Rochko bbc7afa2a2
Fix being able to post URLs longer than 4096 characters (#17908) 2022-03-30 14:46:03 +02:00
Claire 51a297e128 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-30 13:40:10 +02:00
Claire 894956e20c
Fix /api/v1/admin/accounts (#17887)
* Fix /api/v1/admin/accounts

Compatibility was broken since #17009 which changed the underlying filter class
without changing the controller.

This commits restore support for the old parameters.

* Add /api/v2/admin/accounts with the new parameters

* Add tests

* Add missing filter for `silenced` status

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-03-28 23:57:38 +02:00
Claire 8d6f3f8a37 Fix code blocks processing 2022-03-28 23:40:45 +02:00
Claire 61cefbebf7 Add advanced text formatting back into glitch-soc 2022-03-28 22:21:37 +02:00
Claire f313bf3e97 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-28 16:22:28 +02:00
Claire 30658924a8
Fix test-related issues (#17888)
* Remove obsolete RSS::Serializer test

Since #17828, RSS::Serializer no longer has specific code for deleted statuses,
but it is never called on deleted statuses anyway.

* Rename erroneously-named test files

* Fix failing test

* Fix test deprecation warnings

* Update CircleCI Ruby orb

1.4.0 has a bug that does not match all the test files due to incorrect
globbing
2022-03-28 12:43:58 +02:00
Claire aaa9ec340b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/formatter.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
- `app/models/status.rb`:
  Upstream refactored status edit handling and moved code to
  `app/models/concerns/status_snapshot_concern.rb`.
  Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
  Not really a conflict, just a line added too close to one modified by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
  Not really a conflict, upstream modified a line adjacent to one added by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream refactored formatting, glitch-soc changed the markup slightly.
  Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
2022-03-26 19:18:55 +01:00
Eugen Rochko cefa526c6d
Refactor formatter (#17828)
* Refactor formatter

* Move custom emoji pre-rendering logic to view helpers

* Move more methods out of Formatter

* Fix code style issues

* Remove Formatter

* Add inline poll options to RSS feeds

* Remove unused helper method

* Fix code style issues

* Various fixes and improvements

* Fix test
2022-03-26 02:53:34 +01:00
Eugen Rochko 71f2b95106
Fix edits with no actual changes being allowed (#17843)
* Fix edits with no actual changes being allowed locally

* Fix edits with no actual changes being allowed through ActivityPub

* Fix false positive changes caused by description processing in model

* Fix not recording poll expiration update

* Fix test

* Revert changes to ProcessStatusUpdateService

* Various fixes and improvements

* Fix code style issues

* Various changes and improvements

* Add guard clause
2022-03-26 00:38:44 +01:00
Eugen Rochko b58db8f12e
Add workaround for YouTube Shorts links (#17869)
* Add workaround for YouTube Shorts links

* Update link_details_extractor_spec.rb
2022-03-25 19:31:35 +01:00
Claire c79a03b319 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream changed their README, we have a completely different one.
  Kept our README.
2022-03-15 20:45:27 +01:00
Eugen Rochko e6ffbfb5e7
Add `types` param to `GET /api/v1/notifications` in REST API (#17767)
* Add `types` param to `GET /api/v1/notifications` in REST API

* Improve tests
2022-03-15 04:11:29 +01:00
Claire 8d4b5ee3ef Merge branch 'main' into glitch-soc/merge-upstream 2022-03-12 20:20:58 +01:00
Claire 92a86b958e
Fix issues with processing toot edits (#17748)
* Fix searching for an already-known status by URL not working

* Fix Update processing from statuses prior to 20220302232632

`ordered_media_attachment_ids_changed?` would return `true` when going from
`nil` to anything (including `[]`).

* Add tests
2022-03-12 19:33:10 +01:00
Claire bb9a6fb5c2 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-12 11:51:25 +01:00
Eugen Rochko ddbe906c25
Fix not updating a status when newer version is fetched manually (#17745) 2022-03-12 09:11:36 +01:00
Eugen Rochko bc320d6cec
Fix `POST /api/v1/emails/confirmations` not being available after sign-up (#17743) 2022-03-12 04:14:25 +01:00
Claire 24e83246f9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/status.rb`:
  Upstream updated media and edit-related code textually close to glitch-soc
  additions (local-only and content-type).
  Ported upstream changes.
- `app/models/status_edit.rb`:
  Upstream changes textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `app/serializers/activitypub/note_serializer.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (directMessage attribute).
  Ported upstream changes.
- `app/services/remove_status_service.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (DM timeline).
  Ported upstream changes.
- `app/services/update_status_service.rb`:
  Upstream fixed an issue with language selection. Not really a conflict,
  but textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `db/schema.rb`:
  Upstream added columns to the `status_edits` table, the conflict is because
  of an additional column (`content-type`) in glitch-soc.
  Ported upstream changes.
- `package.json`:
  Upstream dependency (express) textually adjacent to a glitch-soc-specific one
  (favico.js) got updated.
  Updated it as well.
2022-03-10 09:52:45 +01:00
Claire 5ccd6cbfda
Add test for reblog race condition fix (#17732)
Follow-up to #17693
2022-03-10 00:11:49 +01:00
Claire 63c9d2bc28
Add tests for CVE-2022-24307 (#17733)
Follow-up to #17426
2022-03-10 00:11:40 +01:00
Eugen Rochko d17fb70131
Change how changes to media attachments are stored for edits (#17696)
* Change how changes to media attachments are stored for edits

Fix not being able to re-order media attachments

* Fix not broadcasting updates when polls/media is changed through ActivityPub

* Various fixes and improvements

* Update app/models/report.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add tracking of media attachment description changes

* Change poll in status edit to have a structure closer to the real one

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-09 09:06:17 +01:00
Eugen Rochko bd53dd5210
Change design of federation pages in admin UI (#17704)
* Change design of federation pages in admin UI

* Fix query performance in instance media attachments measure

* Fix reblogs being included in instance languages dimension
2022-03-09 08:52:32 +01:00
Claire 1fbd1fa5c4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/models/status.rb`:
  Conflict because of slight change in how glitch-soc handles the scope to
  filter out local-only posts for anonymous viewers.
  Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
  Conflict because upstream redesigned that page while glitch-soc had a minor
  change to support hiding the number of followers.
  Ported glitch-soc's change on top of upstream's redesign.

Additional changes:
- `app/models/account_statuses_filter.rb`:
  See change to `app/models/status.rb`.
2022-03-08 20:22:54 +01:00
Eugen Rochko 8f6c67bfde
Fix performance of account timelines (#17709)
* Fix performance of account timelines

* Various fixes and improvements

* Fix duplicate results being returned

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Fix grouping for pinned statuses scope

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-08 09:14:39 +01:00
Eugen Rochko edf09ec747
Add `/api/v1/accounts/familiar_followers` to REST API (#17700)
* Add `/api/v1/accounts/familiar_followers` to REST API

* Change hide network preference to be stored consistently for local and remote accounts

* Add dummy classes to migration

* Apply suggestions from code review

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-07 09:36:47 +01:00
Josh Soref b5329e0035
Spelling (#17705)
* spelling: account

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: affiliated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: appearance

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: autosuggest

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: cacheable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: component

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: conversations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: domain.example

Clarify what's distinct and use RFC friendly domain space.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: environment

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeds

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: functional

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: inefficiency

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: not

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: notifications

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: occurring

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: position

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: progress

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: promotable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: reblogging

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: repetitive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: resolve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: saturated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: similar

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: strategies

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: success

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: targeting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnails

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unauthorized

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unsensitizes

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: validations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: various

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-03-06 22:51:40 +01:00
Claire d9c3109bb6 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-04 01:07:49 +01:00
Claire 6d3fa7828e
Redesign /about when already logged in (#17348)
* Redesign /about when already logged in

* Fix sign up form still showing when OMNIAUTH_ONLY is set

* Fix tests

* Change wording based on suggestions

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-03-03 16:14:44 +01:00
Claire 8743b1ea40 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/trends/links/index.html.haml`:
  Not really a conflict, upstream change textually too close to a glitch-soc
  change (removed `javascript_pack_tag` to accomodate for glitch-soc's theming
  system).
  Ported upstream changes.
- `app/views/admin/trends/links/preview_card_providers/index.html.haml`:
  Not really a conflict, upstream change textually too close to a glitch-soc
  change (removed `javascript_pack_tag` to accomodate for glitch-soc's theming
  system).
  Ported upstream changes.
- `app/views/admin/trends/statuses/index.html.haml`:
  Not really a conflict, upstream change textually too close to a glitch-soc
  change (removed `javascript_pack_tag` to accomodate for glitch-soc's theming
  system).
  Ported upstream changes.
- `app/views/admin/trends/tags/index.html.haml`:
  Not really a conflict, upstream change textually too close to a glitch-soc
  change (removed `javascript_pack_tag` to accomodate for glitch-soc's theming
  system).
  Ported upstream changes.
2022-03-02 20:48:27 +01:00
Eugen Rochko 02b8d63fce
Fix report category not being saved in REST API (#17682) 2022-03-02 18:57:08 +01:00
Claire 425a6c90c4 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-02 18:02:48 +01:00
Eugen Rochko 50ea54b3ed
Change authorized applications page (#17656)
* Change authorized applications page

* Hide revoke button for superapps and suspended accounts

* Clean up db/schema.rb
2022-03-01 16:48:58 +01:00
Claire be493b6c0d Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/account.rb`:
  Not a real conflict, just upstream getting rid of unused constants too close
  to glitch-soc-specific contents.
  Removed unused constants like upstream did.
- `app/models/trends.rb`:
  Conflict because glitch-soc disabled email notifications for trending links.
  Upstream has refactored this quite a bit and added trending posts.
  Took upstream code, but disabling the extra trending stuff will come in
  another commit.
- `app/views/admin/trends/links/index.html.haml`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes accordingly.
2022-02-26 09:29:23 +01:00
Eugen Rochko 27965ce5ed
Add trending statuses (#17431)
* Add trending statuses

* Fix dangling items with stale scores in localized sets

* Various fixes and improvements

- Change approve_all/reject_all to approve_accounts/reject_accounts
- Change Trends::Query methods to not mutate the original query
- Change Trends::Query#skip to offset
- Change follow recommendations to be refreshed in a transaction

* Add tests for trending statuses filtering behaviour

* Fix not applying filtering scope in controller
2022-02-25 00:34:14 +01:00
Eugen Rochko a29a982eaa
Change e-mail domain blocks to block IPs dynamically (#17635)
* Change e-mail domain blocks to block IPs dynamically

* Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

* Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-24 17:28:23 +01:00
Sumak 91cc8d1e63
fix #17601 change `:keys` to `:attribute_names` (#17637) 2022-02-24 14:46:21 +01:00
Claire 3d25fe0426 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream changed the workflow a bit.
  Conflict comes from us pushing to ghcr while upstream pushes to dockerhub.
  Ported the upstream changes while still pushing to ghcr.
2022-02-23 18:13:12 +01:00
luzpaz 73f5e4a1d9
Fix various typos (#17621)
Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,ro`
2022-02-22 20:14:17 +01:00
Claire f224237862 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `db/schema.rb`:
  Conflict due to glitch-soc adding the `content_type` column on status edits
  and thus having a different schema version number.
  Solved by taking upstream's schema version number, as it is higher than
  glitch-soc's.
2022-02-17 10:58:44 +01:00
Claire 8f537a1168
Change relays handling to not record boosts (#17571)
* Change relays handling to not record boosts

* Update tests
2022-02-16 14:36:44 +01:00
Jeong Arm 2fd2666eea
Add test for user matching ip (#17572) 2022-02-16 13:14:53 +01:00
Eugen Rochko 564efd0651
Add appeals (#17364)
* Add appeals

* Add ability to reject appeals and ability to browse pending appeals in admin UI

* Add strikes to account page in settings

* Various fixes and improvements

- Add separate notification setting for appeals, separate from reports
- Fix style of links in report/strike header
- Change approving an appeal to not restore statuses (due to federation complexities)
- Change style of successfully appealed strikes on account settings page
- Change account settings page to only show unappealed or recently appealed strikes

* Change appealed_at to overruled_at

* Fix missing method error
2022-02-14 21:27:53 +01:00
Claire 51cad4f7fb Merge branch 'main' into glitch-soc/merge-upstream 2022-02-11 21:53:20 +01:00
Claire d4e6774a0c
Fix Undo Announce sometimes inlining the originally Announced status (#17516)
* Change tests to have more specific expectations on sent ActivityPub payloads

* Check that payload doesn't actually contain the contents of the boosted toot

* Fix Undo Announce sometimes inlining the originally Announced status
2022-02-11 14:52:07 +01:00
Claire 472bc89611
Fix some flaky tests that randomly failed because of item ordering (#17509) 2022-02-10 22:00:10 +01:00
Claire 2af03164cb
Improve tests involving push_bulk (#17508)
sidekiq-bulk's push_bulk can either accept arguments directly or run them
through a block.

Setting expectations on the result of evaluating the blocks allows testing
more code (the block itself) and the test is moved closer to the *interface*
of the tested code than its precise implementation.
2022-02-10 19:42:45 +01:00
Claire f1a6f9062e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
  Upstream moved things around in a place where glitch-soc had support for
  an extra parameter (`content_type`).
  Follow upstream but reintroduce `content_type`.
2022-02-10 19:09:27 +01:00
Claire da91b18a8b
Fix NoMethodError in StatusUpdateDistributionWorker (#17499)
* Add tests

* Fix NoMethodError in StatusUpdateDistributionWorker

* Fix tests
2022-02-10 14:57:10 +01:00
Claire 63854bee6c
Fix poll votes not being properly reset on poll change (#17498)
* Fix poll votes not being properly reset on poll change

* Fix and add tests

* Fix poll update handling when the number of options changes
2022-02-10 14:26:54 +01:00
Eugen Rochko 63002cde03
Add editing for published statuses (#17320)
* Add editing for published statuses

* Fix change of multiple-choice boolean in poll not resetting votes

* Remove the ability to update existing media attachments for now
2022-02-10 00:15:30 +01:00
Claire 322e907e04 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Upstream renamed some helper functions that were used in a part of the
  settings page which glitch-soc slightly changed the layout of.
  Ported the change.
2022-02-09 17:28:33 +01:00
Eugen Rochko 3aebe711fd
Change languages to be listed under standard instead of native name in admin UI (#17485) 2022-02-09 04:15:38 +01:00
Claire 692963d43b Merge branch 'main' into glitch-soc/merge-upstream 2022-02-08 18:23:53 +01:00
Eugen Rochko b6d7726ecb
Remove language detection through cld3 (#17478)
* Remove language detection through cld3

* Update app/helpers/languages_helper.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-08 02:41:17 +01:00
Eugen Rochko f1f6ddd536
Fix structured data parsing from links choking on bad data (#17403)
* Fix structured data parsing from links choking on bad data

- Fix og:url meta tag being prioritized over canonical link tag
- Fix structured data parsing choking on commented-out CDATA declarations
- Fix HTML entities in title, description, provider_name, author_name
- Change structured data parsing to attempt every JSON-LD script tag

* Remove unnecessary slash escapes from CDATA regex pattern
2022-02-07 18:16:31 +01:00
Claire 73a782391c
Fix replies collection incorrectly looping (#17462)
* Refactor tests

* Add tests

* Fix replies collection incorrectly looping
2022-02-07 17:06:43 +01:00
Claire aa832d623a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `CHANGELOG.md`:
  Upstream added newlines.
  Conflicts are because the CHANGELOG was independently merged from 3.4.6 on
  last security update.
  Took upstream's version.
- `app/helpers/context_helper.rb`:
  Conflicts because of extra vocabulary in glitch-soc. The conflicts were
  actually handled in last security merge.
  Kept our version.
2022-02-06 15:34:42 +01:00
Claire 92658f0fb0
Fix instance actor not being dereferenceable (#17457)
* Add tests

* Fix instance actor not being dereferenceable

* Fix tests

* Fix tests for real
2022-02-06 15:31:03 +01:00
Claire c8b1e72a4f
Fix compacted JSON-LD possibly causing compatibility issues on forwarding (#17428) 2022-02-03 14:09:04 +01:00
Claire 7b969436a0 Fix compacted JSON-LD possibly causing compatibility issues on forwarding 2022-02-02 19:55:57 +01:00
Claire 7679ddcd5e Merge branch 'main' into glitch-soc/merge-upstream 2022-01-30 22:33:30 +01:00
Claire f5639e1cbe
Change public profile pages to be disabled for unconfirmed users (#17385)
Fixes #17382

Note that unconfirmed and unapproved accounts can still be searched for
and their (empty) account retrieved using the REST API.
2022-01-28 14:24:37 +01:00
Claire b2915613fb Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Upstream-updated lib textually too close to glitch-soc-only dep.
  Updated like upstream.
2022-01-28 08:58:32 +01:00
Claire e38fc319dc
Refactor and improve tests (#17386)
* Change account and user fabricators to simplify and improve tests

- `Fabricate(:account)` implicitly fabricates an associated `user` if
  no `domain` attribute is given (an account with `domain: nil` is
  considered a local account, but no user record was created), unless
  `user: nil` is passed
- `Fabricate(:account, user: Fabricate(:user))` should still be possible
  but is discouraged.

* Fix and refactor tests

- avoid passing unneeded attributes to `Fabricate(:user)` or
  `Fabricate(:account)`
- avoid embedding `Fabricate(:user)` into a `Fabricate(:account)` or the other
  way around
- prefer `Fabricate(:user, account_attributes: …)` to
  `Fabricate(:user, account: Fabricate(:account, …)`
- also, some tests were using remote accounts with local user records, which is
  not representative of production code.
2022-01-28 00:46:42 +01:00
Claire ad6ddb9bdd Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `config/environments/production.rb`:
  Upstream changed a header but we had different default headers.
  Applied the same change, and also dropped HSTS headers redundant with
  Rails'.
2022-01-26 22:32:21 +01:00
Eugen Rochko 6505b39e5d
Fix poll updates being saved as status edits (#17373)
Fix #17344
2022-01-26 18:05:39 +01:00
Claire 61ef81c548 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `spec/models/status_spec.rb`:
  Upstream added tests too close to glitch-soc-specific tests.
  Kept both tests.
2022-01-23 18:24:01 +01:00
Claire 0a120d86d2
Fix error-prone SQL queries (#15828)
* Fix error-prone SQL queries in Account search

While this code seems to not present an actual vulnerability, one could
easily be introduced by mistake due to how the query is built.

This PR parameterises the `to_tsquery` input to make the query more robust.

* Harden code for Status#tagged_with_all and Status#tagged_with_none

Those two scopes aren't used in a way that could be vulnerable to an SQL
injection, but keeping them unchanged might be a hazard.

* Remove unneeded spaces surrounding tsquery term

* Please CodeClimate

* Move advanced_search_for SQL template to its own function

This avoids one level of indentation while making clearer that the SQL template
isn't build from all the dynamic parameters of advanced_search_for.

* Add tests covering tagged_with, tagged_with_all and tagged_with_none

* Rewrite tagged_with_none to avoid multiple joins and make it more robust

* Remove obsolete brakeman warnings

* Revert "Remove unneeded spaces surrounding tsquery term"

The two queries are not strictly equivalent.

This reverts commit 86f16c537e06c6ba4a8b250f25dcce9f049023ff.
2022-01-23 18:10:10 +01:00
Claire fe89554a54 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
  Upstream refactored how `Create` activities are handled and how values are
  extracted from `Create`d objects. This conflicted with how glitch-soc
  supported the `directMessage` flag to explicitly distinguish between
  limited and direct messages.
  Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Upstream largely refactored that file and changed some of the logic.
  This conflicted with glitch-soc's handling of the direct timeline and
  the options to allow replies and boosts in public feeds.
  Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored to move mention-related ActivityPub deliveries to
  `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
  to not send local-only toots to remote mentioned users.
  Took upstream's version, as the check is not needed anymore, since it is
  performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
  Upstream added support for `update` toot events, while glitch-soc had
  support for an extra timeline support, `direct`.
  Ported upstream changes and extended them to the `direct` timeline.

Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
  Added code to handle the `directMessage` flag and take it into account
  to compute visibility.
- `app/lib/feed_manager.rb`:
  Extended upstream's support of `update` toot events to glitch-soc's
  `direct` timeline.
2022-01-19 23:52:48 +01:00
Eugen Rochko 1060666c58
Add support for editing for published statuses (#16697)
* Add support for editing for published statuses

* Fix references to stripped-out code

* Various fixes and improvements

* Further fixes and improvements

* Fix updates being potentially sent to unauthorized recipients

* Various fixes and improvements

* Fix wrong words in test

* Fix notifying accounts that were tagged but were not in the audience

* Fix mistake
2022-01-19 22:37:27 +01:00
Claire b3bf32a21e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/reports/show.html.haml`:
  Conflicts due to glitch-soc's theming system.
2022-01-17 10:45:25 +01:00
Eugen Rochko 14f436c457
Add notifications for statuses deleted by moderators (#17204) 2022-01-17 09:41:33 +01:00
Claire 2d4b4b0b45 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/accounts_controller.rb`:
  Upstream introduced support for private pinned toots, but glitch-soc's query
  was a bit different as it filtered out local-only toots.
  Used upstream's query, while adding local-only filtering back.
- `app/controllers/activitypub/collections_controller.rb`:
  Same thing with regards to local-only posts.
- `app/validators/status_pin_validator.rb`:
  Not a real conflict, but the line below was different in glitch-soc due to
  the configurable pinned toots limit.
2022-01-17 08:28:52 +01:00
Claire d5c9feb7b7
Add support for private pinned posts (#16954)
* Add support for private pinned toots

* Allow local user to pin private toots

* Change wording to avoid "direct message"
2022-01-17 00:49:55 +01:00
Claire b61c3ddff8 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream added some text, but our README is completely different.
  Kept our README unchanged.
2022-01-16 18:11:20 +01:00
Eugen Rochko 8e84ebf0cb
Remove IP tracking columns from users table (#16409) 2022-01-16 13:23:50 +01:00
Claire d4f315f304 Merge branch 'main' into glitch-soc/merge-upstream 2021-12-18 11:44:54 +01:00
Claire 76761d5fc0
Add ability for admins to delete canonical email blocks (#16644)
* Add admin option to remove canonical email blocks from a deleted account

* Add tootctl canonical_email_blocks to inspect and remove canonical email blocks
2021-12-17 23:02:14 +01:00
Claire 7f803c41e2
Add ability to purge undeliverable domains from admin interface (#16686)
* Add ability to purge undeliverable domains from admin interface

* Add tests
2021-12-17 23:01:21 +01:00
Claire b2526316f5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/pending_accounts/index.html.haml`:
  Removed upstream, while it had glitch-soc-specific changes to accomodate
  for glitch-soc's theming system.
  Removed the file.

Additional changes:
- `app/views/admin/accounts/index.html.haml':
  Accomodate for glitch-soc's theming system.
2021-12-16 16:20:44 +01:00
Eugen Rochko 0fb9536d38
Add batch suspend for accounts in admin UI (#17009) 2021-12-05 21:48:39 +01:00
Claire 9b861d56a9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Copied upstream changes.
- `app/controllers/settings/identity_proofs_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `app/controllers/well_known/keybase_proof_config_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `lib/mastodon/statuses_cli.rb`:
  Minor conflict due to an optimization that wasn't shared between
  the two versions. Copied upstream's version.
2021-11-26 22:53:55 +01:00
Jeong Arm 1c826471e7
Fix admin statuses order(#16937) (#16969)
* Fix #16937

* Add test for statuses order
2021-11-26 22:12:27 +01:00
Eugen Rochko 7de0ee7aba
Remove Keybase integration (#17045) 2021-11-26 05:58:18 +01:00
Claire afd71867ba Revert "Fix trends admin page crashing"
This reverts commit 5f10e64330.
2021-11-26 01:15:29 +01:00
Claire 5f10e64330 Fix trends admin page crashing 2021-11-26 00:50:13 +01:00
Claire 443ec4f8ba Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/tags/index.html.haml`:
  Removed upstream while it had changes in glitch-soc to accomodate for the
  theming system.

Additional changes to accomodate for the theming system:
- `app/views/admin/trends/links/preview_card_providers/index.html.haml`
- `app/views/admin/trends/links/index.html.haml`
- `app/views/admin/trends/tags/index.html.haml`
- `app/views/admin/tags/show.html.haml`
2021-11-25 23:50:35 +01:00
Claire 013bee6afb
Fix filtering DMs from non-followed users (#17042) 2021-11-25 23:46:30 +01:00
Claire 534b0a2704 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-25 18:42:18 +01:00
Eugen Rochko 6e50134a42
Add trending links (#16917)
* Add trending links

* Add overriding specific links trendability

* Add link type to preview cards and only trend articles

Change trends review notifications from being sent every 5 minutes to being sent every 2 hours

Change threshold from 5 unique accounts to 15 unique accounts

* Fix tests
2021-11-25 13:07:38 +01:00
Claire 02a87431cf
Fix error when suspending user with an already-existing canonical email block (#17036)
* Fix error when suspending user with an already-existing canonical email block

Fixes #17033

While attempting to create a `CanonicalEmailBlock` with an existing hash would
raise an `ActiveRecord::RecordNotUnique` error, this being done within a
transaction would cancel the whole transaction. For this reason, checking for
uniqueness in Rails would query the database within the transaction and avoid
invalidating the whole transaction for this reason.

A race condition is still possible, where multiple accounts sharing a canonical
email would be blocked in concurrent transactions, in which only one would
succeed, but that is way less likely to happen that the current issue, and can
always be retried after the first failure, unlike the current situation.

* Add tests
2021-11-24 17:41:03 +01:00
Claire 4bb6b1a1e7 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-06 00:15:38 +01:00
Claire 87085a5152
Fix AccountNote not having a maximum length (#16942) 2021-11-06 00:12:25 +01:00
Eugen Rochko 39cdf61ab7
Add support for structured data and more OpenGraph tags to link cards (#16938)
Save preview cards under their canonical URL

Increase max redirects to follow from 2 to 3
2021-11-05 23:23:05 +01:00
Claire 989c67d29d
Fix handling announcements with links (#16941)
Broken since #15827
2021-11-05 21:14:35 +01:00
Claire e855df149b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream added new configuration options, uncommented by default.
  Commented them.
- `Gemfile.lock`:
  Upstream updated dependencies textually close to glitch-soc-specific
  dependencies.
  Updated those upstream dependencies.
2021-10-25 20:28:51 +02:00
Claire ec059317fa
Fix some link previews being incorrectly generated from other prior links (#16885)
* Add tests

* Fix some link previews being incorrectly generated from different prior links

PR #12403 added a cache to avoid redundant queries when the OEmbed endpoint can
be guessed from the URL. This caching mechanism is not perfectly correct as
there is no guarantee that all pages from a given domain share the same
OEmbed provider endpoint.

This PR prevents the FetchOEmbedService from caching OEmbed endpoint that
cannot be generalized by replacing a fully-qualified URL from the endpoint's
parameters, greatly reducing the number of incorrect cached generalizations.
2021-10-21 20:39:35 +02:00
Claire 3f9b28ce26
Add support for fetching Create and Announce activities by URI (#16383)
* Add support for fetching Create and Announce activities by URI

This should improve compatibility with ZAP and offer a way to fetch boosts,
which is currently not possible.

* Add tests
2021-10-21 01:14:04 +02:00
Takeshi Umeda 17f4e457b3
Add remove from followers api (#16864)
* Add followed_by? to account_interactions

* Add RemoveFromFollowersService

* Fix AccountBatch to use RemoveFromFollowersService

* Add remove from followers API
2021-10-18 12:02:35 +02:00
Claire b2eaf5562d Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream added some documentation in the README, but our README is pretty
  different and barebone.
  Kept our README.
2021-10-14 22:14:26 +02:00
Claire 694c073d1f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/admin/dashboard_controller.rb`:
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
- `app/views/admin/dashboard/index.html.haml`
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
2021-10-14 21:44:57 +02:00
Jeong Arm f4081d1564
Allow keeping only 1 boosts/favs on auto deleting posts (#16653)
* Allow keeping 1 boosts/favs on auto deleting posts

* Fix tests
2021-10-14 21:11:14 +02:00
Claire 216570ad98
Fix scheduled statuses decreasing statuses counts (#16791)
* Add tests

* Fix scheduled statuses decreasing statuses counts

Fixes #16774
2021-10-14 19:59:21 +02:00
Claire 1665b84285 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, just a glitch-soc-only dependency textually too close
  to an updated upstream one.
- `yarn.lock`:
  Not really a conflict, just a glitch-soc-only dependency textually too close
  to an updated upstream one.
2021-10-13 15:50:24 +02:00
Claire 5159ba26e4
Fix error when rendering public pages with media attachments (#16763)
* Add tests

* Fix error when rendering public pages with media attachments

* Add tests

* Fix tests

* Please CodeClimate
2021-10-13 15:27:19 +02:00
Claire 079ed6c26d Merge branch 'main' into glitch-soc/merge-upstream 2021-10-06 16:46:32 +02:00
Claire 84ceebe1c4
Fix media attachment size validation not correctly accounting for file type (#16819)
* Fix media attachment size validation not correctly accounting for file type

Fixes a regression introduced in #16724 caused by the fact that kt-paperclip
now correctly runs validations before processing, meaning that file size
verification could not rely on our before_post_processing hook.

Moved the `before_post_processing` hooks to `before_validate` to make sure
the media attachment type is set correctly before the file gets validated.

* Add tests
2021-10-06 14:49:32 +01:00
Claire 56f9c44f33 Merge branch 'main' into glitch-soc/merge-upstream 2021-09-30 11:45:13 +02:00
Claire 24f9ea7818
Fix webauthn secure key authentication (#16792)
* Add tests

* Fix webauthn secure key authentication

Fixes #16769
2021-09-30 05:26:29 +02:00
Claire 29b5c5f483 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/custom_emoji.rb`:
  Slight refactor upstream, next to a line that was different in glitch-soc
  because of our local configurable limits on custom emoji size.
  Ported upstream changes.
- `yarn.lock`:
  Not really a conflict, upstream dependency textually too close to a
  glitch-soc-only dependency.
  Updated upstream dependency as upstream.
2021-09-30 00:01:20 +02:00
Claire fc3ae1343d
Switch from unmaintained paperclip to kt-paperclip (#16724)
* Switch from unmaintained paperclip to kt-paperclip

* Drop some compatibility monkey-patches not required by kt-paperclip

* Drop media spoof check monkey-patching

It's broken with kt-paperclip and hopefully it won't be needed anymore

* Fix regression introduced by paperclip 6.1.0

* Do not rely on pathname to call FastImage

* Add test for ogg vorbis file with cover art

* Add audio/vorbis to the accepted content-types

This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…

* Restore missing for_as_default method

* Refactor Attachmentable concern and delay Paperclip's content-type spoof check

Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.

* Please CodeClimate

* Add audio/vorbis to the unreliable set

It doesn't correspond to a file format and thus has no extension associated.
2021-09-29 23:52:36 +02:00
Claire 3622110778 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `streaming/index.js`:
  Filtering code for streaming notifications has been refactored upstream, but
  glitch-soc had similar code for local-only toots in the same places.
  Ported upstream changes, but did not refactor local-only filtering.
2021-09-26 18:28:59 +02:00
Eugen Rochko 52e5c07948
Change routing paths to use usernames in web UI (#16171) 2021-09-26 05:46:13 +02:00
Claire bf827b17e1 Merge branch 'main' into glitch-soc/merge-upstream 2021-09-19 13:07:27 +02:00
Claire e0af97164a
Fix followers synchronization mechanism not working when URI has empty path (#16744)
Follow-up to #16510, forgot the controller exposing the actual followers…
2021-09-15 18:51:16 +02:00
Claire f1e6884d6b Merge branch 'main' into glitch-soc/merge-upstream 2021-09-14 13:50:30 +02:00
Claire db57bff11d
Stop setting a shortcode to newly-created media attachments (#16730)
* Stop setting a shortcode to newly-created media attachments

The WebUI has stopped using the “short media URL” in ages. This isn't used
anywhere except for mail notifications.

Deprecating it would allow us to eventually get rid of at least a database
column and corruption-prone index, as well as a controller.

* Fix tests
2021-09-13 18:59:37 +02:00
Claire 12cd097e7c
Fix addressing of remote groups' followers (#16700)
Fixes #16699
2021-09-08 23:33:23 +02:00
Claire f9185c72a9 Merge branch 'main' into glitch-soc/merge-upstream 2021-09-02 10:52:09 +02:00
Claire 2b18f7a943
Fix processing mentions to domains with non-ascii TLDs (#16689)
Fixes #16602
2021-09-01 22:06:40 +02:00
Claire 463d23dfd5 Merge branch 'main' into glitch-soc/merge-upstream 2021-08-27 11:29:03 +02:00
Claire 94bcf45321
Fix authentication failures after going halfway through a sign-in attempt (#16607)
* Add tests

* Add security-related tests

My first (unpublished) attempt at fixing the issues introduced (extremely
hard-to-exploit) security vulnerabilities, addressing them in a test.

* Fix authentication failures after going halfway through a sign-in attempt

* Refactor `authenticate_with_sign_in_token` and `authenticate_with_two_factor` to make the two authentication steps more obvious
2021-08-25 22:52:41 +02:00
Claire 4e2a8c9b38 Merge branch 'main' into glitch-soc/merge-upstream 2021-08-20 13:01:50 +02:00
Claire 6702148472
Add tests for SuspendAccountService and UnsuspendAccountService (#16627)
* Add tests for SuspendAccountService

* Add tests for UnsuspendAccountService
2021-08-20 10:53:33 +01:00
Claire 5efb1ff337
Fix followers synchronization mechanism not working when URI has empty path (#16510)
* Fix followers synchronization mechanism not working when URI has empty path

To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.

Also adds tests and rename them to reflect the proper method names.

* Move url prefix regexp to its own constant
2021-08-11 17:48:42 +02:00
Claire 47ea9b2d67 Merge branch 'main' into glitch-soc/merge-upstream 2021-08-11 16:41:59 +02:00
Claire 13b08610a0
Fix crash when encountering invalid account fields (#16598)
* Add test

* Fix crash when encountering invalid account fields
2021-08-11 16:40:55 +02:00
Claire 8ec4be4233 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream-updated dependency (redis) textually too
  close to glitch-soc-only dependecy.
  Updated redis gem like upstream did.
2021-08-09 23:25:49 +02:00
Claire 4ac78e2a06
Add feature to automatically delete old toots (#16529)
* Add account statuses cleanup policy model

* Record last inspected toot to delete to speed up successive calls to statuses_to_delete

* Add service to cleanup a given account's statuses within a budget

* Add worker to go through account policies and delete old toots

* Fix last inspected status id logic

All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.

* Add tests

* Refactor scheduler and add tests

* Add user interface

* Add support for discriminating based on boosts/favs

* Add UI support for min_reblogs and min_favs, rework UI

* Address first round of review comments

* Replace Snowflake#id_at_start with with_random parameter

* Add tests

* Add tests for StatusesCleanupController

* Rework settings page

* Adjust load-avoiding mechanisms

* Please CodeClimate
2021-08-09 23:11:50 +02:00
Claire 763ab0c7eb
Fix owned account notes not being deleted when an account is deleted (#16579)
* Add account_notes relationship

* Add tests

* Fix owned account notes not being deleted when an account is deleted

* Add post-migration to clean up orphaned account notes
2021-08-08 15:29:57 +02:00
Claire d1c87be8cf Merge branch 'main' into glitch-soc/merge-upstream 2021-07-17 17:45:35 +02:00
Claire 30ce6e395c
Fix user email address being banned on self-deletion (#16503)
* Add tests

* Fix user email address being banned on self-deletion

Fixes #16498
2021-07-14 05:35:49 +02:00
Claire fc500a6062 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `CONTRIBUTING.md`:
  Upstream file has changed and we're quoting it.
  Ported the changes.
- `README.md`:
  Upstream file has changed but we have a completely different one.
  Kept our version.
- `lib/mastodon/version.rb`:
  Upstream has changed from `tootsuite/mastodon` to `mastodon/mastodon`,
  but we're still `glitch-soc/mastodon`.
  Kept our version.
- `spec/presenters/instance_presenter_spec.rb`:
  Upstream has changed from `tootsuite/mastodon` to `mastodon/mastodon`,
  but we're still `glitch-soc/mastodon`.
  Kept our version.
2021-07-13 17:57:15 +02:00
Claire 5a1e072517
Change references to tootsuite/mastodon to mastodon/mastodon (#16491)
* Change references to tootsuite/mastodon to mastodon/mastodon

* Remove obsolete test fixture

* Replace occurrences of tootsuite/mastodon with mastodon/mastodon in CHANGELOG

And a few other places
2021-07-13 15:46:20 +02:00
Claire 3160e050a9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/helpers/accounts_helper.rb`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
- `app/views/accounts/_header.html.haml`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
- `app/views/directories/index.html.haml`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
2021-07-08 16:17:19 +02:00
Eugen Rochko 771c9d4ba8
Add ability to skip sign-in token authentication for specific users (#16427)
Remove "active within last two weeks" exception for sign in token requirement

Change admin reset password to lock access until the password is reset
2021-07-08 05:31:28 +02:00
Claire 225c6582d1
Add tests for BootstrapTimelineService (#16476) 2021-07-07 21:12:43 +02:00
Ikko Ashimine 67226acf7e
Fix typo in tag_feed_spec.rb (#16466)
existant -> existent
2021-07-05 19:16:21 +02:00
Claire 49219508bc
Fix anonymous access to outbox not being cached by the reverse proxy (#16458)
* Fix anonymous access to outbox not being cached by the reverse proxy

Up until now, anonymous access to outbox was marked as public, but with a
0 duration for caching, which means remote proxies would only serve from cache
when the server was completely overwhelmed.

Changed that cache duration to one minute, so that repeated anonymous access
to one account's outbox can be appropriately cached.

Also added `Signature` to the `Vary` header in case a page is requested, so
that authenticated fetches are never served from cache (which only contains
public toots).

* Remove Vary: Accept header from webfinger controller

Indeed, we have stopped returning xrd, and only ever return jrd, so the
Accept request header does not matter anymore.

* Cache negative webfinger hits for 3 minutes
2021-07-03 21:13:47 +02:00
Claire 37fd9670a5 Merge branch 'main' into glitch-soc/merge-upstream 2021-06-23 10:46:24 +02:00
Claire 63b807cffc
Fix serialization of followers/following counts when user hides their network (#16418)
* Add tests

* Fix serialization of followers/following counts when user hides their network

Fixes #16382

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2021-06-21 20:14:47 +02:00
Eugen Rochko d174d12c83
Add authentication history (#16408) 2021-06-21 17:07:30 +02:00
Claire 7ac8bcf209 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-10 21:45:32 +02:00
Claire afb7882189
Fix blocking someone not clearing up list feeds (#16205) 2021-05-10 17:31:55 +02:00
Claire fbf097f7e1 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-08 00:01:47 +02:00
Eugen Rochko 1294f9ee4f
Remove PubSubHubbub-related columns from accounts table (#16170) 2021-05-07 19:32:58 +02:00
Claire 50b430d9a2 Merge branch 'main' into glitch-soc/merge-upstream
- `app/views/statuses/_simple_status.html.haml`:
  Small markup change in glitch-soc, on a line that has been modified by
  upstream. Ported upstream changes.
2021-05-07 18:21:59 +02:00
Eugen Rochko 74081433d0
Change trending hashtags to be affected be reblogs (#16164)
If a status with a hashtag becomes very popular, it stands to
reason that the hashtag should have a chance at trending

Fix no stats being recorded for hashtags that are not allowed
to trend, and stop ignoring bots

Remove references to hashtags in profile directory from the code
and the admin UI
2021-05-07 14:33:43 +02:00
Eugen Rochko 2c77d97e0d
Add joined date to profiles in web UI (#16169) 2021-05-07 14:33:19 +02:00
Claire 566fc90913
Add Ruby 3.0 support (#16046)
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0

Also improve the Terrapin monkey-patch for the stderr/stdout issue.

* Fix keyword argument handling throughout the codebase

* Monkey-patch Paperclip to fix keyword arguments handling in validators

* Change validation_extensions to please CodeClimate

* Bump microformats from 4.2.1 to 4.3.1

* Allow Ruby 3.0

* Add Ruby 3.0 test target to CircleCI

* Add test for admin dashboard warnings

* Fix admin dashboard warnings on Ruby 3.0
2021-05-06 14:22:54 +02:00
Claire 8c44b723bb
Change confirmations controller to redirect to / for approved users (#16151)
Clicking the confirmation link multiple times currently leads to entering
account settings, which can be confusing. This commit changes that so that
it redirects to the root path, so it behaves the same way as clicking only
once in most cases.
2021-05-03 15:45:19 +02:00
abcang 7f0c49c58a
Improve tag search query (#16104) 2021-04-25 06:33:28 +02:00
Eugen Rochko daccc07dc1
Change auto-following admin-selected accounts, show in recommendations (#16078) 2021-04-24 17:01:43 +02:00
Claire a6564d56d6
Fix edge case where accepted follow cannot be processed because of follow limit (#16098) 2021-04-23 22:51:21 +02:00
Eugen Rochko 9cc283f0b4
Change the nouns "toot" and "status" to "post" (#16080) 2021-04-21 18:31:24 +02:00
Claire 536892b8ae Merge branch 'main' into glitch-soc/merge-upstream 2021-04-21 13:52:03 +02:00
Claire 0b36e3419d
Fix processing of remote Delete activities (#16084)
* Add tests

* Ensure deleted statuses are marked as such

* Save some redis memory by not storing URIs in delete_upon_arrival values

* Avoid possible race condition when processing incoming Deletes

* Avoid potential duplicate Delete forwards

* Lower lock durations to reduce issues in case of hard crash of the Rails process

* Check for `lock.aquired?` and improve comment

* Refactor RedisLock usage in app/lib/activitypub

* Fix using incorrect or non-existent sender for relaying Deletes
2021-04-21 04:46:09 +02:00
Claire e2a2bc9021 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream updated copyright year, we don't mention it so kept our version.
- `app/controllers/admin/dashboard_controller.rb`:
  Not really a conflict, upstream change (removing the spam checker) too close
  to glitch-soc changes. Ported upstream changes.
- `app/models/form/admin_settings.rb`:
  Same.
- `app/services/remove_status_service.rb`:
  Same.
- `app/views/admin/settings/edit.html.haml`:
  Same.
- `config/settings.yml`:
  Same.
- `config/environments/production.rb`:
  Not a real conflict, upstream added a default HTTP header, but we have
  extra headers in glitch-soc.
  Added the header.
2021-04-20 12:17:14 +02:00
Eugen Rochko b3ceb3dcc4
Add canonical e-mail blocks for suspended accounts (#16049)
Prevent new accounts from being created using the same underlying
e-mail as a suspended account using extensions and period
permutations. Stores e-mails as a SHA256 hash
2021-04-17 03:14:25 +02:00
Eugen Rochko 3b8d085436
Fix app name, website and redirect URIs not having a maximum length (#16042)
Fix app scopes not being validated
2021-04-15 16:28:43 +02:00
Eugen Rochko ce2148c571
Add `policy` param to `POST /api/v1/push/subscriptions` (#16040)
With possible values `all`, `followed`, `follower`, and `none`,
control from whom notifications will generate a Web Push alert
2021-04-15 05:00:25 +02:00
Eugen Rochko 120965eb0b
Change Web Push API deliveries to use request pooling (#16014) 2021-04-12 14:25:34 +02:00
Eugen Rochko f7117646af
Add cold-start follow recommendations (#15945) 2021-04-12 12:37:14 +02:00
Eugen Rochko 619fad6cf8
Remove spam check and dependency on nilsimsa gem (#16011) 2021-04-11 11:22:50 +02:00
Eugen Rochko 7183d9a113
Change multiple mentions with same username to render with domain (#15718)
Fix #15506
2021-04-10 11:51:02 +02:00
Claire 5c225b03db Fix glitch-soc-specific health check spec 2021-04-05 15:21:38 +02:00
Claire 15efa32cca Merge branch 'main' into glitch-soc/merge-upstream 2021-03-31 10:34:29 +02:00
Claire a650a1157d
Fix /admin/tags/:id crashing since Rails 6.1 update (#15953)
Raw SQL passed to `pluck` now has to be explicitly marked as SQL via
Arel.sql, see https://github.com/rails/rails/pull/27947
2021-03-26 18:36:16 +01:00
Claire f60c99a8fb Merge branch 'main' into glitch-soc/merge-upstream 2021-03-24 10:53:50 +01:00
Claire cbd0ee1d07
Update Mastodon to Rails 6.1 (#15910)
* Update devise-two-factor to unreleased fork for Rails 6 support

Update tests to match new `rotp` version.

* Update nsa gem to unreleased fork for Rails 6 support

* Update rails to 6.1.3 and rails-i18n to 6.0

* Update to unreleased fork of pluck_each for Ruby 6 support

* Run "rails app:update"

* Add missing ActiveStorage config file

* Use config.ssl_options instead of removed ApplicationController#force_ssl

Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.

* Fix nonce directives by removing Rails 5 specific monkey-patching

* Fix fixture_file_upload deprecation warning

* Fix yield-based test failing with Rails 6

* Use Rails 6's index_with when possible

* Use ActiveRecord::Cache::Store#delete_multi from Rails 6

This will yield better performances when deleting an account

* Disable Rails 6.1's automatic preload link headers

Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.

In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.

* Switch to Rails 6.0 default config

* Switch to Rails 6.1 default config

* Do not include autoload paths in the load path
2021-03-24 10:44:31 +01:00
Claire 1c4dee4554
Fix Mastodon not understanding as:Public and Public (#15948)
Fixes #5551
2021-03-24 10:19:40 +01:00
Claire 051efed5ed
Bypass MX validation for explicitly allowed domains (#15930)
* Bypass MX validation for explicitly allowed domains

This spares some lookups and prevent issues in some edge cases with
local domains.

* Add tests

* Fix test
2021-03-19 23:48:47 +01:00
Claire 3ad6ef72cb Merge branch 'main' into glitch-soc/merge-upstream 2021-03-19 13:57:15 +01:00
Claire 741d0952b1
Improve account counters handling (#15913)
* Improve account counters handling

* Use ActiveRecord::Base::sanitize_sql to pass values instead of interpolating them

Keep using string interpolation for `key` as it is safe and using
“ActiveRecord::Base::sanitize_sql_hash_for_assignment” would require stitching
bits of SQL in a way that is not more easily checked for safety.

* Add migration hook to catch PostgreSQL versions earlier than 9.5
2021-03-19 13:14:57 +01:00
Claire b358229834
Further preparation for Rails 6 (#15916)
* Use ActiveRecord::Result#to_ary instead of deprecated to_hash

They do the same thing, and to_hash has been removed from Rails 6.1

* Explicitly name polymorphic indexes to workaround a bug in Rails 6.1

cf. https://github.com/rails/rails/issues/41693

* Fix incorrect usage of “foreign_key” in migration script

* Use `ActiveModel::Errors#delete` instead of deprecated clear method

* Fix link headers tests on Rails 6.1

Rails 6.1 adds values to the Link header by default, thus it is not a
LinkHeader object anymore. Fix the test to parse the Link header instead
of assuming it is a LinkHeader.
2021-03-19 02:45:34 +01:00