Commit Graph

81 Commits (2c0eaf50f6441da34783d982519889818901d467)

Author SHA1 Message Date
jsgoldstein e82d84469e Add new public status index (#26344)
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-08-24 16:40:04 +02:00
Claire 58b52b7a33 Add stricter protocol fields validation for accounts (#25937) 2023-07-20 18:23:48 +02:00
Claire e8631f8e90 Remove unfollowed hashtag posts from home feed (#26028) 2023-07-17 13:56:28 +02:00
Matt Jankowski 011006c01b Fix `RSpec/VerifiedDoubles` cop (#25469) 2023-06-22 14:55:22 +02:00
Claire 4e0718dbf5 Add per-test timeouts to AutoStatusesCleanupScheduler tests (#24841) 2023-06-20 18:54:05 +02:00
Matt Jankowski eb1f69c2d4 Update rubocop-rspec to version 2.22.0, fix `RSpec/IndexedLet` cop (#24698) 2023-06-14 16:44:37 +02:00
Matt Jankowski 8c7da82c64 Speed improvement for `AccountsStatusesCleanupScheduler` spec (#25406) 2023-06-14 09:56:11 +02:00
Matt Jankowski bc1dae0bc4 Fix RSpec/ExpectChange cop (#25101) 2023-05-24 11:23:40 +02:00
Claire 53e3a1b06d Fix infinite loop in AccountsStatusesCleanupScheduler (#24840) 2023-05-04 10:13:00 +02:00
Matt Jankowski 8dec215a57 Fix RSpec/ContextWording cop (#24739) 2023-05-04 05:49:08 +02:00
Claire 90b45c5040 Fix various edge cases with local moves (#24812) 2023-05-03 19:19:25 +02:00
Claire a4e3f98fc7 Change Move handler to also move list memberships (#24808) 2023-05-03 14:03:38 +02:00
Claire ca3003835d Improve automatic post cleanup worker performances (#24785) 2023-05-03 10:31:40 +02:00
Claire abce7cf2b9 Rewrite import feature (#21054) 2023-05-02 12:08:48 +02:00
Claire 59a73ea537 Fix AccountsStatusesCleanupScheduler not spreading deletes across accounts correctly (#24607) 2023-04-23 22:25:40 +02:00
Christian Schmidt 72b4973f50 Requeue expiration notification (#24311) 2023-03-29 10:52:40 +02:00
Matt Jankowski 7a8257b638 Coverage improvement round-out following up previous work (#23987) 2023-03-10 13:33:30 +01:00
Matt Jankowski 9e213fd37d Add basic coverage for some worker jobs (#23943) 2023-03-04 16:56:09 +01:00
Stanislav Dobrovolschii eaea285368 Add rspecs for FollowRecommendationsScheduler (#23890) 2023-03-02 10:05:05 +01:00
Claire bc4837a600 Change auto-deletion throttling constants to better scale with server size (#23320) 2023-02-23 16:52:40 +01:00
Nick Schonning dd46810e8a Enable Style/FrozenStringLiteralComment for specs (#23790) 2023-02-22 09:55:31 +09:00
Nick Schonning 247bf6e812 Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning a4cbfac4d9 Autofix Rubocop RSpec/BeEq (#23740) 2023-02-20 06:14:50 +01:00
Nick Schonning c0f0c42326 Audofix Rubocop Style/WordArray (#23739) 2023-02-20 06:14:10 +01:00
Nick Schonning 754c417e29 Autofix Rubocop RSpec/LeadingSubject (#23670) 2023-02-20 13:24:14 +09:00
Nick Schonning f6c4699d73 Autofix Rubocop RSpec/EmptyLineAfterSubject (#23719) 2023-02-20 02:46:00 +01:00
Nick Schonning a25f6522da Enable Rubocop RSpec/NotToNot (#23723) 2023-02-20 02:33:27 +01:00
Nick Schonning 6988b04ec7 Autofix Rubocop Style/TrailingCommaInArrayLiteral (#23692) 2023-02-19 06:54:30 +09:00
Nick Schonning 31ca76fe2c Run rubocop formatting except line length (#23632) 2023-02-18 06:56:20 +09:00
Claire a173d5248f Fix `UserCleanupScheduler` crash when an unconfirmed account has a moderation note (#23318)
* Fix `UserCleanupScheduler` crash when an unconfirmed account has a moderation note

* Add tests
2023-02-07 01:14:44 +01:00
Claire 5437dda76b Fix followers count not being updated when migrating follows (#19998)
Fixes #19900
2022-11-07 15:38:55 +01:00
Eugen Rochko 7a482567f8 Add retention policy for cached content and media (#19232) 2022-09-27 03:08:19 +02:00
Eugen Rochko f4e91041b2 Add ability to filter followed accounts' posts by language (#19095) 2022-09-20 23:51:21 +02:00
Eugen Rochko 7300beadd9 Remove digest e-mails (#17985)
* Remove digest e-mails

* Remove digest-related code
2022-08-25 23:38:22 +02:00
Eugen Rochko 553889bc7c Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire 612c6255dc Improve tests involving push_bulk (#17508)
sidekiq-bulk's push_bulk can either accept arguments directly or run them
through a block.

Setting expectations on the result of evaluating the blocks allows testing
more code (the block itself) and the test is moved closer to the *interface*
of the tested code than its precise implementation.
2022-02-10 19:42:45 +01:00
Claire c6de53c1a9 Fix NoMethodError in StatusUpdateDistributionWorker (#17499)
* Add tests

* Fix NoMethodError in StatusUpdateDistributionWorker

* Fix tests
2022-02-10 14:57:10 +01:00
Claire 1f07ab014d Refactor and improve tests (#17386)
* Change account and user fabricators to simplify and improve tests

- `Fabricate(:account)` implicitly fabricates an associated `user` if
  no `domain` attribute is given (an account with `domain: nil` is
  considered a local account, but no user record was created), unless
  `user: nil` is passed
- `Fabricate(:account, user: Fabricate(:user))` should still be possible
  but is discouraged.

* Fix and refactor tests

- avoid passing unneeded attributes to `Fabricate(:user)` or
  `Fabricate(:account)`
- avoid embedding `Fabricate(:user)` into a `Fabricate(:account)` or the other
  way around
- prefer `Fabricate(:user, account_attributes: …)` to
  `Fabricate(:user, account: Fabricate(:account, …)`
- also, some tests were using remote accounts with local user records, which is
  not representative of production code.
2022-01-28 00:46:42 +01:00
Eugen Rochko 06b698a723 Add support for editing for published statuses (#16697)
* Add support for editing for published statuses

* Fix references to stripped-out code

* Various fixes and improvements

* Further fixes and improvements

* Fix updates being potentially sent to unauthorized recipients

* Various fixes and improvements

* Fix wrong words in test

* Fix notifying accounts that were tagged but were not in the audience

* Fix mistake
2022-01-19 22:37:27 +01:00
Claire 44c8b6ff2f Add ability to purge undeliverable domains from admin interface (#16686)
* Add ability to purge undeliverable domains from admin interface

* Add tests
2021-12-17 23:01:21 +01:00
Claire 925adbf7af Fix AccountNote not having a maximum length (#16942) 2021-11-06 00:12:25 +01:00
Claire 91bd8b921b Fix handling announcements with links (#16941)
Broken since #15827
2021-11-05 21:14:35 +01:00
Claire 1692e0b381 Fix followers synchronization mechanism not working when URI has empty path (#16510)
* Fix followers synchronization mechanism not working when URI has empty path

To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.

Also adds tests and rename them to reflect the proper method names.

* Move url prefix regexp to its own constant
2021-08-11 17:48:42 +02:00
Claire 1fcf310c60 Add feature to automatically delete old toots (#16529)
* Add account statuses cleanup policy model

* Record last inspected toot to delete to speed up successive calls to statuses_to_delete

* Add service to cleanup a given account's statuses within a budget

* Add worker to go through account policies and delete old toots

* Fix last inspected status id logic

All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.

* Add tests

* Refactor scheduler and add tests

* Add user interface

* Add support for discriminating based on boosts/favs

* Add UI support for min_reblogs and min_favs, rework UI

* Address first round of review comments

* Replace Snowflake#id_at_start with with_random parameter

* Add tests

* Add tests for StatusesCleanupController

* Rework settings page

* Adjust load-avoiding mechanisms

* Please CodeClimate
2021-08-09 23:11:50 +02:00
Eugen Rochko a8bf951926 Change Web Push API deliveries to use request pooling (#16014) 2021-04-12 14:25:34 +02:00
ThibG 82d014fd08 Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00
ThibG 694a5e33d5 Add follower synchronization mechanism (#14510)
* Add support for followers synchronization on the receiving end

Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.

* Add tests for followers synchronization on the receiving end

* Add support for follower synchronization on the sender's end

* Add tests for the sending end

* Switch from AS attributes to HTTP header

Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
   receiving instance (where “receiving instance” is determined by accounts
   sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor

Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.

* Add environment variable to disable followers synchronization

Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.

The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
   netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
   netloc as a managed account must be managed by the same Mastodon instance
   as well.

As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.

Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.

* Only set follower synchronization header on non-public statuses

This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)

* Change how ActivityPub::SynchronizeFollowersService handles follow requests

If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.

* Integrate review feeback

- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes

* Only select required fields when computing followers_hash

* Use actor URI rather than webfinger domain in synchronization endpoint

* Change hash computation to be a XOR of individual hashes

Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.

* Marginally improve followers_hash computation speed

* Further improve hash computation performances by using pluck_each
2020-10-21 18:04:09 +02:00
Eugen Rochko 02c4c91faa Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
Eugen Rochko 89fd56627c Fix various warnings in rspec (#14729) 2020-09-04 20:22:26 +02:00
ThibG 3e51b04d40 Change move handler to carry blocks over (#14144)
* Change move handler to carry blocks and mutes over

When user A blocks user B and B moves to a new account C, make A block C
accordingly.

Note that it only works if A's instance is aware of the Move, that is,
if B is on A's instance or has followers there.

* Also notify instances with known people blocking you when moving

* Add automatic account notes when blocking/muting an account that had no note
2020-07-01 13:51:15 +02:00