* Change tests to have more specific expectations on sent ActivityPub payloads
* Check that payload doesn't actually contain the contents of the boosted toot
* Fix Undo Announce sometimes inlining the originally Announced status
sidekiq-bulk's push_bulk can either accept arguments directly or run them
through a block.
Setting expectations on the result of evaluating the blocks allows testing
more code (the block itself) and the test is moved closer to the *interface*
of the tested code than its precise implementation.
* Add editing for published statuses
* Fix change of multiple-choice boolean in poll not resetting votes
* Remove the ability to update existing media attachments for now
* Change account and user fabricators to simplify and improve tests
- `Fabricate(:account)` implicitly fabricates an associated `user` if
no `domain` attribute is given (an account with `domain: nil` is
considered a local account, but no user record was created), unless
`user: nil` is passed
- `Fabricate(:account, user: Fabricate(:user))` should still be possible
but is discouraged.
* Fix and refactor tests
- avoid passing unneeded attributes to `Fabricate(:user)` or
`Fabricate(:account)`
- avoid embedding `Fabricate(:user)` into a `Fabricate(:account)` or the other
way around
- prefer `Fabricate(:user, account_attributes: …)` to
`Fabricate(:user, account: Fabricate(:account, …)`
- also, some tests were using remote accounts with local user records, which is
not representative of production code.
* Add support for editing for published statuses
* Fix references to stripped-out code
* Various fixes and improvements
* Further fixes and improvements
* Fix updates being potentially sent to unauthorized recipients
* Various fixes and improvements
* Fix wrong words in test
* Fix notifying accounts that were tagged but were not in the audience
* Fix mistake
* Add tests
* Fix some link previews being incorrectly generated from different prior links
PR #12403 added a cache to avoid redundant queries when the OEmbed endpoint can
be guessed from the URL. This caching mechanism is not perfectly correct as
there is no guarantee that all pages from a given domain share the same
OEmbed provider endpoint.
This PR prevents the FetchOEmbedService from caching OEmbed endpoint that
cannot be generalized by replacing a fully-qualified URL from the endpoint's
parameters, greatly reducing the number of incorrect cached generalizations.
* Add support for fetching Create and Announce activities by URI
This should improve compatibility with ZAP and offer a way to fetch boosts,
which is currently not possible.
* Add tests
* Add followed_by? to account_interactions
* Add RemoveFromFollowersService
* Fix AccountBatch to use RemoveFromFollowersService
* Add remove from followers API
* Add account statuses cleanup policy model
* Record last inspected toot to delete to speed up successive calls to statuses_to_delete
* Add service to cleanup a given account's statuses within a budget
* Add worker to go through account policies and delete old toots
* Fix last inspected status id logic
All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.
* Add tests
* Refactor scheduler and add tests
* Add user interface
* Add support for discriminating based on boosts/favs
* Add UI support for min_reblogs and min_favs, rework UI
* Address first round of review comments
* Replace Snowflake#id_at_start with with_random parameter
* Add tests
* Add tests for StatusesCleanupController
* Rework settings page
* Adjust load-avoiding mechanisms
* Please CodeClimate
* Add account_notes relationship
* Add tests
* Fix owned account notes not being deleted when an account is deleted
* Add post-migration to clean up orphaned account notes
* Change ResolveAccountService's handling of skip_webfinger
Change it so it never makes any webfinger query, as the name would imply.
* Add tests
* Change FollowService to not take an URI for target_account
* Restore domain-block check in FollowService
* Fix tests
* Delete status records by batches of 50
* Do not precompute values that are only used once
* Do not generate redis events for removal of public toots older than two weeks
* Filter reported toots a priori for polls and status deletion
* Do not process reblogs when cleaning up public timelines
As in Mastodon proper, reblogs don't appear in public TLs
* Clean the deleted account's own feed in one go
* Refactor Account#clean_feed_manager and List#clean_feed_manager
* Delete instead of destroy a few more associations
* Fix preloading
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix deleting polls not deleting notifications
* Fix fav notification deletion when deleting a toot
* Refactor DeleteAccountService spec
* Add DeleteAccountService tests for other associations and notifications
* Add favourite handling spec in status removal
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix ResolveAccountService accepting mismatching acct: URI
* Set attributes that should be updated regardless of suspension
* Fix key fetching
* Automatically merge remote accounts with duplicate `uri`
* Add tests
* Add "tootctl accounts fix-duplicates"
Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Improve searching for private toots from URL
Most of the time, when sharing toots, people use the toot URL rather than
the toot URI, which makes sense since it is the user-facing URL.
In Mastodon's case, the URL and URI are different, and Mastodon does not
have an index on URL, which means searching a private toot by URL is done
with a slow query that will only succeed for very recent toots.
This change gets rid of the slow query, and attempts to guess the URI from
URL instead, as Mastodon's are predictable.
* Add tests
* Only return status with guessed uri if url matches
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix webfinger redirect handling in ResolveAccountService
ResolveAccountService#process_webfinger! handled a one-step webfinger
redirection, but only accepting the result if it matched the exact URI passed
as input, defeating the point of a redirection check.
Instead, use the same logic as in `ActivityPub::FetchRemoteAccountService`,
updating the resulting `acct:` URI with the result of the first webfinger
query.
* Add tests
* Add support for followers synchronization on the receiving end
Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.
* Add tests for followers synchronization on the receiving end
* Add support for follower synchronization on the sender's end
* Add tests for the sending end
* Switch from AS attributes to HTTP header
Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
receiving instance (where “receiving instance” is determined by accounts
sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor
Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.
* Add environment variable to disable followers synchronization
Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.
The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
netloc as a managed account must be managed by the same Mastodon instance
as well.
As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.
Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.
* Only set follower synchronization header on non-public statuses
This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)
* Change how ActivityPub::SynchronizeFollowersService handles follow requests
If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.
* Integrate review feeback
- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes
* Only select required fields when computing followers_hash
* Use actor URI rather than webfinger domain in synchronization endpoint
* Change hash computation to be a XOR of individual hashes
Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.
* Marginally improve followers_hash computation speed
* Further improve hash computation performances by using pluck_each
* Add bell button
Fix#4890
* Remove duplicate type from post-deployment migration
* Fix legacy class type mappings
* Improve query performance with better index
* Fix validation
* Remove redundant index from notifications
* Split media cleanup from reject-media domain blocks to its own service
* Slightly improve ClearDomainMediaService error handling
* Lower DomainClearMediaWorker to lowest-priority queue
* Do not catch ActiveRecord::RecordNotFound in domain block workers
* Fix DomainBlockWorker spec labels
* Add some specs
* Change domain blocks to immediately mark accounts as suspended
Rather than doing so sequentially, account after account, while cleaning
their data. This doesn't change much about the time the block takes to
complete, but it immediately prevents interaction with the blocked domain,
while up to now, it would only be guaranteed when the process ends.
Also:
- Fix locks not being removed when jobs go to the dead job queue
- Add UI for managing locks to the Sidekiq dashboard
- Remove unused Sidekiq workers
Fix#13349
Mastodon enforces the “sensitive” flag on media attachments whenever a toot
is posted with a Content Warning. However, it does so *after* potentially
converting the Content Warning to toot text (when there is no toot text),
which leads to inconsistent and surprising behavior for API clients.
This commit fixes this inconsistency.
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.
Fix#7812, fix#12246
* Remove “protocol” argument and return value, as only ActivityPub is supported
* Remove FetchRemoteAccountService, only use ActivityPub::FetchRemoteAccountService
* Fix tests
Claire