* Fix being able to import more than allowed number of follows
Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.
This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).
* Catch validation errors to not re-queue failed follows
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/lib/feed_manager.rb`:
Not a real conflict, glitch-soc-only DM-related method
too close to changed upstream stuff.
Ported upstream changes.
- `app/services/batched_remove_status_service.rb`:
Additional logic in glitch-soc to clear DMs from timelines.
Ported upstream changes and fixed the DM TL clearing logic.
- `app/workers/scheduler/feed_cleanup_scheduler.rb`:
Additional code in glitch-soc to clear DM timelines.
Ported upstream changes.
* Delete status records by batches of 50
* Do not precompute values that are only used once
* Do not generate redis events for removal of public toots older than two weeks
* Filter reported toots a priori for polls and status deletion
* Do not process reblogs when cleaning up public timelines
As in Mastodon proper, reblogs don't appear in public TLs
* Clean the deleted account's own feed in one go
* Refactor Account#clean_feed_manager and List#clean_feed_manager
* Delete instead of destroy a few more associations
* Fix preloading
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix account deletion workers being queued multiple times for a single account
* Fix poll votes being unnecessarily instantiated on poll deletion
* Fix favourites being unnecessarily instantiated on status deletion
* Remove inaccurate comments
* Delete polls instead of destroying them
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix ResolveAccountService accepting mismatching acct: URI
* Set attributes that should be updated regardless of suspension
* Fix key fetching
* Automatically merge remote accounts with duplicate `uri`
* Add tests
* Add "tootctl accounts fix-duplicates"
Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/models/form/admin_settings.rb`:
New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
New setting added upstream. Ported it.
Conflicts:
- `.github/ISSUE_TEMPLATE/bug_report.md`:
Upstream added the `bug` label to bug reports.
Did the same.
- `app/services/fan_out_on_write_service.rb`:
Upstream put DMs back into timelines, glitch-soc was already doing it.
Ignored upstream changes.
Nginx can be configured to bypass proxy cache when a special header
is in the request. If the response is cacheable, it will replace
the cache for that request. Proxy caching of media files is
desirable when using object storage as a way of minimizing bandwidth
costs, but has the drawback of leaving deleted media files for
a configured amount of cache time. A cache buster can make those
media files immediately unavailable. This especially makes sense
when suspending and unsuspending an account.
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
Conflict caused because of glitch-soc's theming system.
Took upstream's new code and applied the theming system changes on top
of it.
- `app/controllers/concerns/two_factor_authentication_concern.rb`:
Conflict caused because of glitch-soc's theming system.
Took upstream's new code and applied the theming system changes on top
of it.
Conflicts:
- `app/controllers/follower_accounts_controller.rb`:
Conflict due to upstream changing suspension logic while
glitch-soc has an extra option to hide followers count.
Ported upstream changes.
Conflicts:
- `.github/dependabot.yml`:
Updated upstream, we deleted it to not be flooded by Depandabot.
Kept deleted.
- `Gemfile.lock`:
Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
Upstream updated the serializer to support timed mutes, while
glitch-soc added a custom API ages ago to get information that
is already available elsewhere.
Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
🤷
- `app/models/user.rb`:
New user setting added upstream while we have glitch-soc-specific
user settings. Added upstream's user setting.
- `config/settings.yml`:
Upstream added a new user setting close to a user setting we had
changed the defaults for. Added the new upstream setting.
- `package.json`:
Upstream dependency updated “too close” to a glitch-soc-specific
dependency. No real conflict. Updated the dependency.
* Add support for followers synchronization on the receiving end
Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.
* Add tests for followers synchronization on the receiving end
* Add support for follower synchronization on the sender's end
* Add tests for the sending end
* Switch from AS attributes to HTTP header
Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
receiving instance (where “receiving instance” is determined by accounts
sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor
Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.
* Add environment variable to disable followers synchronization
Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.
The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
netloc as a managed account must be managed by the same Mastodon instance
as well.
As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.
Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.
* Only set follower synchronization header on non-public statuses
This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)
* Change how ActivityPub::SynchronizeFollowersService handles follow requests
If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.
* Integrate review feeback
- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes
* Only select required fields when computing followers_hash
* Use actor URI rather than webfinger domain in synchronization endpoint
* Change hash computation to be a XOR of individual hashes
Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.
* Marginally improve followers_hash computation speed
* Further improve hash computation performances by using pluck_each
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
Upstream changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/models/user.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc not preventing moved accounts from logging
in.
Ported upstream changes while keeping the ability for moved accounts to log
in.
- `app/policies/status_policy.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's ability to hide followers count.
Ported upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `package.json`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
* Add bell button
Fix#4890
* Remove duplicate type from post-deployment migration
* Fix legacy class type mappings
* Improve query performance with better index
* Fix validation
* Remove redundant index from notifications
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb
All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
Not a “real” conflict, but change too close to a change we made to
fix the vanilla WebUI locally pushing authored local-only toots in the
public TL view.
Conflicts:
- `.env.production.sample`:
Upstream changed it completely.
Changed ours to merge upstream's new structure, but
keeping most of the information.
Conflicts:
- `app/javascript/packs/public.js`:
Conflict because part of that file has been split to
`app/javascript/core/settings.js`. Ported those changes
there.
* Change move handler to carry blocks and mutes over
When user A blocks user B and B moves to a new account C, make A block C
accordingly.
Note that it only works if A's instance is aware of the Move, that is,
if B is on A's instance or has followers there.
* Also notify instances with known people blocking you when moving
* Add automatic account notes when blocking/muting an account that had no note
Conflicts:
- `.env.production.sample`:
Upstream deleted it, I decided to keep it.
- `package.json`:
Upstream updated a dependency textually too close to wavesurfer.js
which has been deleted from upstream but is kept in glitch-soc for now.
* Add UserNote model
* Add UI for user notes
* Put comment in relationships entity
* Add API to create user notes
* Copy user notes to new account when receiving a Move activity
* Address some of the review remarks
* Replace modal by inline edition
* Please CodeClimate
* Button design changes
* Change design again
* Cancel note edition when pressing Escape
* Fixes
* Tweak design again
* Move “Add note” item, and allow users to add notes to themselves
* Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
- Fix audio attachments not being represented in OpenGraph tags
- Fix audio being represented as "1 image" in OpenGraph descriptions
- Fix video metadata being overwritten by paperclip-av-transcoder
- Fix embedded player not using Mastodon's UI
- Fix audio/video progress bars not moving smoothly
- Fix audio/video buffered bars not displaying correctly
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
Conflict due to glitch-soc having to take care of local-only
pinned toots in that controller.
Took upstream's changes and restored the local-only special
handling.
- `app/controllers/auth/sessions_controller.rb`:
Minor conflicts due to the theming system, applied upstream
changes, adapted the following two files for glitch-soc's
theming system:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`
- `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
Minor conflict due to glitch-soc having to handle local-only
toots specially. Applied upstream changes and restored
the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
Minor conflict due to the theming system.
- `package.json`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
- `yarn.lock`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
* Split media cleanup from reject-media domain blocks to its own service
* Slightly improve ClearDomainMediaService error handling
* Lower DomainClearMediaWorker to lowest-priority queue
* Do not catch ActiveRecord::RecordNotFound in domain block workers
* Fix DomainBlockWorker spec labels
* Add some specs
* Change domain blocks to immediately mark accounts as suspended
Rather than doing so sequentially, account after account, while cleaning
their data. This doesn't change much about the time the block takes to
complete, but it immediately prevents interaction with the blocked domain,
while up to now, it would only be guaranteed when the process ends.
Conflicts:
- `Gemfile.lock`:
Not a real conflict, just a glitch-soc-only dependency too close to a
dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
Not a real conflict, just a change too close to glitch-soc-changed code
for optionally showing boosts in public timelines.
Applied upstream changes.
- `app/views/layouts/application.html.haml`:
Upstream a new, static CSS file, conflict due to glitch-soc's theming
system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
both files are very different. Removed 'unsafe-inline' as well.
Conflicts:
- `app/javascript/packs/public.js`:
Upstream modified code that we split between multiple files due
to glitch-soc's theming system.
Ported those changes.
* Fix PostgreSQL load when linking in announcements
Fixes#13245 by caching status lookups
Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.
Since it's only used by announcements so far, there won't
be much statuses to cache.
* Perform status lookup when saving announcements, not when rendering them
* Change EntityCache#status to fetch URLs instead of looking into the database
* Move announcement link lookup to publishing worker
* Address issues pointed out during review
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
Upstream bumped poll option character limit, but we already had
a higher one, kept ours.
- `app/validators/poll_validator.rb`:
Upstream bumped poll option character limit, but we already had
a higher one, kept ours.
- `config/initializers/content_security_policy.rb`:
Upstream added a rule, the way we compute ours is different, but
that added rule has been ported.
- `package.json`:
No real conflict, dependency update. Performed the same update.
- `yarn.lock`:
No real conflict, dependency update. Performed the same update.
Also:
- Fix locks not being removed when jobs go to the dead job queue
- Add UI for managing locks to the Sidekiq dashboard
- Remove unused Sidekiq workers
Fix#13349
Conflicts:
- `README.md`:
Our README.md files are completely different. Discarded upstream changes.
- `app/javascript/core/admin.js`:
Updating rails-ujs, no real conflict, but a comment to close to changed
code. Various glitch-soc-only files have been updated to match those changes,
though.
- `package.json`:
No real conflict, just an additional dependency in glitch-soc that was too
close to something updated upstream. Took upstream's changes.
Conflicts:
- `app/serializers/rest/account_serializer.rb`:
Upstream added code too close to glitch-soc-specific followers-hiding code.
Ported upstream changes.
Change `all_day` to be a visual client-side cue only
Publish immediately if `scheduled_at` is in the past
Add `published_at` and `updated_at` to announcements JSON
* Add announcements
Fix#11006
* Add reactions to announcements
* Add admin UI for announcements
* Add unit tests
* Fix issues
- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"
* Fix scheduler unpublishing announcements before they are due
* Fix filter params not being passed to announcements filter
Conflicts:
- `Gemfile.lock`:
No real conflict, glitch-soc-only dependency (redcarpet) too close to an
upstream one (rdf-normalize)
- `README.md`:
we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
Upstream added code to keep bookmarked statuses, we were already doing so
with slightly different code. Discarded upstream's changes.
- `package.json`:
No real conflict, glitch-soc-only dependency (favico.js) too close to
an upstream one
Conflicts:
- `app/controllers/application_controller.rb`:
Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
Conflict due to theming system.
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`
conflict because of the poll option limit being different than upstream's
* Fix being able to follow oneself by moving to an account that was following the old one
* Add specs
* Add spec to catch MoveWorker issue with local followers following both accounts
* Fix move worker breaking when a local account follows both source and target accounts
* Fix migration from remote to local account not sending Undo Follow
* Fix show_reblogs not being preserved for moved account's followers
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
Conflict because we changed the number of default results to be
configurable
- app/lib/settings/scoped_settings.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb
All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
* Add soft delete for statuses to allow them to appear instant
* Allow reporting soft-deleted statuses and show them in the admin UI
* Change index for getting an account's statuses
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
* Add database columns for adding notes to domain blocks/restrctions
* Add admin UI to set private and public comments when blocking a domain
* Add text for private and public comments on domain blocks
* Show domain block comments in admin UI
* Add comments to the domain block undo page
* Make UnblockDomainService more robust regarding upgraded domain blocks
* Allow editing domain blocks
* Rename button from “undo domain block” to “view domain block” in account admin UI
* Change test to unsilence silenced users from upgraded blocks
* Add `tootctl preview_cards remove`
* fix code style
* Remove `Scheduler::PreviewCardsCleanupScheduler` file
* fix code style again
Add exclude case where image_file_name is blank
* Added a function to output confirmation if the specified number of days is less than 2 weeks
- Change the maximum count of retry for web push notification (Default -> 5).
- In case of high load of subscribe server, the retries will be repeated many times.
- Because the retries occupy the default queue, maximum retry count should be reduced.
- Change the maximum count of retry for web push notification (Default -> 5).
- In case of high load of subscribe server, the retries will be repeated many times.
- Because the retries occupy the default queue, maximum retry count should be reduced.
* Remove Salmon and PubSubHubbub endpoints
* Add error when trying to follow OStatus accounts
* Fix new accounts not being created in ResolveAccountService
* Add request pool to improve delivery performance
Fix#7909
* Ensure connection is closed when exception interrupts execution
* Remove Timeout#timeout from socket connection
* Fix infinite retrial loop on HTTP::ConnectionError
* Close sockets on failure, reduce idle time to 90 seconds
* Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server
* Use a shared pool size, 512 by default, to stay below open file limit
* Add some tests
* Add more tests
* Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds
* Use a shared pool that returns preferred connection but re-purposes other ones when needed
* Fix wrong connection being returned on subsequent calls within the same thread
* Reduce mutex calls on flushes from 2 to 1 and add test for reaping
Conflicts:
- app/models/account.rb
- app/views/settings/profiles/show.html.haml
- spec/controllers/api/v1/accounts/credentials_controller_spec.rb
Conflicts were due to an increase in account bio length upstream, which
is already covered in glitch-soc through `MAX_BIO_CHARS`.
HTTP 401 responses returned by Mastodon's inbox controller may
be temporary if, for instance, the requesting user's actor/key json
could not be retrieved in a timely fashion. This changes allow retries
instead of dropping the message entirely.
Also added HTTP 408 as that error is by nature temporary.
* Do not retry processing ActivityPub jobs raising validation errors
Jobs yielding validation errors most probably won't ever be accepted,
so it makes sense not to clutter the queues with retries.
* Lower RecordInvalid error reporting to debug log level
* Remove trailing whitespace
* Refactor imports
* Export show_reblogs when exporting list of followed users
* Add support for importing show_reblogs with following collection
* Fix tests
Conflicts:
- app/controllers/following_accounts_controller.rb
Conflicts were due to glitch-soc's theming system.
- app/javascript/packs/public.js
Some code has been change upstream, but it has been
moved to app/javascript/core/settings.js in glitch-soc.
Applied the changes there.
Conflicts:
- app/services/remove_status_service.rb
Conflict because we still handle the direct timeline.
Took upstream changes, with that one extra function call.
- config/locales/ca.yml
Conflict because theme names were updated.
Decided to *keep* the theme names even if they are useless
to avoid future conflicts.
- config/locales/oc.yml
Decided to *keep* the theme names even if they are useless
to avoid future conflicts.
- config/locales/pl.yml
Decided to *keep* the theme names even if they are useless
to avoid future conflicts.
* Fix poll update handler calling method was that was not available
Fix regression from #10209
* Refactor VoteService
* Refactor ActivityPub::DistributePollUpdateWorker and optimize it
* Fix typo
* Fix typo
* Process incoming poll tallies update
* Send Update on poll vote
* Do not send Updates for a poll more often than once every 3 minutes
* Include voters in people to notify of results update
* Schedule closing poll worker on poll creation
* Add new notification type for ending polls
* Add front-end support for ended poll notifications
* Fix UpdatePollSerializer
* Fix Updates not being triggered by local votes
* Fix tests failure
* Fix web push notifications for closing polls
* Minor cleanup
* Notify voters of both remote and local polls when those close
* Fix delivery of poll updates to mentioned accounts and voters
* Fetch up to 5 replies when discovering a new remote status
This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.
* Add some tests for ActivityPub::FetchRepliesWorker
* Add specs for ActivityPub::FetchRepliesService
* Serialize up to 5 public self-replies for ActivityPub notes
* Add specs for ActivityPub::NoteSerializer
* Move exponential backoff logic to a worker concern
* Fetch first page of paginated collections when fetching thread replies
* Add specs for paginated collections in replies
* Move Note replies serialization to a first CollectionPage
The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.
* Use pluck(:uri) instead of map(&:uri) to improve performances
* Fix fetching replies when they are in a CollectionPage
Reject those from accounts with no local followers, from relays
that are not enabled, which do not address local accounts and are
not replies to accounts that do have local followers
- Reduce time-to-digest from 20 to 7 days
- Fetch mentions starting from +1 day since last login
- Fix case when last login is more recent than last e-mail
- Do not render all mentions, only 40, but show number in subject
- Do not send digest to moved accounts
- Do send digest to silenced accounts
* Do not LDS-sign Follow, Accept, Reject, Undo, Block
* Do not use LDS for Create activities of private toots
* Minor cleanup
* Ignore unsigned activities instead of misattributing them
* Use status.distributable? instead of querying visibility directly
- Some associations were missing from the clean-up
- Some attributes were not reset on suspension
- Skip federation and streaming deletes when purging a dead domain
- Move account association definitions to concern
* Eliminate extra accounts select query from FollowService
* Optimistically update follow state in web UI and hide loading bar
Fix#6205
* Asynchronize NotifyService in FollowService
And fix failing test
* Skip Webfinger resolve routine when called from FollowService if possible
If an account is ActivityPub, then webfinger re-resolving is not necessary
when called from FollowService. Improve options of ResolveAccountService
ThibG