Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream-updated dependency (redis) textually too
close to glitch-soc-only dependecy.
Updated redis gem like upstream did.
* Add account statuses cleanup policy model
* Record last inspected toot to delete to speed up successive calls to statuses_to_delete
* Add service to cleanup a given account's statuses within a budget
* Add worker to go through account policies and delete old toots
* Fix last inspected status id logic
All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.
* Add tests
* Refactor scheduler and add tests
* Add user interface
* Add support for discriminating based on boosts/favs
* Add UI support for min_reblogs and min_favs, rework UI
* Address first round of review comments
* Replace Snowflake#id_at_start with with_random parameter
* Add tests
* Add tests for StatusesCleanupController
* Rework settings page
* Adjust load-avoiding mechanisms
* Please CodeClimate
* Add account_notes relationship
* Add tests
* Fix owned account notes not being deleted when an account is deleted
* Add post-migration to clean up orphaned account notes
Conflicts:
- `app/serializers/rest/instance_serializer.rb`:
Upstream changed the fields returned by /api/v1/instance by adding a
`configuration` field holding a lot of useful information making our
`max_toot_chars` and `poll_limits` fields obsolete.
Keeping those around for now for compatibility.
- `app/validators/status_length_validator.rb`:
No real conflict, just URL_PLACEHOLDER_CHARS introduced too close to
MAX_CHARS which is defined differently in glitch-soc.
Ported upstream changes.
Conflicts:
- `app/helpers/accounts_helper.rb`:
Conflict due to upstream changing how followers count is displayed while we
have an option to hide followers count.
Ported upstream change.
- `app/views/accounts/_header.html.haml`:
Conflict due to upstream changing how followers count is displayed while we
have an option to hide followers count.
Ported upstream change.
- `app/views/directories/index.html.haml`:
Conflict due to upstream changing how followers count is displayed while we
have an option to hide followers count.
Ported upstream change.
3.4.1 dropped account_stats.lock_version, but in a way breaking the usual
upgrade path by requiring services to be reloaded after the post-migrations.
Indeed, `self.locking_column = nil` was not enough for Rails to ignore the
`lock_version` column when preparing statements on application load, resulting
in some ActiveRecord queries (typically those involving
`includes(:account_stat)`) erroring out with:
ActiveRecord::StatementInvalid (PG::UndefinedColumn: ERROR: column account_stats.lock_version does not exist
* Fix account deletion sometimes failing because of optimistic locks
In some rare occasions[1], deleting accounts would fail with a
`StaleObjectError` exception.
Indeed, account deletion manually sets the `AccountStat` values without
handling cases where the optimistic locking on `AccountStat` would fail.
To my knowledge, with the rewrite of account counters in #15913, the
`DeleteAccountService` is now the only place that changes the counters in
a way that is not atomic.
Since in this specific case, we do not care about the previous values of the
account counters, it appears we don't need locking at all for this table
anymore.
[1]: https://discourse.joinmastodon.org/t/account-cant-be-deleted/3602
* Bump MAX_SUPPORTED_VERSION in maintenance script
Since #16173, `account_summaries` and `follow_recommendations` are not
populated at creation time, which causes concurrent refresh to fail.
As we currently only use those materialized views right after explicitly
refreshing them, this commit changes refreshes to not be performed
concurrently. This will fix the issue and ensure the refresh completes
faster while using less resources.
- `app/views/statuses/_simple_status.html.haml`:
Small markup change in glitch-soc, on a line that has been modified by
upstream. Ported upstream changes.
If a status with a hashtag becomes very popular, it stands to
reason that the hashtag should have a chance at trending
Fix no stats being recorded for hashtags that are not allowed
to trend, and stop ignoring bots
Remove references to hashtags in profile directory from the code
and the admin UI
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0
Also improve the Terrapin monkey-patch for the stderr/stdout issue.
* Fix keyword argument handling throughout the codebase
* Monkey-patch Paperclip to fix keyword arguments handling in validators
* Change validation_extensions to please CodeClimate
* Bump microformats from 4.2.1 to 4.3.1
* Allow Ruby 3.0
* Add Ruby 3.0 test target to CircleCI
* Add test for admin dashboard warnings
* Fix admin dashboard warnings on Ruby 3.0
Express follow_recommendations in terms of account_summaries rather than
accounts, integrate filters that are unconditionally used, and materialize
the resulting view.
This should result in the bulk of the computation being performed only once
instead of **once per recommendation language**.
* Fix media processing getting stuck on too much stdin/stderr
See thoughtbot/terrapin#5
* Remove dependency on paperclip-av-transcoder gem
* Remove dependency on streamio-ffmpeg gem
* Disable stdin on ffmpeg process
Conflicts:
- `README.md`:
Upstream updated copyright year, we don't mention it so kept our version.
- `app/controllers/admin/dashboard_controller.rb`:
Not really a conflict, upstream change (removing the spam checker) too close
to glitch-soc changes. Ported upstream changes.
- `app/models/form/admin_settings.rb`:
Same.
- `app/services/remove_status_service.rb`:
Same.
- `app/views/admin/settings/edit.html.haml`:
Same.
- `config/settings.yml`:
Same.
- `config/environments/production.rb`:
Not a real conflict, upstream added a default HTTP header, but we have
extra headers in glitch-soc.
Added the header.
Prevent new accounts from being created using the same underlying
e-mail as a suspended account using extensions and period
permutations. Stores e-mails as a SHA256 hash
* Update devise-two-factor to unreleased fork for Rails 6 support
Update tests to match new `rotp` version.
* Update nsa gem to unreleased fork for Rails 6 support
* Update rails to 6.1.3 and rails-i18n to 6.0
* Update to unreleased fork of pluck_each for Ruby 6 support
* Run "rails app:update"
* Add missing ActiveStorage config file
* Use config.ssl_options instead of removed ApplicationController#force_ssl
Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.
* Fix nonce directives by removing Rails 5 specific monkey-patching
* Fix fixture_file_upload deprecation warning
* Fix yield-based test failing with Rails 6
* Use Rails 6's index_with when possible
* Use ActiveRecord::Cache::Store#delete_multi from Rails 6
This will yield better performances when deleting an account
* Disable Rails 6.1's automatic preload link headers
Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.
In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.
* Switch to Rails 6.0 default config
* Switch to Rails 6.1 default config
* Do not include autoload paths in the load path
* Improve account counters handling
* Use ActiveRecord::Base::sanitize_sql to pass values instead of interpolating them
Keep using string interpolation for `key` as it is safe and using
“ActiveRecord::Base::sanitize_sql_hash_for_assignment” would require stitching
bits of SQL in a way that is not more easily checked for safety.
* Add migration hook to catch PostgreSQL versions earlier than 9.5
* Fix misuse of foreign_type
* Fix use of removed "add_template_helper"
* Use response.media_type instead of response.content_type in tests
* Fix CSV export controller test on Rails 6
Rails 6 sets a "filename*" field in the Content-Disposition header to
explicitly encode the filename as UTF-8.
This changes checks the first part of the Content-Disposition header so
it matches in both Rails 5 and Rails 6.
* Fix emoji formatting with Rails 6
* Make emoji output more idiomatic and robust
* Switch from redis-rails gem to built-in Rails redis cache storage
Conflicts:
- `app/javascript/styles/mastodon/modal.scss`:
For some reason we changed the file loading path in glitch-soc,
but now upstream has completely changed how the logo is loaded.
Applied upstream changes.
An uncommon but somewhat difficult to digagnose issue is dealing with
improperly-seeded databases. In such cases, instance-signed fetches will
fail with a ActiveRecord::RecordNotFound error, usually caught and handled
as generic 404, leading people to think the remote resource itself has not
been found, while it's the local instance actor that does not exist.
This commit changes the code so that failure to find the instance actor
automatically creates a new one, so that improperly-seeded databases do
not cause any issue.
Conflicts:
- `app/models/public_feed.rb`:
Upstream refactored a bit, glitch-soc had specific code for local-only
statuses.
Updated glitch-soc's specific code accordingly.
* disable NewCops
* update TargetRubyVersion
* Fix Lint/MissingSuper for ActiveModelSerializers::Model
* Fix Lint/MissingSuper for feed
* Fix Lint/FloatComparison
* Do not use instance variables
Conflicts:
- `config/webpack/configuration.js`:
Upstream updated the `js-yaml` dependency, which changed how to call it.
Those changes conflicted because that code is pretty different in glitch-soc
which has to deal with its more complex theming system.
Proceeded to the same compatibility changes in glitch-soc's code.
- `package.json` and `yarn.lock`:
Not really a conflict, just glitch-soc-specific dependencies textually too
close to some dependencies updated upstream.
* Fix being able to import more than allowed number of follows
Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.
This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).
* Catch validation errors to not re-queue failed follows
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/lib/feed_manager.rb`:
Not a real conflict, glitch-soc-only DM-related method
too close to changed upstream stuff.
Ported upstream changes.
- `app/services/batched_remove_status_service.rb`:
Additional logic in glitch-soc to clear DMs from timelines.
Ported upstream changes and fixed the DM TL clearing logic.
- `app/workers/scheduler/feed_cleanup_scheduler.rb`:
Additional code in glitch-soc to clear DM timelines.
Ported upstream changes.
* Delete status records by batches of 50
* Do not precompute values that are only used once
* Do not generate redis events for removal of public toots older than two weeks
* Filter reported toots a priori for polls and status deletion
* Do not process reblogs when cleaning up public timelines
As in Mastodon proper, reblogs don't appear in public TLs
* Clean the deleted account's own feed in one go
* Refactor Account#clean_feed_manager and List#clean_feed_manager
* Delete instead of destroy a few more associations
* Fix preloading
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix external user creation failing when invite request text is required
Also fixes tootctl-based user creation.
* Add test about invites when invite request text is otherwise required
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix deleting polls not deleting notifications
* Fix fav notification deletion when deleting a toot
* Refactor DeleteAccountService spec
* Add DeleteAccountService tests for other associations and notifications
* Add favourite handling spec in status removal
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix account deletion workers being queued multiple times for a single account
* Fix poll votes being unnecessarily instantiated on poll deletion
* Fix favourites being unnecessarily instantiated on status deletion
* Remove inaccurate comments
* Delete polls instead of destroying them
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix admin being able to suspend their own instance account
* Add text about the instance's own actor in admin view
* Change instance actor notice from flash message to template
* Do not list local instance actor in account moderation list
Conflicts:
- `app/models/form/admin_settings.rb`:
New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
New setting added upstream. Ported it.
* Add indication to admin UI of whether a report has been forwarded
* Rework how forwarded status is displayed
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Add honeypot fields to limit non-specialized spam
Add two honeypot fields: a fake website input and a fake password confirmation
one. The label/placeholder/aria-label tells not to fill them, and they are
hidden in CSS, so legitimate users should not fall into these.
This should cut down on some non-Mastodon-specific spambots.
* Require a 3 seconds delay before submitting the registration form
* Fix tests
* Move registration form time check to model validation
* Give people a chance to clear the honeypot fields
* Refactor honeypot translation strings
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/services/remove_status_service.rb`:
Conflict caused by us having a distinc Direct timeline.
Ported upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
Conflict between glitch-soc's variable character limit and upstream
refactoring that part of the code.
Ported upstream changes.
Conflicts:
- `.github/ISSUE_TEMPLATE/bug_report.md`:
Upstream added the `bug` label to bug reports.
Did the same.
- `app/services/fan_out_on_write_service.rb`:
Upstream put DMs back into timelines, glitch-soc was already doing it.
Ignored upstream changes.
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
Conflict caused because of glitch-soc's theming system.
Took upstream's new code and applied the theming system changes on top
of it.
- `app/controllers/concerns/two_factor_authentication_concern.rb`:
Conflict caused because of glitch-soc's theming system.
Took upstream's new code and applied the theming system changes on top
of it.
If someone tries logging in to an account and is prompted for a 2FA
code or sign-in token, even if the account's password or e-mail is
updated in the meantime, the session will show the prompt and allow
the login process to complete with a valid 2FA code or sign-in token
Conflicts:
- `app/controllers/relationships_controller.rb`:
Upstream changed a line too close to a glitch-soc only line related to
glitch-soc's theming system.
Applied upstream changes accordingly.
Do not downcase the queried tag before passing it to postgres when searching:
- tags are not downcased on creation
- `arel_table[:name].lower.matches(pattern)` generates an ILIKE anyway
- if Postgres and Rails happen to use different case-folding rules,
downcasing before query but not before insertion may mean that some
tags with some casings are not searchable
Conflicts:
- `app/controllers/follower_accounts_controller.rb`:
Conflict due to upstream changing suspension logic while
glitch-soc has an extra option to hide followers count.
Ported upstream changes.
Conflicts:
- `.github/dependabot.yml`:
Updated upstream, we deleted it to not be flooded by Depandabot.
Kept deleted.
- `Gemfile.lock`:
Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
Upstream updated the serializer to support timed mutes, while
glitch-soc added a custom API ages ago to get information that
is already available elsewhere.
Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
🤷
- `app/models/user.rb`:
New user setting added upstream while we have glitch-soc-specific
user settings. Added upstream's user setting.
- `config/settings.yml`:
Upstream added a new user setting close to a user setting we had
changed the defaults for. Added the new upstream setting.
- `package.json`:
Upstream dependency updated “too close” to a glitch-soc-specific
dependency. No real conflict. Updated the dependency.
* Add support for followers synchronization on the receiving end
Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.
* Add tests for followers synchronization on the receiving end
* Add support for follower synchronization on the sender's end
* Add tests for the sending end
* Switch from AS attributes to HTTP header
Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
receiving instance (where “receiving instance” is determined by accounts
sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor
Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.
* Add environment variable to disable followers synchronization
Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.
The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
netloc as a managed account must be managed by the same Mastodon instance
as well.
As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.
Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.
* Only set follower synchronization header on non-public statuses
This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)
* Change how ActivityPub::SynchronizeFollowersService handles follow requests
If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.
* Integrate review feeback
- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes
* Only select required fields when computing followers_hash
* Use actor URI rather than webfinger domain in synchronization endpoint
* Change hash computation to be a XOR of individual hashes
Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.
* Marginally improve followers_hash computation speed
* Further improve hash computation performances by using pluck_each
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.
Fix#14091
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
Upstream changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/models/user.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc not preventing moved accounts from logging
in.
Ported upstream changes while keeping the ability for moved accounts to log
in.
- `app/policies/status_policy.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's ability to hide followers count.
Ported upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `package.json`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
* Add bell button
Fix#4890
* Remove duplicate type from post-deployment migration
* Fix legacy class type mappings
* Improve query performance with better index
* Fix validation
* Remove redundant index from notifications
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb
All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
Conflicts:
- `config/webpack/shared.js`:
Upstream has changed how Tesseract.js gets included and dropped a dependency.
The conflict is caused by glitch-soc having different code due to its
theming system.
Ported upstream changes.
- `lib/mastodon/version.rb`:
Upstream refactor/code style change in a place we replaced upstream's
repo URL with ours.
Ported upstram changes, keeping our repo URL.
- `yarn.lock`:
Upstream dropped dependencies, one of which was textually too close to
a glitch-soc-specific dependency. Not a real conflict.
* Add database support for list show-reply preferences
* Add backend support to read and update list-specific show_replies settings
* Add basic UI to set list replies setting
* Add specs for list replies policy
* Switch "cycling" reply policy link to a set of radio inputs
* Capitalize replies_policy strings
* Change radio button design to be consistent with that of the directory explorer
* Make Array-creation behavior of Paginable more predictable
Paginable.paginate_by_id usually returns ActiveRecord::Relation, but it
returns an Array if min_id option is present. The behavior caused problems
fixed with the following commits:
- 976938bf30
- 5a8fa3ad95
- 1c70d8189c
To prevent from recurring similar problems, this commit introduces two
changes:
- The scope now always returns an Array whether min_id option is present
or not.
- The scope is renamed to to_a_paginated_by_id to clarify it returns an
Array.
* Transform Paginable.to_a_paginated_by_id from a scope to a class method
https://api.rubyonrails.org/classes/ActiveRecord/Scoping/Named/ClassMethods.html#method-i-scope
> The method is intended to return an ActiveRecord::Relation object, which
> is composable with other scopes.
Paginable.to_a_paginated_by_id returns an Array and is not appropriate
as a scope.
Conflicts:
- `app/controllers/accounts_controller.rb`:
Upstream change too close to a glitch-soc change related to
instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
Minor conflict due to glitch-soc's handling of Direct Messages,
merged upstream changes.
- `yarn.lock`:
Not really a conflict, caused by glitch-soc-only dependencies
being textually too close to updated upstream dependencies.
Merged upstream changes.
* feat: add possibility of adding WebAuthn security keys to use as 2FA
This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: add request for WebAuthn as second factor at login if enabled
This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: add possibility of deleting WebAuthn Credentials
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: disable WebAuthn when an Admin disables 2FA for a user
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA
Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.
* We had to change a little the flow for setting up TOTP, given that now
it's possible to setting up again if you already had TOTP, in order to
let users modify their authenticator app – given that now it's not
possible for them to disable TOTP and set it up again with another
authenticator app.
So, basically, now instead of storing the new `otp_secret` in the
user, we store it in the session until the process of set up is
finished.
This was because, as it was before, when users clicked on 'Edit' in
the new two-factor methods lists page, but then went back without
finishing the flow, their `otp_secret` had been changed therefore
invalidating their previous authenticator app, making them unable to
log in again using TOTP.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* refactor: fix eslint errors
The PR build was failing given that linting returning some errors.
This commit attempts to fix them.
* refactor: normalize i18n translations
The build was failing given that i18n translations files were not
normalized.
This commits fixes that.
* refactor: avoid having the webauthn gem locked to a specific version
* refactor: use symbols for routes without '/'
* refactor: avoid sending webauthn disabled email when 2FA is disabled
When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.
* refactor: avoid creating new env variable for webauthn_origin config
* refactor: improve flash error messages for webauthn pages
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
Not a “real” conflict, but change too close to a change we made to
fix the vanilla WebUI locally pushing authored local-only toots in the
public TL view.
Since #14145, the `set_type_and_extension` has been moved from
`before_post_process` to `before_file_post_process`, but while the former
runs before all validations performed by Paperclip, the latter is dependent
on the order validations and hooks are defined.
In our case, this meant video files could be checked against the generic 10MB
limit, causing validation failures, which, internally, make Paperclip skip
post-processing, and thus, transcoding of the video file.
The actual validation would then happen after the type is correctly set, so
the large file would pass validation, but without being transcoded first.
This commit moves the hook definition so that it is run before checking for
the file size.
Conflicts:
- `.env.production.sample`:
Upstream changed it completely.
Changed ours to merge upstream's new structure, but
keeping most of the information.
Conflicts:
- `app/javascript/packs/public.js`:
Conflict because part of that file has been split to
`app/javascript/core/settings.js`. Ported those changes
there.
Conflicts:
- `.env.production.sample`:
Upstream deleted it, I decided to keep it.
- `package.json`:
Upstream updated a dependency textually too close to wavesurfer.js
which has been deleted from upstream but is kept in glitch-soc for now.
* Add UserNote model
* Add UI for user notes
* Put comment in relationships entity
* Add API to create user notes
* Copy user notes to new account when receiving a Move activity
* Address some of the review remarks
* Replace modal by inline edition
* Please CodeClimate
* Button design changes
* Change design again
* Cancel note edition when pressing Escape
* Fixes
* Tweak design again
* Move “Add note” item, and allow users to add notes to themselves
* Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
- Fix audio attachments not being represented in OpenGraph tags
- Fix audio being represented as "1 image" in OpenGraph descriptions
- Fix video metadata being overwritten by paperclip-av-transcoder
- Fix embedded player not using Mastodon's UI
- Fix audio/video progress bars not moving smoothly
- Fix audio/video buffered bars not displaying correctly
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
Conflict due to glitch-soc having to take care of local-only
pinned toots in that controller.
Took upstream's changes and restored the local-only special
handling.
- `app/controllers/auth/sessions_controller.rb`:
Minor conflicts due to the theming system, applied upstream
changes, adapted the following two files for glitch-soc's
theming system:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`
- `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
Minor conflict due to glitch-soc having to handle local-only
toots specially. Applied upstream changes and restored
the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
Minor conflict due to the theming system.
- `package.json`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
- `yarn.lock`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
* Fix admin page crashing when trying to block an invalid domain name
Fixes#13880
* Fix trailing and leading spaces not being properly stripped for domain blocks
Conflicts:
- `Gemfile.lock`:
Not a real conflict, just a glitch-soc-only dependency too close to a
dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
Not a real conflict, just a change too close to glitch-soc-changed code
for optionally showing boosts in public timelines.
Applied upstream changes.
- `app/views/layouts/application.html.haml`:
Upstream a new, static CSS file, conflict due to glitch-soc's theming
system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
both files are very different. Removed 'unsafe-inline' as well.
* Improve RSS entries for statuses
- Render polls in both accounts and tags serializers
- Refactor RSS serializers
- Change title preview to include ellipsis when truncated
- Change title preview to show CW instead of toot text
- Add tests
* Remove title from OEmbed serialization
Twitter doesn't serialize title either, and tihs allows us to move the
title formatting code to the RSS serializers.
Conflicts:
- `app/controllers/statuses_controller.rb`:
Upstream disabled the embed controller for reblogs.
Not a real conflict, but glitch-soc has an extra line to deal
with its theming system.
Ported upstream changes.
- `app/javascript/packs/public.js`:
Upstream made changes to get rid of most inline CSS, this changes
javascript for public pages, which in glitch are split between
different files. Ported those changes.
- `app/models/status.rb`:
Upstream changed the block check in `Status#permitted_for` to
include domain-block checks. Not a real conflict with glitch-soc,
but our scope is slightly different, as our scope for
unauthenticated access do not include instance-local toots.
Ported upstream changes.
- `app/serializers/rest/instance_serializer.rb`:
Not a real conflict, upstream added a new field to the instance
serializer, the conflict is one line above since we added more of
that.
Ported upstream changes.
- `app/views/settings/profiles/show.html.haml`:
Upstream got rid of most inline CSS and moved hidden elements
to data attributes in the process, in fields were we have
different values.
Ported upstream changes while keeping our glitch-specific
values.
- `app/views/statuses/_simple_status.html.haml`:
Upstream got rid of inline CSS on an HAML line we treat
differently, stripping empty text nodes.
Ported upstream changes to the style attribute, keeping
the empty text node stripping behavior.
When deduplicating account usernames for OAuthable users, the routine did check if any account was known with that username, including remote accounts. This caused some unnecessary deduplication, and usernames ending with unexpected trailing _1.
This fixes#13580
Follow-up to #13452, fixing broken `uri.nil?` test.
Also remove the separate check for `uri` presence, as that would result
in a “Please review 2 errors below” while only one would be listed.
Conflicts:
- `package.json`:
No real conflict, upstream updated a dependency which is
textually adjacent to a glitch-soc-specific dependency.
Updated it.
- `yarn.lock`:
No real conflict, upstream updated a dependency which is
textually adjacent to a glitch-soc-specific dependency.
Updated it.
* Fix “Email changed” notification sometimes having wrong e-mail
Fixes#6778
The root of the issue is that `send_devise_notification` was called before
the changes were properly commited to the database, causing the mailer to
pick previous values if running too early.
Devise's documentation provides guidance on how to handle that[1][2], however,
I have found it to not be working, as the following happens, in that order:
- `send_devise_notification` is called for the `email_changed` notification.
In that case, `changed?` is false and `saved_changes?` is true, so
if we use the former, we have the same issue.
- the `after_commit` hook is called
- `send_devise_notification` is called for the `confirmation_instructions`
notification.
In that case, `changed?` is still false, and `saved_changes?` still true,
so if we use the latter, that second notification email is simply not
going to be sent (as we would be queuing the notification *after*
executing the after_commit hook).
This is because it may be called from either an `after_update` or
`after_commit` hook, the difference not being a call to `save` but the
transaction actually being committed to the database. This may arguably
be a bug in Devise, or Devise's notification.
The proposed workaround is inspired by Devise's documentation but checks
whether a transaction is open to make the call whether to immediately
send the notification or defer it to the `after_commit` hook.
[1]: https://www.rubydoc.info/github/plataformatec/devise/Devise%2FModels%2FAuthenticatable:send_devise_notification
[2]: 406915cb78/lib/devise/models/authenticatable.rb (L133-L194)
* Fix cases when sending notifications without changing the model
* Defer sending if and only if in transaction including current record
Conflicts:
- `app/javascript/packs/public.js`:
Upstream modified code that we split between multiple files due
to glitch-soc's theming system.
Ported those changes.
* Fix PostgreSQL load when linking in announcements
Fixes#13245 by caching status lookups
Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.
Since it's only used by announcements so far, there won't
be much statuses to cache.
* Perform status lookup when saving announcements, not when rendering them
* Change EntityCache#status to fetch URLs instead of looking into the database
* Move announcement link lookup to publishing worker
* Address issues pointed out during review
Conflicts:
- `README.md`:
Our README.md files are completely different. Discarded upstream changes.
- `app/javascript/core/admin.js`:
Updating rails-ujs, no real conflict, but a comment to close to changed
code. Various glitch-soc-only files have been updated to match those changes,
though.
- `package.json`:
No real conflict, just an additional dependency in glitch-soc that was too
close to something updated upstream. Took upstream's changes.
* Change ActivityPub follower/following collections to not link first page
* Add support for hiding followers and following of remote users
* Switch to using a single `hide_collections` column
* Address code style remarks
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
Conflict due to upstream adding a new parameter (with_rate_limit),
too close to glitch-soc's own additional parameter (content_type).
Added upstream's parameter.
- `app/services/post_status_service.rb`:
Conflict due to upstream adding a new parameter (rate_limit),
too close to glitch-soc's own additional parameter (content_type).
Added upstream's parameter.
- `app/views/settings/preferences/appearance/show.html.haml`:
Conflict due to us not exposing theme settings here (as we have
a different flavour/skin menu).
Took upstream change, while still not exposing theme settings.
- `config/webpack/shared.js`:
Coding style fixes for a part we have rewritten.
Discarded upstream changes.
Conflicts:
- `app/serializers/rest/account_serializer.rb`:
Upstream added code too close to glitch-soc-specific followers-hiding code.
Ported upstream changes.
Change `all_day` to be a visual client-side cue only
Publish immediately if `scheduled_at` is in the past
Add `published_at` and `updated_at` to announcements JSON
While making browser requests in the other sessions after a password
change or reset does not allow you to be logged in and correctly
invalidates the session making the request, sessions have API tokens
associated with them, which can still be used until that session
is invalidated.
This is a security issue for accounts that were already compromised
some other way because it makes it harder to throw out the hijacker.
* Add announcements
Fix#11006
* Add reactions to announcements
* Add admin UI for announcements
* Add unit tests
* Fix issues
- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"
* Fix scheduler unpublishing announcements before they are due
* Fix filter params not being passed to announcements filter
Allow browsing and filtering all relationships instead of just
followers, unify the codebase with the user-facing relationship
manager, add ability to see who the user invited
Conflicts:
- `app/javascript/packs/public.js`:
Upstream removed an unused function in code that has
been refactored a bit. Removed that function in the corresponding
places.
* Fix unused role routes being generated
* Remove unused JavaScript code
* Refactor filters code to be DRYer
* Fix `.count == 0` comparisons to `.empty?` in views
* Fix filters in views
Conflicts:
- `Gemfile.lock`:
No real conflict, glitch-soc-only dependency (redcarpet) too close to an
upstream one (rdf-normalize)
- `README.md`:
we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
Upstream added code to keep bookmarked statuses, we were already doing so
with slightly different code. Discarded upstream's changes.
- `package.json`:
No real conflict, glitch-soc-only dependency (favico.js) too close to
an upstream one
Fixes#12795
It was already possible to create domain blocks for TLDs, but those
weren't enforced, nor editable. This commit changes it so that they
are enforced and editable.
Conflicts:
- `app/controllers/application_controller.rb`:
Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
Conflict due to theming system.
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.
Fix#7812, fix#12246
Conflicts:
- `config/locales/en.yml`
No real conflict, upstream added a translatable string “too close” to
one specific to glitch-soc
- `lib/mastodon/statuses_cli.rb`
Fixes made upstream, while changed in glitch-soc to keep bookmarked statuses
- `package.json`
No real conflict, additional dependency in glitch-soc
* Revert "Fix ignoring whole status because of one invalid hashtag (#11621)"
This reverts commit 123ffd0a45.
* Fix statuses being rejected because of invalid hashtag names
* Add spec for invalid hashtag names in statuses
* Add test for featured tags controller
Conflicts:
- package.json
Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
Not really a conflict, caused by an additional dependency in glitch-soc.
* Show badge on group actor in WebUI
* Do not notify in case of by following group actor
* If you mention group actor, also mention group actor followers
* Relax characters that can be used in username (same as Application)
* Revert "Relax characters that can be used in username (same as Application)"
This reverts commit 7e10a137b878d0db1b5252c52106faef5e09ca4b.
* Delete display_name method
* Add follow_request notification type
The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.
Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.
Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.
* Show follow request link in single-column if there are pending requests, even if account isn't locked
* Push follow requests from notifications to the follow_requests list
* Offer to accept or reject follow request from the notification
* Redesign follow request notification
Before this patch, if remote poll options have leading or trailing spaces,
the information stored locally won't match them, causing federated voting to
fail.
This might be somewhat controversial, but putting a redirection up
makes the account unusable, which works against use cases such as
using the moved account as backup in case the new one temporarily fails
(sure, the redirection can be temporarily removed, but it is a hassle)
* Add backend support for bookmarks
Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.
* Add spec for bookmark endpoints
* Add front-end support for bookmarks
* Introduce OAuth scopes for bookmarks
* Add bookmarks to archive takeout
* Fix migration
* Coding style fixes
* Fix rebase issue
* Update bookmarked_statuses to latest UI changes
* Update bookmark actions to properly reflect status changes in state
* Add bookmarks item to single-column layout
* Make active bookmarks red
* Add ability to add oneself to lists
* Change search results to include oneself when searching through followers
* Mark follow relation as optional in ListAccount
Conflicts:
- README.md
- app/helpers/statuses_helper.rb
Upstream moved account helpers to their own file, we had extra
helpers there, moved too.
- app/lib/sanitize_config.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- config/locales/simple_form.en.yml
- spec/lib/sanitize_config_spec.rb
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive
Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
Conflicts:
- app/models/media_attachment.rb
Upstream raised max image size from 8MB to 10MB while our limit is
configurable. Raised the default to 10MB.
* Change animated GIF detection to not shell out to ImageMagick
Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>
* Change video encoding parameters to limit to 10800 video frames
Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>
* Limit GIF image size further
Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>
* Always strip metadata from video files
* Fix code style issues
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
* Add voters count to polls
* Add ActivityPub serialization and parsing of voters count
* Add support for voters count in WebUI
* Move incrementation of voters count out of redis lock
* Reword “voters” to “people”
* Addition of update activity distribution by alias, minor correction
* Distribute Update activity after adding alias
* Add uniqueness verification to alias uri
* accept acct starting with @
* fix double-quoted to single-quoted
* Add search and sort functions to hashtag admin UI
* Move scope processing from tags_controller to tag_filter
* Fix based on method naming conventions
* Fixed not to get 500 errors for invalid requests
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
Conflict because we changed the number of default results to be
configurable
- app/lib/settings/scoped_settings.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
Conflicts:
- app/views/directories/index.html.haml
Upstream has redesigned the profile directory, and we
had a glitch-soc-specific change to hide follower counts.
Ported that change to the new design.
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb
All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
* Add soft delete for statuses to allow them to appear instant
* Allow reporting soft-deleted statuses and show them in the admin UI
* Change index for getting an account's statuses
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
* Add automatic blocklist display in /about/blocks
Inspired by https://github.com/Gargron/mastodon.social-misc
* Add admin option to set who can see instance blocks
* Normalize locales files
* Rename “Sandbox” to “Silence” for consistency
* Disable /about/blocks when in whitelist mode
* Optionally display rationale for domain blocks
* Only display domain blocks that have user-facing limitations, and order them
* Redesign table of blocked domains to better handle long domain names and rationales
* Change domain blocks ordering now that rationales aren't displayed right away
* Only show explanation for block severities actually in use
* Reword instance block explanations and add disclaimer for public fetch mode
* Add more accurate hashtag search
Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly
* Fix search analyzer dropping non-ascii characters
* Add more accurate account search
When ElasticSearch is available, a more accurate search is implemented:
- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)
The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.
* Support single-letter usernames
* Fix tests
* Fix not picking up account updates
* Add weights and normalization for scores, skip zero terms queries
* Use local counts for accounts index, adjust search parameters
* Fix mistakes
* Using updated_at of accounts is inadequate for remote accounts
Fix percentages in usage breakdown having too many digits
Change trending hashtags to only ask for review if a hashtag enters
the top 3 position in the set, since it's the only items shown in
the default web UI
* Add database columns for adding notes to domain blocks/restrctions
* Add admin UI to set private and public comments when blocking a domain
* Add text for private and public comments on domain blocks
* Show domain block comments in admin UI
* Add comments to the domain block undo page
* Make UnblockDomainService more robust regarding upgraded domain blocks
* Allow editing domain blocks
* Rename button from “undo domain block” to “view domain block” in account admin UI
* Change test to unsilence silenced users from upgraded blocks
* Fix account tags not being saved correctly
Regression from b838607
Fix Tag#discoverable not returning tags where listable is nil instead of true
Add notice when saving hashtags in admin UI
Change public hashtag and directory pages to return 404 for forbidden tags
* Remove unused locale string
Claire