Commit Graph

1056 Commits (e57047f213522b68f9e713270c7d0fcd0f829b92)

Author SHA1 Message Date
abcang 2b9d71cc18 Improved performance of notification preloading (#15640)
* Improved performance of notification preloading

* Remove Cacheable from Notification

* Fix test
2021-01-31 21:24:57 +01:00
abcang 82eb3a575e Fix N+1 query when rendering with StatusSerializer (#15641) 2021-01-31 21:24:17 +01:00
luigi c19fe86d66 Optimize some regex matching (#15528)
* Use Regex#match?

* Replace =~ too

* Avoid to call match? from Nil

* Keep value of Regexp.last_match
2021-01-22 10:09:08 +01:00
ThibG 0306d0d4b0 Fix sign-up restrictions based on IP addresses not being enforced (#15607)
Fixes #15606

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-21 06:18:30 +01:00
ThibG 2582047c36 Use Rails' index_by where it makes sense (#15542)
* Use Rails' index_by where it makes sense

* Fix tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-12 09:27:38 +01:00
luigi 944b059f50 Optimize map { ... }.compact calls (#15513)
* Optimize map { ... }.compact

using Enumerable#filter_map, supported since Ruby 2.7

* Add poyfill for Enumerable#filter_map
2021-01-10 00:32:01 +01:00
abcang 41f9aa5dbb Fix rubocop config and warnings (#15503)
* disable NewCops

* update TargetRubyVersion

* Fix Lint/MissingSuper for ActiveModelSerializers::Model

* Fix Lint/MissingSuper for feed

* Fix Lint/FloatComparison

* Do not use instance variables
2021-01-07 09:40:55 +01:00
ThibG 82d014fd08 Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00
ThibG 8d75aa630f Improve account deletion performances further (#15407)
* Delete status records by batches of 50

* Do not precompute values that are only used once

* Do not generate redis events for removal of public toots older than two weeks

* Filter reported toots a priori for polls and status deletion

* Do not process reblogs when cleaning up public timelines

As in Mastodon proper, reblogs don't appear in public TLs

* Clean the deleted account's own feed in one go

* Refactor Account#clean_feed_manager and List#clean_feed_manager

* Delete instead of destroy a few more associations

* Fix preloading

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-22 23:57:46 +01:00
ThibG f894bb9220 Fix external user creation failing when invite request text is required (#15405)
* Fix external user creation failing when invite request text is required

Also fixes tootctl-based user creation.

* Add test about invites when invite request text is otherwise required

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-22 17:14:32 +01:00
Eugen Rochko 3489758ff2 Fix unnecessary queries when batch-removing statuses, 100x faster (#15387) 2020-12-22 17:13:55 +01:00
ThibG 806f7ea54f Fix some notifications not being deleted on poll/status deletion (#15402)
* Fix deleting polls not deleting notifications

* Fix fav notification deletion when deleting a toot

* Refactor DeleteAccountService spec

* Add DeleteAccountService tests for other associations and notifications

* Add favourite handling spec in status removal

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-21 18:22:17 +01:00
ThibG ad1a3c6435 Fix invitation links not working when invite request text is required (#15385)
Fixes #15383

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-21 00:47:02 +01:00
ThibG bbf0e7107c Fix AccountDeletionWorker crashing and clogging sidekiq queues (#15380)
* Fix account deletion workers being queued multiple times for a single account

* Fix poll votes being unnecessarily instantiated on poll deletion

* Fix favourites being unnecessarily instantiated on status deletion

* Remove inaccurate comments

* Delete polls instead of destroying them

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-20 18:25:00 +01:00
Eugen Rochko e66964d8f0 Fix follow limit preventing re-following of a moved account (#14207) 2020-12-18 09:18:31 +01:00
Eugen Rochko 816df80b86 Add option to obfuscate domain name in public list of domain blocks (#15355)
- Replace the middle of the domain with * characters (except for periods)
- Add SHA-256 digest of the domain name in tooltip
2020-12-18 08:30:41 +01:00
ThibG 02bd1060d8 Fix admins being able to suspend their instance actor (#14567)
* Fix admin being able to suspend their own instance account

* Add text about the instance's own actor in admin view

* Change instance actor notice from flash message to template

* Do not list local instance actor in account moderation list
2020-12-15 17:23:58 +01:00
ThibG 3debd888a6 Add indication to admin UI of whether a report has been forwarded (#13237)
* Add indication to admin UI of whether a report has been forwarded

* Rework how forwarded status is displayed

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-15 04:30:15 +01:00
ThibG 64eaaff345 Add ability to require invite request text (#15326)
Fixes #15273

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-14 10:03:09 +01:00
Eugen Rochko df8874b24e Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
ThibG e1ef5f3b31 Add honeypot fields and minimum fill-out time for sign-up form (#15276)
* Add honeypot fields to limit non-specialized spam

Add two honeypot fields: a fake website input and a fake password confirmation
one. The label/placeholder/aria-label tells not to fill them, and they are
hidden in CSS, so legitimate users should not fall into these.

This should cut down on some non-Mastodon-specific spambots.

* Require a 3 seconds delay before submitting the registration form

* Fix tests

* Move registration form time check to model validation

* Give people a chance to clear the honeypot fields

* Refactor honeypot translation strings

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-10 06:27:26 +01:00
trwnh eb9be7be0c rename replies_policy enumerables (#15304) 2020-12-09 04:34:17 +01:00
Eugen Rochko 48534762cf Fix slow distinct queries where grouped queries are faster (#15287)
About 2x speed-up on inboxes query
2020-12-07 12:08:30 +01:00
ThibG 1fce69291f Fix account merging in maintenance script (#15264)
Also include AccountNote and AccountDeletionRequest to the list of classes
needing to be reassigned the merged account.
2020-12-04 02:23:51 +01:00
ThibG 39824d29f1 Fix followers hash cache not being invalidated on account merge (#15256)
Also clear relationships cache.
2020-12-02 21:20:00 +01:00
Eugen Rochko 876df5bb18 Add `tootctl accounts merge` (#15201)
* Add `tootctl accounts merge`

* Update lib/mastodon/accounts_cli.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-11-23 17:50:16 +01:00
ThibG 8e79bac43d Add import/export feature for bookmarks (#14956)
* Add ability to export bookmarks

* Add support for importing bookmarks

* Add bookmark import tests

* Add bookmarks export test
2020-11-19 17:48:13 +01:00
Eugen Rochko 7e2920d9b2 Fix streaming API allowing connections to persist after access token invalidation (#15111)
Fix #14816
2020-11-12 23:05:24 +01:00
Eugen Rochko 90faa8039c Fix 2FA/sign-in token sessions being valid after password change (#14802)
If someone tries logging in to an account and is prompted for a 2FA
code or sign-in token, even if the account's password or e-mail is
updated in the meantime, the session will show the prompt and allow
the login process to complete with a valid 2FA code or sign-in token
2020-11-12 23:05:01 +01:00
ThibG 472b0e77e7 Fix possible inconsistencies in tag search (#14906)
Do not downcase the queried tag before passing it to postgres when searching:
- tags are not downcased on creation
- `arel_table[:name].lower.matches(pattern)` generates an ILIKE anyway
- if Postgres and Rails happen to use different case-folding rules,
  downcasing before query but not before insertion may mean that some
  tags with some casings are not searchable
2020-11-12 18:35:23 +01:00
Takeshi Umeda 1dde6183a3 Add follow selected followers button (#15148)
* Add follow selected followers button

* Fix unused variable

* Fix i18n normalize
2020-11-12 16:58:00 +01:00
Eugen Rochko f9dfbaf6b2 Fix updating account counters when account_stat is not yet created (#15108) 2020-11-09 16:00:23 +01:00
Eugen Rochko 38be9af436 Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
ThibG b1a78a5cce Change order of announcements in admin page to sort them newest-first (#15091)
* Change order of announcements in admin page to sort show newly-created first

Fixes #15090

* Use reverse-chronological rather than creation date only
2020-11-04 21:15:22 +01:00
Takeshi Umeda 385c2eac6f Add account sensitized (#14361)
* Add account sensitized

* Fix i18n normalize

* Fix description and spec

* Fix spec

* Fix wording
2020-11-04 20:45:01 +01:00
ThibG 694a5e33d5 Add follower synchronization mechanism (#14510)
* Add support for followers synchronization on the receiving end

Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.

* Add tests for followers synchronization on the receiving end

* Add support for follower synchronization on the sender's end

* Add tests for the sending end

* Switch from AS attributes to HTTP header

Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
   receiving instance (where “receiving instance” is determined by accounts
   sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor

Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.

* Add environment variable to disable followers synchronization

Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.

The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
   netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
   netloc as a managed account must be managed by the same Mastodon instance
   as well.

As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.

Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.

* Only set follower synchronization header on non-public statuses

This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)

* Change how ActivityPub::SynchronizeFollowersService handles follow requests

If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.

* Integrate review feeback

- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes

* Only select required fields when computing followers_hash

* Use actor URI rather than webfinger domain in synchronization endpoint

* Change hash computation to be a XOR of individual hashes

Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.

* Marginally improve followers_hash computation speed

* Further improve hash computation performances by using pluck_each
2020-10-21 18:04:09 +02:00
OSAMU SATO ba9252b32d Add duration parameter to muting. (#13831)
* Adding duration to muting.

* Remove useless checks
2020-10-13 01:01:14 +02:00
Eugen Rochko da4c730c47 Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Eugen Rochko adaf3f7b7a Remove dependency on goldfinger gem (#14919)
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.

Fix #14091
2020-10-08 00:34:57 +02:00
ThibG 79b2d30a19 Add option to disable swiping motions across the WebUI (#13885)
Fixes #13882
2020-09-30 19:31:03 +02:00
Takeshi Umeda 67ee32365d Fix a slow query for TagFeed (#14861)
* Fix a slow query for TagFeed

* rename tags to tag_ids
2020-09-23 16:01:54 +02:00
Eugen Rochko 02c4c91faa Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
kawaguchi f13d563cb7 Fix validates :sign_count of WebauthnCredential (#14806) 2020-09-16 20:16:46 +02:00
Eugen Rochko e514304a76 Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
Eugen Rochko face3be227 Fix reported statuses not being included in warning e-mail (#14778) 2020-09-13 12:52:17 +02:00
tateisu 9d4769941f allow pagination by min_id and max_id (#14776)
* allow pagination by min_id and max_id

* also AccountConversation allows min_id,max_id pair

* also home,list TL allows min_id,max_id pair
2020-09-12 17:09:49 +02:00
Eugen Rochko 7fa0da9e98 Change REST API to return empty data for suspended accounts (#14765) 2020-09-11 15:16:29 +02:00
abcang e8c39853ad Changed tag most_used to recently_used (#14760) 2020-09-07 17:47:41 +02:00
Eugen Rochko 78ceda1af6 Refactor how public and tag timelines are queried (#14728) 2020-09-07 11:02:04 +02:00
ThibG 50d0c1e95f Add configuration option to filter replies in lists (#9205)
* Add database support for list show-reply preferences

* Add backend support to read and update list-specific show_replies settings

* Add basic UI to set list replies setting

* Add specs for list replies policy

* Switch "cycling" reply policy link to a set of radio inputs

* Capitalize replies_policy strings

* Change radio button design to be consistent with that of the directory explorer
2020-09-01 13:31:28 +02:00