Commit Graph

1033 Commits (ce8f33dd2f0fddf4bb30cbcdece8ed102838a0fb)

Author SHA1 Message Date
Akihiko Odaki 64ef37b89d
Introduce ApplicationController#cache_collection_paginated_by_id (#14677)
* Replace incorrect use of distinct with group

Some uses of ActiveRecord::QueryMethods#distinct pass field names but they
are incorrect for the current version of Rails.

ActiveRecord::QueryMethods#group provides the expected behavior and
benefits performance. See commit 6da24aad4c.

* Introduce ApplicationController#cache_collection_paginated_by_id

ApplicationController#cache_collection_paginated_by_id fuses
ApplicationController#cache_collection and Paginable.paginate_by_id.

An advantage of this method is that it prevents from modifying scope which
Paginable.paginate_by_id may provide.
ApplicationController#cache_collection always return an array and there
is no possibility of the scope modification. It is also clear for a
programmer, considering the implication of "cache".

This method can also emit more efficient queries by using
Cacheable.cache_ids before calling Paginable.paginate_by_id.
2020-08-28 12:31:56 +02:00
Akihiko Odaki b63ede5005
Eagerly load statuses with the main query in Api::V1::BookmarksController (#14674)
This is same with commit 552e886b64 except
that it was for Api::V1::FavouritesController while this is for
Api::V1::BookmarksController.
2020-08-28 12:30:23 +02:00
Akihiko Odaki e26e7a1cb5
Replace incorrect use of distinct with group (#14675)
Some uses of ActiveRecord::QueryMethods#distinct pass field names but they
are incorrect for the current version of Rails.

ActiveRecord::QueryMethods#group provides the expected behavior and
benefits performance. See commit 6da24aad4c.
2020-08-28 12:29:59 +02:00
Akihiko Odaki 552e886b64
Eagerly load statuses with the main query in Api::V1::FavouritesController (#14673)
The old implementation had two queries:
1. The query constructed in Api::V1::FavouritesController#results
2. The query constructed in #cached_favourites, which is merged with 1.

Both of them are issued againt PostgreSQL. The combination of the two
queries caused the following problems:
- The small window between the two queries involves race conditions.
- Minor performance inefficiency.

Moreover, the construction of query 2, which involves merging with query
1 has a bug. Query 1 is finalized with paginate_by_id, but paginate_by_id
returns an array when min_id parameter is specified. The behavior prevents
from merging the query, and in the real world, ActiveRecord simply ignores
the merge (!), which results in querying the entire scan of statuses and
favourites table.

This change fixes these issues by simply letting query 1 get all the works
done.
2020-08-28 09:27:33 +02:00
Akihiko Odaki 41eeb9ebaa
Use Status.group instead of Status.distinct in HashQueryService (#14662)
DISTINCT clause removes duplicated records according to all the selected
attributes. In reality, it can remove duplicated records only looking at
statuses.id, but the clause confuses the query planner and yields
insufficient performance.
The behavior is also problematic if the scope produced by HashQueryService
is used to query columns without id (using pluck method, for example). The
scope is expected to contain unique statuses, but the uniquness will be
evaluated with some arbitrary columns other than id.

GROUP BY clause resolves those problem by explicitly specifying the
column to take into account for the record distinction.

A workaround for the problem of DISTINCT clause in
Api::V1::Timelines::TagController is no longer necessary and removed.
2020-08-25 13:39:35 +02:00
ThibG b241f20bd2
Add support for latest HTTP Signatures spec draft (#14556)
* Add support for latest HTTP Signatures spec draft

https://www.ietf.org/id/draft-ietf-httpbis-message-signatures-00.html

- add support for the “hs2019” signature algorithm (assumed to be equivalent
  to RSA-SHA256, since we do not have a mechanism to specify the algorithm
  within the key metadata yet)
- add support for (created) and (expires) pseudo-headers and related
  signature parameters, when using the hs2019 signature algorithm
- adjust default “headers” parameter while being backwards-compatible with
  previous implementation
- change the acceptable time window logic from 12 hours surrounding the “date”
  header to accepting signatures created up to 1 hour in the future and
  expiring up to 1 hour in the past (but only allowing expiration dates up to
  12 hours after the creation date)
  This doesn't conform with the current draft, as it doesn't permit accounting
  for clock skew.
  This, however, should be addressed in a next version of the draft:
  https://github.com/httpwg/http-extensions/pull/1235

* Add additional signature requirements

* Rewrite signature params parsing using Parslet

* Make apparent which signature algorithm Mastodon on verification failure

Mastodon uses RSASSA-PKCS1-v1_5, which is not recommended for new applications,
and new implementers may thus unknowingly use RSASSA-PSS.

* Add workaround for PeerTube's invalid signature header

The previous parser allowed incorrect Signature headers, such as
those produced by old versions of the `http-signature` node.js package,
and seemingly used by PeerTube.

This commit adds a workaround for that.

* Fix `signature_key_id` raising an exception

Previously, parsing failures would result in `signature_key_id` being nil,
but the parser changes made that result in an exception.

This commit changes the `signature_key_id` method to return `nil` in case
of parsing failures.

* Move extra HTTP signature helper methods to private methods

* Relax (request-target) requirement to (request-target) || digest

This lets requests from Plume work without lowering security significantly.
2020-08-24 18:21:07 +02:00
santiagorodriguez96 e8d41bc2fe
Add WebAuthn as an alternative 2FA method (#14466)
* feat: add possibility of adding WebAuthn security keys to use as 2FA

This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add request for WebAuthn as second factor at login if enabled

This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add possibility of deleting WebAuthn Credentials

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: disable WebAuthn when an Admin disables 2FA for a user

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA

Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.

* We had to change a little the flow for setting up TOTP, given that now
  it's possible to setting up again if you already had TOTP, in order to
  let users modify their authenticator app – given that now it's not
  possible for them to disable TOTP and set it up again with another
  authenticator app.
  So, basically, now instead of storing the new `otp_secret` in the
  user, we store it in the session until the process of set up is
  finished.
  This was because, as it was before, when users clicked on 'Edit' in
  the new two-factor methods lists page, but then went back without
  finishing the flow, their `otp_secret` had been changed therefore
  invalidating their previous authenticator app, making them unable to
  log in again using TOTP.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* refactor: fix eslint errors

The PR build was failing given that linting returning some errors.
This commit attempts to fix them.

* refactor: normalize i18n translations

The build was failing given that i18n translations files were not
normalized.
This commits fixes that.

* refactor: avoid having the webauthn gem locked to a specific version

* refactor: use symbols for routes without '/'

* refactor: avoid sending webauthn disabled email when 2FA is disabled

When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.

* refactor: avoid creating new env variable for webauthn_origin config

* refactor: improve flash error messages for webauthn pages

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
2020-08-24 16:46:27 +02:00
ThibG f6a82cb2cd
Fix not being able to unbookmark toots when blocked by their author (#14604)
* Fix not being able to unbookmark toots when blocked by their author

* Add tests
2020-08-19 19:02:06 +02:00
ThibG 8d217d7231
Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
ThibG f55dd193f9
Fix RSS feeds not being cachable (#14368)
* Add tests for some cachable responses

This only covers responses that we should have managed to make cachable
so far. It's not the case of all responses that should be cachable in
the end.

* Fix RSS feeds not being cachable
2020-07-22 11:44:02 +02:00
Ariel 2ada2ae18a
Fix/14021 behaviour on add or remove toots (#14212)
* Add toot send by current user at local state after send a new toot

Related to #14021

* Decrement toot counter at profile when remove a toot

Related to #14021

* Remove semicolon at end of line
2020-07-19 17:04:02 +02:00
ThibG 0a8a3fe595
Fix being unable to unboost when blocked by their author (#14308)
Fixes #14307
2020-07-15 14:43:19 +02:00
abcang 6a96af4d20
Fix rubocop warning (#14288)
* Fix rubocop warning

* use limit variable

* use ContextCreatingMethods option
2020-07-14 19:05:07 +02:00
Eugen Rochko 6e25574ce5
Fix media attachments enumeration (#14254)
* Fix media attachment enumeration

* Switch media_attachments id to snowflake ids

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-07-07 15:26:51 +02:00
Eugen Rochko 844870273f
Fix other sessions not being logged out on password change (#14252)
While OAuth tokens were immediately revoked, accessing the home
controller immediately generated new OAuth tokens and "revived"
the session due to a combination of using remember_me tokens and
overwriting the `authenticate_user!` method
2020-07-07 15:26:31 +02:00
ThibG 65506bac3f
Add user notes on accounts (#14148)
* Add UserNote model

* Add UI for user notes

* Put comment in relationships entity

* Add API to create user notes

* Copy user notes to new account when receiving a Move activity

* Address some of the review remarks

* Replace modal by inline edition

* Please CodeClimate

* Button design changes

* Change design again

* Cancel note edition when pressing Escape

* Fixes

* Tweak design again

* Move “Add note” item, and allow users to add notes to themselves

* Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”
2020-06-30 19:19:50 +02:00
Eugen Rochko 64aac30733
Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
fuyu a279acc730
Fix not working I18n on 2FA and Sign in token page (#14087) 2020-06-20 13:30:13 +02:00
ThibG ac8a788370
Fix functional user requirements in whitelist mode (#14093)
Fixes #14092
2020-06-19 19:18:47 +02:00
Eugen Rochko 72a7cfaa39
Add e-mail-based sign in challenge for users with disabled 2FA (#14013) 2020-06-09 10:23:06 +02:00
Takeshi Umeda 8b6d97fb7c
Add limit parameter to rss (#13743) 2020-06-09 00:18:47 +02:00
ThibG bf6745b9c3
Fix unpermitted operations on custom emojis leading to cryptic errors (#13951)
* Display appropriate error when performing unpermitted operation on custom emoji

Fixes #13897

* Remove links to custom emoji actions not performable by moderators
2020-06-05 15:23:27 +02:00
Eugen Rochko f669b8bcce
Fix wrong route helper in encrypted messages controller (#13952)
And add `created_at` to encrypted message serializer
2020-06-03 20:32:15 +02:00
ThibG bf94934623
Fix account redirect confirmation message talking about moved followers (#13950)
Fixes #13949
2020-06-03 20:18:19 +02:00
Eugen Rochko 5d8398c8b8
Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
ThibG 71fce71c94
Fix webfinger returning wrong status code on malformed or missing param (#13759)
Fixes #13757
2020-05-14 23:28:06 +02:00
ThibG 4bcef12bad
Fix sr locale being selected over sr-Latn (#13693)
* Fix sr locale being selected over sr-Latn

* Update tests
2020-05-11 01:09:21 +02:00
Eugen Rochko 8be4c2ba21
Add ability to remove identity proofs from account (#13682)
Fix #12613
2020-05-10 11:21:10 +02:00
Takeshi Umeda 26b08a3c54
Add remote only to public timeline (#13504)
* Add remote only to public timeline

* Fix code style
2020-05-10 10:36:18 +02:00
ThibG 45202f79ef
Remove confusing “You are already signed in.” flash message (#13547)
When attempting to access the log-in page while already logged in,
Devise's `require_no_authentication` kicks in and sets a flash
message “You are already signed in.”

In almost all cases, this also causes a redirect to /web, which
does not display or clear flash messages, thus leaving the message
to a potentially much later date, like for instance, accessing
/preferences several minutes after being redirected to /web.
2020-05-10 10:16:39 +02:00
ThibG f1e0fa80f6
Fix own following/followers not showing muted users (#13614)
Fixes #13612
2020-05-08 20:36:34 +02:00
Eugen Rochko 5cff7910c2
Add more ActivityPub controller tests (#13590) 2020-05-03 22:19:24 +02:00
Eugen Rochko 988b0493fe
Add more tests for ActivityPub controllers (#13585) 2020-05-03 16:30:36 +02:00
sternenseemann 679980f77c
Allow users to delete their header and avatar (#13234)
This is achieved by sending a DELETE request to
/settings/profile/pictures/{avatar,header} via a link that is part of
the upload form's hint of the respective picture.
2020-04-20 14:03:03 +02:00
Takeshi Umeda 2c7128c7f0
Add local only to hashtag timeline (#13502) 2020-04-18 21:52:39 +02:00
Eugen Rochko 5edff32733
Change delivery failure tracking to work with hostnames instead of URLs (#13437) 2020-04-15 20:33:24 +02:00
Eugen Rochko c9efb400b4
Add rate limit for reporting (#13390) 2020-04-05 14:40:08 +02:00
Eugen Rochko f65568f1d4
Add ability to filter audit log in admin UI (#13381) 2020-04-03 13:06:34 +02:00
Takeshi Umeda e4617c8ed8
Fix ImportsController param to permit :mode (#13347) 2020-03-31 12:43:42 +02:00
ThibG 0d117c106a
Fix 404 and 410 API errors being silently discarded in WebUI (#13279)
* Fix 404 and 410 API errors being silently discarded in WebUI

Fixes #13278

* Return more appropriate error when user replies to a deleted toot

* Please CodeClimate

* Fix 404/410 errors on fetching account timelines & identity proofs

* Refactor error handling

* Move error message string to statuses.errors
2020-03-28 17:59:45 +01:00
Eugen Rochko bea0bb39d6
Add option to include resolved DNS records when blacklisting e-mail domains in admin UI (#13254)
* Add shortcuts to blacklist a user's e-mail domain in admin UI

* Add option to blacklist resolved MX and IP records for e-mail domains
2020-03-12 22:35:20 +01:00
Eugen Rochko f556f79b77
Add titles to warning presets in admin UI (#13252) 2020-03-12 17:57:59 +01:00
ThibG b154428e14
Add federation support for the "hide network" preference (#11673)
* Change ActivityPub follower/following collections to not link first page

* Add support for hiding followers and following of remote users

* Switch to using a single `hide_collections` column

* Address code style remarks
2020-03-09 00:10:29 +01:00
Eugen Rochko 9660aa4543
Change local media attachments to perform heavy processing asynchronously (#13210)
Fix #9106
2020-03-08 23:56:18 +01:00
ThibG 2423d2f677
Add ability to delete files uploaded for settings in admin UI (#13192)
* Allow deleting site uploads

* Refactor and move links into hints

* Fix i18n tests

* Fix HTML output of site_upload_delete_hint
2020-03-08 16:00:24 +01:00
Eugen Rochko 339ce1c4e9
Add specific rate limits for posting and following (#13172) 2020-03-08 15:17:39 +01:00
Eugen Rochko 0ae5c6312f
Remove useless `respond_to` calls (#13208) 2020-03-06 01:29:38 +01:00
Eugen Rochko 0c28a505dd
Fix leak of arbitrary statuses through unfavourite action in REST API (#13161) 2020-02-27 12:32:54 +01:00
ThibG 7face973fa
Fix dismissing an announcement twice raising an obscure error (#13124) 2020-02-24 22:21:40 +01:00
ThibG d8e9bae482
Fix account JSON/RSS not being cacheable due to wrong mime type comparison (#13116)
`request.format` is not a symbol but a `Mime::Type`, so the condition actually
never matched, and a session was created even for those requests, preventing
caching.
2020-02-19 22:31:53 +01:00