Commit Graph

1306 Commits (d82b747d997ef19efe282aca0903c06b15b53aac)

Author SHA1 Message Date
Jeong Arm e71efc0497 Remove set-cookie header on custom.css (#16314)
* Remove set-cookie header on custom.css

* Additional fix for set-cookie
2021-05-30 17:57:47 +02:00
Claire d0e2ae1769 Fix media proxy RedisLocks auto-releasing too fast (#16291)
Follow-up to #16276
2021-05-22 15:00:33 +02:00
Eugen Rochko 91819606f9 Change trending hashtags to be affected be reblogs (#16164)
If a status with a hashtag becomes very popular, it stands to
reason that the hashtag should have a chance at trending

Fix no stats being recorded for hashtags that are not allowed
to trend, and stop ignoring bots

Remove references to hashtags in profile directory from the code
and the admin UI
2021-05-07 14:33:43 +02:00
Claire 86f5fad111 Add Ruby 3.0 support (#16046)
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0

Also improve the Terrapin monkey-patch for the stderr/stdout issue.

* Fix keyword argument handling throughout the codebase

* Monkey-patch Paperclip to fix keyword arguments handling in validators

* Change validation_extensions to please CodeClimate

* Bump microformats from 4.2.1 to 4.3.1

* Allow Ruby 3.0

* Add Ruby 3.0 test target to CircleCI

* Add test for admin dashboard warnings

* Fix admin dashboard warnings on Ruby 3.0
2021-05-06 14:22:54 +02:00
Takeshi Umeda c958bc7297 Add management of delivery availability in Federation settings (#15771)
* Add management of delivery availavility in Federation settings

* fix translate

* Remove useless object creation

* Fix DeepSource issue

* Add shortcut for all

* Fix DeepSource(skipcq)

* Change 'remove' to 'clear'

* Fix style

* Change class method name (exhausted_deliveries_key_by)
2021-05-05 23:39:02 +02:00
Eugen Rochko 5b255015f5 Fix error when trying to render component for media without meta (#16112) 2021-05-05 21:16:55 +02:00
Eugen Rochko d0c3b94271 Fix database serialization failure returning HTTP 500 (#16101)
Database serialization failure occurs when a read-replica is used
and a query takes long enough that rows on the primary database
become unavailable. It should return HTTP 503 as it is temporary.

Re-order rescue definitions according to their status codes
2021-05-05 19:44:35 +02:00
Claire db57eaf207 Change confirmations controller to redirect to / for approved users (#16151)
Clicking the confirmation link multiple times currently leads to entering
account settings, which can be confusing. This commit changes that so that
it redirects to the root path, so it behaves the same way as clicking only
once in most cases.
2021-05-03 15:45:19 +02:00
abcang dec6f34546 Further improve the media attached status query for accounts (#16106) 2021-04-26 18:57:46 +02:00
abcang fa2d62e6e2 Improve media attached status query (#16105) 2021-04-25 06:34:48 +02:00
Eugen Rochko f5d59b3979 Change auto-following admin-selected accounts, show in recommendations (#16078) 2021-04-24 17:01:43 +02:00
Eugen Rochko 2cab9c9f06 Add `policy` param to `POST /api/v1/push/subscriptions` (#16040)
With possible values `all`, `followed`, `follower`, and `none`,
control from whom notifications will generate a Web Push alert
2021-04-15 05:00:25 +02:00
Eugen Rochko 92b2d926bf Add cold-start follow recommendations (#15945) 2021-04-12 12:37:14 +02:00
Eugen Rochko 2e8a673376 Remove spam check and dependency on nilsimsa gem (#16011) 2021-04-11 11:22:50 +02:00
Eugen Rochko 19df881320 Add system checks to dashboard in admin UI (#15989) 2021-04-03 14:12:30 +02:00
Eugen Rochko 2e0cdf30d5 Change health check (#15988) 2021-04-03 02:39:04 +02:00
Claire 7e74891ff0 Fix /admin/tags/:id crashing since Rails 6.1 update (#15953)
Raw SQL passed to `pluck` now has to be explicitly marked as SQL via
Arel.sql, see https://github.com/rails/rails/pull/27947
2021-03-26 18:36:16 +01:00
Claire 151e79fe72 Add warning in admin dashboard if some required queues are not handled (#15954) 2021-03-26 18:22:54 +01:00
Eugen Rochko 6440cd74e9 Add `email` param to `POST /api/v1/emails/confirmations` (#15949)
Allow changing e-mail as long as the account is unconfirmed
2021-03-25 02:46:13 +01:00
Claire b2a89bf38e Update Mastodon to Rails 6.1 (#15910)
* Update devise-two-factor to unreleased fork for Rails 6 support

Update tests to match new `rotp` version.

* Update nsa gem to unreleased fork for Rails 6 support

* Update rails to 6.1.3 and rails-i18n to 6.0

* Update to unreleased fork of pluck_each for Ruby 6 support

* Run "rails app:update"

* Add missing ActiveStorage config file

* Use config.ssl_options instead of removed ApplicationController#force_ssl

Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.

* Fix nonce directives by removing Rails 5 specific monkey-patching

* Fix fixture_file_upload deprecation warning

* Fix yield-based test failing with Rails 6

* Use Rails 6's index_with when possible

* Use ActiveRecord::Cache::Store#delete_multi from Rails 6

This will yield better performances when deleting an account

* Disable Rails 6.1's automatic preload link headers

Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.

In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.

* Switch to Rails 6.0 default config

* Switch to Rails 6.1 default config

* Do not include autoload paths in the load path
2021-03-24 10:44:31 +01:00
Claire 6024c263bf Further preparation for Rails 6 (#15916)
* Use ActiveRecord::Result#to_ary instead of deprecated to_hash

They do the same thing, and to_hash has been removed from Rails 6.1

* Explicitly name polymorphic indexes to workaround a bug in Rails 6.1

cf. https://github.com/rails/rails/issues/41693

* Fix incorrect usage of “foreign_key” in migration script

* Use `ActiveModel::Errors#delete` instead of deprecated clear method

* Fix link headers tests on Rails 6.1

Rails 6.1 adds values to the Link header by default, thus it is not a
LinkHeader object anymore. Fix the test to parse the Link header instead
of assuming it is a LinkHeader.
2021-03-19 02:45:34 +01:00
Claire b35955af26 Fix cache_collection crashing when given an empty collection (#15921)
* Fix cache_collection crashing when given an empty collection

* Add tests
2021-03-18 00:41:32 +01:00
Eugen Rochko 28399e88b5 Add `POST /api/v1/emails/confirmations` to REST API (#15816)
Only available to the application the user originally signed-up with
2021-03-01 18:39:47 +01:00
Eugen Rochko ade70733f7 Add `details` to error response for `POST /api/v1/accounts` in REST API (#15803) 2021-03-01 04:59:13 +01:00
Claire d1deaee77f Fix crash on receiving requests with missing Digest header (#15782)
* Fix crash on receiving requests with missing Digest header

Return an error pointing out that Digest is missing, instead of crashing.

Fixes #15743

* Fix from review feedback
2021-02-26 17:40:27 +01:00
Eugen Rochko 4930e71ae7 Add server rules (#15769) 2021-02-21 19:50:12 +01:00
Justin Tracey cbcf083cb4 replace all instances of "ends_with?" with "end_with?" (#15745)
The "ends_with?" method is just a Rails alias of Ruby's "end_with?" method.
Using the latter makes the code less brittle.
2021-02-19 09:56:14 +01:00
Eugen Rochko e4ba5635ee Add `GET /api/v1/accounts/lookup` REST API (#15740) 2021-02-16 15:28:32 +01:00
Eugen Rochko ecfcdc856b Refactor Api::Web::SettingsController (#15717) 2021-02-12 07:19:15 +01:00
Cecylia Bocovich 6c300f08e5 Onion service related changes to HTTPS handling (#15560)
* Enable secure cookie flag for https only

* Disable force_ssl for .onion hosts only

Co-authored-by: Aiden McClelland <me@drbonez.dev>
2021-02-11 04:40:13 +01:00
Claire 4028717020 Create instance actor if it hasn't been properly seeded (#15693)
An uncommon but somewhat difficult to digagnose issue is dealing with
improperly-seeded databases. In such cases, instance-signed fetches will
fail with a ActiveRecord::RecordNotFound error, usually caught and handled
as generic 404, leading people to think the remote resource itself has not
been found, while it's the local instance actor that does not exist.

This commit changes the code so that failure to find the instance actor
automatically creates a new one, so that improperly-seeded databases do
not cause any issue.
2021-02-09 18:12:54 +01:00
abcang 2b9d71cc18 Improved performance of notification preloading (#15640)
* Improved performance of notification preloading

* Remove Cacheable from Notification

* Fix test
2021-01-31 21:24:57 +01:00
ThibG 320355ce28 Fix /activity endpoint not require signature in authorized fetch mode (#15592)
Fixes #15589

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-19 06:47:36 +01:00
ThibG 2582047c36 Use Rails' index_by where it makes sense (#15542)
* Use Rails' index_by where it makes sense

* Fix tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-12 09:27:38 +01:00
Levi Bard 87d45a9cd8 Fix muting users with duration via the REST api (#15516) 2021-01-10 12:47:21 +01:00
luigi 944b059f50 Optimize map { ... }.compact calls (#15513)
* Optimize map { ... }.compact

using Enumerable#filter_map, supported since Ruby 2.7

* Add poyfill for Enumerable#filter_map
2021-01-10 00:32:01 +01:00
ThibG 8a54766750 Skip processing Update activities on unknown accounts (#15514)
This also skips fetching the actor completely.

This will be useful if we end up distributing Update activities linked to
account suspensions more widely (they are currently only delivered to
the suspended account's followers), as currently, instances not knowing
about the suspended account would fetch it to then process the suspension.

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-08 12:26:29 +01:00
ThibG e56b3f03aa Fix profile update not being sent on profile/header picture deletion (#15461)
Fixes #15460

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-30 23:19:18 +01:00
trwnh 33e4e1cfe4 Use existing FeaturedTag serializer and delete AccountFeaturedTag serializer (#15415)
* Update featured_tags_controller.rb

* Update featured_tag_serializer.rb

* Update featured_tag_serializer.rb

* Delete account_featured_tag_serializer.rb

* please codeclimate

* please codeclimate
2020-12-23 16:43:38 +01:00
Eugen Rochko 816df80b86 Add option to obfuscate domain name in public list of domain blocks (#15355)
- Replace the middle of the domain with * characters (except for periods)
- Add SHA-256 digest of the domain name in tooltip
2020-12-18 08:30:41 +01:00
Eugen Rochko 76b0f84cd5 Add stoplight for object storage failures, return HTTP 503 (#13043) 2020-12-15 12:55:29 +01:00
Eugen Rochko df8874b24e Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
ThibG e1ef5f3b31 Add honeypot fields and minimum fill-out time for sign-up form (#15276)
* Add honeypot fields to limit non-specialized spam

Add two honeypot fields: a fake website input and a fake password confirmation
one. The label/placeholder/aria-label tells not to fill them, and they are
hidden in CSS, so legitimate users should not fall into these.

This should cut down on some non-Mastodon-specific spambots.

* Require a 3 seconds delay before submitting the registration form

* Fix tests

* Move registration form time check to model validation

* Give people a chance to clear the honeypot fields

* Refactor honeypot translation strings

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-10 06:27:26 +01:00
Eugen Rochko 48534762cf Fix slow distinct queries where grouped queries are faster (#15287)
About 2x speed-up on inboxes query
2020-12-07 12:08:30 +01:00
Takeshi Umeda fabb864526 Fix incorrect conditions for suspended accounts in Get API for account featured tags (#15270) 2020-12-04 04:22:35 +01:00
ThibG a609802736 Fix not being able to unfavorite toots one has lost access to (#15192)
Fixes #15191
2020-11-21 06:18:09 +01:00
ThibG 8e79bac43d Add import/export feature for bookmarks (#14956)
* Add ability to export bookmarks

* Add support for importing bookmarks

* Add bookmark import tests

* Add bookmarks export test
2020-11-19 17:48:13 +01:00
Eugen Rochko 90faa8039c Fix 2FA/sign-in token sessions being valid after password change (#14802)
If someone tries logging in to an account and is prompted for a 2FA
code or sign-in token, even if the account's password or e-mail is
updated in the meantime, the session will show the prompt and allow
the login process to complete with a valid 2FA code or sign-in token
2020-11-12 23:05:01 +01:00
Takeshi Umeda f0a79a9fa8 Add interrelationship icon (#15149)
* Add interrelationship icon

* Fix arrow for rtl

* Fix to predefined color
2020-11-12 17:43:12 +01:00
Takeshi Umeda 1dde6183a3 Add follow selected followers button (#15148)
* Add follow selected followers button

* Fix unused variable

* Fix i18n normalize
2020-11-12 16:58:00 +01:00
Eugen Rochko 38be9af436 Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
ThibG b1a78a5cce Change order of announcements in admin page to sort them newest-first (#15091)
* Change order of announcements in admin page to sort show newly-created first

Fixes #15090

* Use reverse-chronological rather than creation date only
2020-11-04 21:15:22 +01:00
Takeshi Umeda 385c2eac6f Add account sensitized (#14361)
* Add account sensitized

* Fix i18n normalize

* Fix description and spec

* Fix spec

* Fix wording
2020-11-04 20:45:01 +01:00
ThibG 9500d00e5f Tweak signature verification (#15069)
* Add more specific error message when request body digest is invalid

This may help other implementors debug their implementation.

* Relax Host parameter requirement to GET requests

The only POST requests processed by Mastodon need objects/actors (including
their host) to be explicitly mentioned in the request's body, so replaying
a legitimate request to another host should not be a security issue.

* Support Digest headers using multiple algorithms or lowercase alogirthm names
2020-11-01 23:38:31 +01:00
Mélanie Chauvel c93e98ad47 Sort filters by “keyword or phrase” in Settings (#15050) 2020-10-27 03:00:06 +01:00
ThibG 694a5e33d5 Add follower synchronization mechanism (#14510)
* Add support for followers synchronization on the receiving end

Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.

* Add tests for followers synchronization on the receiving end

* Add support for follower synchronization on the sender's end

* Add tests for the sending end

* Switch from AS attributes to HTTP header

Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
   receiving instance (where “receiving instance” is determined by accounts
   sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor

Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.

* Add environment variable to disable followers synchronization

Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.

The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
   netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
   netloc as a managed account must be managed by the same Mastodon instance
   as well.

As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.

Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.

* Only set follower synchronization header on non-public statuses

This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)

* Change how ActivityPub::SynchronizeFollowersService handles follow requests

If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.

* Integrate review feeback

- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes

* Only select required fields when computing followers_hash

* Use actor URI rather than webfinger domain in synchronization endpoint

* Change hash computation to be a XOR of individual hashes

Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.

* Marginally improve followers_hash computation speed

* Further improve hash computation performances by using pluck_each
2020-10-21 18:04:09 +02:00
OSAMU SATO ba9252b32d Add duration parameter to muting. (#13831)
* Adding duration to muting.

* Remove useless checks
2020-10-13 01:01:14 +02:00
Eugen Rochko da4c730c47 Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
ThibG 79b2d30a19 Add option to disable swiping motions across the WebUI (#13885)
Fixes #13882
2020-09-30 19:31:03 +02:00
Eugen Rochko 2539d7d655 Fix not being able to enable status notifications in Web Push API (#14822) 2020-09-21 17:48:55 +02:00
Eugen Rochko 02c4c91faa Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
Eugen Rochko e514304a76 Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
ThibG 739f7815ef Do not serve account actors at all in limited federation mode (#14800)
* Do not serve account actors at all in limited federation mode

When an account is fetched without a signature from an allowed instance,
return an error.

This isn't really an improvement in security, as the only information that was
previously returned was required protocol-level info, and the only personal bit
was the existence of the account. The existence of the account can still be
checked by issuing a webfinger query, as those are accepted without signatures.

However, this change makes it so that unallowed instances won't create account
records on their end when they find a reference to an unknown account.

The previous behavior of rendering a limited list of fields, instead of not
rendering the actor at all, was in order to prevent situations in which two
instances in Authorized Fetch mode or Limited Federation mode would fail to
reach each other because resolving an account would require a signed query…
from an account which can only be fetched with a signed query itself. However,
this should now be fine as fetching accounts is done by signing on behalf of
the special instance actor, which does not require any kind of valid signature
to be fetched.

* Fix tests
2020-09-14 13:04:29 +02:00
Eugen Rochko 403aeeb7e0 Refactor settings controllers (#14767)
- Disallow suspended accounts from revoking sessions and apps
- Allow suspended accounts to access exports
2020-09-11 20:56:35 +02:00
Eugen Rochko 7fa0da9e98 Change REST API to return empty data for suspended accounts (#14765) 2020-09-11 15:16:29 +02:00
abcang e8c39853ad Changed tag most_used to recently_used (#14760) 2020-09-07 17:47:41 +02:00
Eugen Rochko 78ceda1af6 Refactor how public and tag timelines are queried (#14728) 2020-09-07 11:02:04 +02:00
ThibG 1a3405005a Add outbox attribute to instance actor (#14721)
It's not useful for now, but it's required by ActivityPub
2020-09-02 18:42:50 +02:00
Takeshi Umeda cb37bd63dc Added account featured tags API (#11817) 2020-09-02 02:13:10 +02:00
Eugen Rochko c6ad9aae69 Add featured hashtags as an ActivityPub collection (#11595) 2020-09-02 02:11:12 +02:00
ThibG 50d0c1e95f Add configuration option to filter replies in lists (#9205)
* Add database support for list show-reply preferences

* Add backend support to read and update list-specific show_replies settings

* Add basic UI to set list replies setting

* Add specs for list replies policy

* Switch "cycling" reply policy link to a set of radio inputs

* Capitalize replies_policy strings

* Change radio button design to be consistent with that of the directory explorer
2020-09-01 13:31:28 +02:00
dependabot[bot] 61b768572e Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
Akihiko Odaki b096ad1c0f Make Array-creation behavior of Paginable more predictable (#14687)
* Make Array-creation behavior of Paginable more predictable

Paginable.paginate_by_id usually returns ActiveRecord::Relation, but it
returns an Array if min_id option is present. The behavior caused problems
fixed with the following commits:
- 976938bf30
- 5a8fa3ad95
- 1c70d8189c

To prevent from recurring similar problems, this commit introduces two
changes:
- The scope now always returns an Array whether min_id option is present
  or not.
- The scope is renamed to to_a_paginated_by_id to clarify it returns an
  Array.

* Transform Paginable.to_a_paginated_by_id from a scope to a class method

https://api.rubyonrails.org/classes/ActiveRecord/Scoping/Named/ClassMethods.html#method-i-scope
> The method is intended to return an ActiveRecord::Relation object, which
> is composable with other scopes.

Paginable.to_a_paginated_by_id returns an Array and is not appropriate
as a scope.
2020-08-31 12:47:09 +02:00
Akihiko Odaki 1c70d8189c Introduce ApplicationController#cache_collection_paginated_by_id (#14677)
* Replace incorrect use of distinct with group

Some uses of ActiveRecord::QueryMethods#distinct pass field names but they
are incorrect for the current version of Rails.

ActiveRecord::QueryMethods#group provides the expected behavior and
benefits performance. See commit 6da24aad4cafdef8d8a2c92bac2002a5fc2fe9c8.

* Introduce ApplicationController#cache_collection_paginated_by_id

ApplicationController#cache_collection_paginated_by_id fuses
ApplicationController#cache_collection and Paginable.paginate_by_id.

An advantage of this method is that it prevents from modifying scope which
Paginable.paginate_by_id may provide.
ApplicationController#cache_collection always return an array and there
is no possibility of the scope modification. It is also clear for a
programmer, considering the implication of "cache".

This method can also emit more efficient queries by using
Cacheable.cache_ids before calling Paginable.paginate_by_id.
2020-08-28 12:31:56 +02:00
Akihiko Odaki 5a8fa3ad95 Eagerly load statuses with the main query in Api::V1::BookmarksController (#14674)
This is same with commit 976938bf30 except
that it was for Api::V1::FavouritesController while this is for
Api::V1::BookmarksController.
2020-08-28 12:30:23 +02:00
Akihiko Odaki 81666b26a6 Replace incorrect use of distinct with group (#14675)
Some uses of ActiveRecord::QueryMethods#distinct pass field names but they
are incorrect for the current version of Rails.

ActiveRecord::QueryMethods#group provides the expected behavior and
benefits performance. See commit 6da24aad4cafdef8d8a2c92bac2002a5fc2fe9c8.
2020-08-28 12:29:59 +02:00
Akihiko Odaki 976938bf30 Eagerly load statuses with the main query in Api::V1::FavouritesController (#14673)
The old implementation had two queries:
1. The query constructed in Api::V1::FavouritesController#results
2. The query constructed in #cached_favourites, which is merged with 1.

Both of them are issued againt PostgreSQL. The combination of the two
queries caused the following problems:
- The small window between the two queries involves race conditions.
- Minor performance inefficiency.

Moreover, the construction of query 2, which involves merging with query
1 has a bug. Query 1 is finalized with paginate_by_id, but paginate_by_id
returns an array when min_id parameter is specified. The behavior prevents
from merging the query, and in the real world, ActiveRecord simply ignores
the merge (!), which results in querying the entire scan of statuses and
favourites table.

This change fixes these issues by simply letting query 1 get all the works
done.
2020-08-28 09:27:33 +02:00
Akihiko Odaki 44df785077 Use Status.group instead of Status.distinct in HashQueryService (#14662)
DISTINCT clause removes duplicated records according to all the selected
attributes. In reality, it can remove duplicated records only looking at
statuses.id, but the clause confuses the query planner and yields
insufficient performance.
The behavior is also problematic if the scope produced by HashQueryService
is used to query columns without id (using pluck method, for example). The
scope is expected to contain unique statuses, but the uniquness will be
evaluated with some arbitrary columns other than id.

GROUP BY clause resolves those problem by explicitly specifying the
column to take into account for the record distinction.

A workaround for the problem of DISTINCT clause in
Api::V1::Timelines::TagController is no longer necessary and removed.
2020-08-25 13:39:35 +02:00
ThibG d480442e58 Add support for latest HTTP Signatures spec draft (#14556)
* Add support for latest HTTP Signatures spec draft

https://www.ietf.org/id/draft-ietf-httpbis-message-signatures-00.html

- add support for the “hs2019” signature algorithm (assumed to be equivalent
  to RSA-SHA256, since we do not have a mechanism to specify the algorithm
  within the key metadata yet)
- add support for (created) and (expires) pseudo-headers and related
  signature parameters, when using the hs2019 signature algorithm
- adjust default “headers” parameter while being backwards-compatible with
  previous implementation
- change the acceptable time window logic from 12 hours surrounding the “date”
  header to accepting signatures created up to 1 hour in the future and
  expiring up to 1 hour in the past (but only allowing expiration dates up to
  12 hours after the creation date)
  This doesn't conform with the current draft, as it doesn't permit accounting
  for clock skew.
  This, however, should be addressed in a next version of the draft:
  https://github.com/httpwg/http-extensions/pull/1235

* Add additional signature requirements

* Rewrite signature params parsing using Parslet

* Make apparent which signature algorithm Mastodon on verification failure

Mastodon uses RSASSA-PKCS1-v1_5, which is not recommended for new applications,
and new implementers may thus unknowingly use RSASSA-PSS.

* Add workaround for PeerTube's invalid signature header

The previous parser allowed incorrect Signature headers, such as
those produced by old versions of the `http-signature` node.js package,
and seemingly used by PeerTube.

This commit adds a workaround for that.

* Fix `signature_key_id` raising an exception

Previously, parsing failures would result in `signature_key_id` being nil,
but the parser changes made that result in an exception.

This commit changes the `signature_key_id` method to return `nil` in case
of parsing failures.

* Move extra HTTP signature helper methods to private methods

* Relax (request-target) requirement to (request-target) || digest

This lets requests from Plume work without lowering security significantly.
2020-08-24 18:21:07 +02:00
santiagorodriguez96 f142983484 Add WebAuthn as an alternative 2FA method (#14466)
* feat: add possibility of adding WebAuthn security keys to use as 2FA

This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add request for WebAuthn as second factor at login if enabled

This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add possibility of deleting WebAuthn Credentials

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: disable WebAuthn when an Admin disables 2FA for a user

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA

Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.

* We had to change a little the flow for setting up TOTP, given that now
  it's possible to setting up again if you already had TOTP, in order to
  let users modify their authenticator app – given that now it's not
  possible for them to disable TOTP and set it up again with another
  authenticator app.
  So, basically, now instead of storing the new `otp_secret` in the
  user, we store it in the session until the process of set up is
  finished.
  This was because, as it was before, when users clicked on 'Edit' in
  the new two-factor methods lists page, but then went back without
  finishing the flow, their `otp_secret` had been changed therefore
  invalidating their previous authenticator app, making them unable to
  log in again using TOTP.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* refactor: fix eslint errors

The PR build was failing given that linting returning some errors.
This commit attempts to fix them.

* refactor: normalize i18n translations

The build was failing given that i18n translations files were not
normalized.
This commits fixes that.

* refactor: avoid having the webauthn gem locked to a specific version

* refactor: use symbols for routes without '/'

* refactor: avoid sending webauthn disabled email when 2FA is disabled

When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.

* refactor: avoid creating new env variable for webauthn_origin config

* refactor: improve flash error messages for webauthn pages

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
2020-08-24 16:46:27 +02:00
ThibG 4e292c23d6 Fix not being able to unbookmark toots when blocked by their author (#14604)
* Fix not being able to unbookmark toots when blocked by their author

* Add tests
2020-08-19 19:02:06 +02:00
ThibG 8baad22f68 Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
ThibG 7792bab1cd Fix RSS feeds not being cachable (#14368)
* Add tests for some cachable responses

This only covers responses that we should have managed to make cachable
so far. It's not the case of all responses that should be cachable in
the end.

* Fix RSS feeds not being cachable
2020-07-22 11:44:02 +02:00
Ariel ac1a3386bd Fix/14021 behaviour on add or remove toots (#14212)
* Add toot send by current user at local state after send a new toot

Related to #14021

* Decrement toot counter at profile when remove a toot

Related to #14021

* Remove semicolon at end of line
2020-07-19 17:04:02 +02:00
ThibG c9fbf47a23 Fix being unable to unboost when blocked by their author (#14308)
Fixes #14307
2020-07-15 14:43:19 +02:00
abcang aca93fa882 Fix rubocop warning (#14288)
* Fix rubocop warning

* use limit variable

* use ContextCreatingMethods option
2020-07-14 19:05:07 +02:00
Eugen Rochko 0edde9d6c8 Fix media attachments enumeration (#14254)
* Fix media attachment enumeration

* Switch media_attachments id to snowflake ids

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-07-07 15:26:51 +02:00
Eugen Rochko a79c5e5e63 Fix other sessions not being logged out on password change (#14252)
While OAuth tokens were immediately revoked, accessing the home
controller immediately generated new OAuth tokens and "revived"
the session due to a combination of using remember_me tokens and
overwriting the `authenticate_user!` method
2020-07-07 15:26:31 +02:00
ThibG 3afc7977b1 Add user notes on accounts (#14148)
* Add UserNote model

* Add UI for user notes

* Put comment in relationships entity

* Add API to create user notes

* Copy user notes to new account when receiving a Move activity

* Address some of the review remarks

* Replace modal by inline edition

* Please CodeClimate

* Button design changes

* Change design again

* Cancel note edition when pressing Escape

* Fixes

* Tweak design again

* Move “Add note” item, and allow users to add notes to themselves

* Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”
2020-06-30 19:19:50 +02:00
Eugen Rochko a3ce01a102 Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
fuyu ad1dfea4fa Fix not working I18n on 2FA and Sign in token page (#14087) 2020-06-20 13:30:13 +02:00
ThibG 6a3e2b0fe5 Fix functional user requirements in whitelist mode (#14093)
Fixes #14092
2020-06-19 19:18:47 +02:00
Eugen Rochko 2dbf6bc5ad Add e-mail-based sign in challenge for users with disabled 2FA (#14013) 2020-06-09 10:23:06 +02:00
Takeshi Umeda 654c13ecfb Add limit parameter to rss (#13743) 2020-06-09 00:18:47 +02:00
ThibG 8b552d6f0c Fix unpermitted operations on custom emojis leading to cryptic errors (#13951)
* Display appropriate error when performing unpermitted operation on custom emoji

Fixes #13897

* Remove links to custom emoji actions not performable by moderators
2020-06-05 15:23:27 +02:00
Eugen Rochko 9de5c70980 Fix wrong route helper in encrypted messages controller (#13952)
And add `created_at` to encrypted message serializer
2020-06-03 20:32:15 +02:00
ThibG 8d57f6c4b4 Fix account redirect confirmation message talking about moved followers (#13950)
Fixes #13949
2020-06-03 20:18:19 +02:00
Eugen Rochko b864e78db8 Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
ThibG 60bb6ac6f4 Fix webfinger returning wrong status code on malformed or missing param (#13759)
Fixes #13757
2020-05-14 23:28:06 +02:00
ThibG f48e191e8e Fix sr locale being selected over sr-Latn (#13693)
* Fix sr locale being selected over sr-Latn

* Update tests
2020-05-11 01:09:21 +02:00
Eugen Rochko fc7cad8b2d Add ability to remove identity proofs from account (#13682)
Fix #12613
2020-05-10 11:21:10 +02:00
Takeshi Umeda 861661100c Add remote only to public timeline (#13504)
* Add remote only to public timeline

* Fix code style
2020-05-10 10:36:18 +02:00
ThibG e9227d8c10 Remove confusing “You are already signed in.” flash message (#13547)
When attempting to access the log-in page while already logged in,
Devise's `require_no_authentication` kicks in and sets a flash
message “You are already signed in.”

In almost all cases, this also causes a redirect to /web, which
does not display or clear flash messages, thus leaving the message
to a potentially much later date, like for instance, accessing
/preferences several minutes after being redirected to /web.
2020-05-10 10:16:39 +02:00
ThibG 5479d33843 Fix own following/followers not showing muted users (#13614)
Fixes #13612
2020-05-08 20:36:34 +02:00
Eugen Rochko e80d4479c7 Add more ActivityPub controller tests (#13590) 2020-05-03 22:19:24 +02:00
Eugen Rochko 0ca06d0ba9 Add more tests for ActivityPub controllers (#13585) 2020-05-03 16:30:36 +02:00
sternenseemann 674005c08e Allow users to delete their header and avatar (#13234)
This is achieved by sending a DELETE request to
/settings/profile/pictures/{avatar,header} via a link that is part of
the upload form's hint of the respective picture.
2020-04-20 14:03:03 +02:00
Takeshi Umeda f5606cdab4 Add local only to hashtag timeline (#13502) 2020-04-18 21:52:39 +02:00
Eugen Rochko 2d3219549b Change delivery failure tracking to work with hostnames instead of URLs (#13437) 2020-04-15 20:33:24 +02:00
Eugen Rochko 50fa554899 Add rate limit for reporting (#13390) 2020-04-05 14:40:08 +02:00
Eugen Rochko 6932e0e2af Add ability to filter audit log in admin UI (#13381) 2020-04-03 13:06:34 +02:00
Takeshi Umeda 1c2a286244 Fix ImportsController param to permit :mode (#13347) 2020-03-31 12:43:42 +02:00
ThibG 1a993f9675 Fix 404 and 410 API errors being silently discarded in WebUI (#13279)
* Fix 404 and 410 API errors being silently discarded in WebUI

Fixes #13278

* Return more appropriate error when user replies to a deleted toot

* Please CodeClimate

* Fix 404/410 errors on fetching account timelines & identity proofs

* Refactor error handling

* Move error message string to statuses.errors
2020-03-28 17:59:45 +01:00
Eugen Rochko 9ced8b6dd5 Add option to include resolved DNS records when blacklisting e-mail domains in admin UI (#13254)
* Add shortcuts to blacklist a user's e-mail domain in admin UI

* Add option to blacklist resolved MX and IP records for e-mail domains
2020-03-12 22:35:20 +01:00
Eugen Rochko da9d81c4ac Add titles to warning presets in admin UI (#13252) 2020-03-12 17:57:59 +01:00
ThibG cd7c22da32 Add federation support for the "hide network" preference (#11673)
* Change ActivityPub follower/following collections to not link first page

* Add support for hiding followers and following of remote users

* Switch to using a single `hide_collections` column

* Address code style remarks
2020-03-09 00:10:29 +01:00
Eugen Rochko f459919552 Change local media attachments to perform heavy processing asynchronously (#13210)
Fix #9106
2020-03-08 23:56:18 +01:00
ThibG cb93be3b88 Add ability to delete files uploaded for settings in admin UI (#13192)
* Allow deleting site uploads

* Refactor and move links into hints

* Fix i18n tests

* Fix HTML output of site_upload_delete_hint
2020-03-08 16:00:24 +01:00
Eugen Rochko a4668d994b Add specific rate limits for posting and following (#13172) 2020-03-08 15:17:39 +01:00
Eugen Rochko 9619521459 Remove useless `respond_to` calls (#13208) 2020-03-06 01:29:38 +01:00
Eugen Rochko e0db9f37f5 Fix leak of arbitrary statuses through unfavourite action in REST API (#13161) 2020-02-27 12:32:54 +01:00
ThibG c004399975 Fix dismissing an announcement twice raising an obscure error (#13124) 2020-02-24 22:21:40 +01:00
ThibG 8e8d9b3727 Fix account JSON/RSS not being cacheable due to wrong mime type comparison (#13116)
`request.format` is not a symbol but a `Mime::Type`, so the condition actually
never matched, and a session was created even for those requests, preventing
caching.
2020-02-19 22:31:53 +01:00
ThibG ddd9bad7f1 Fix sign-ups without checked user agreement being accepted through the web form (#13088)
* Fix user agreement not being verified

* Fix tests

* Fix up agreement field being dismissed
2020-02-16 12:56:53 +01:00
Eugen Rochko f1317633b5 Fix unfiltered params error when generating ActivityPub tag pagination (#13049) 2020-02-08 17:29:40 +01:00
Eugen Rochko bf4fb26b9d Fix malformed HTML causing uncaught error (#13042)
Fix OEmbed preview API leaking existence of private statuses (see #12930)
2020-02-07 15:24:22 +01:00
Eugen Rochko 5c4b1db54a Change signature verification to ignore signatures with invalid host (#13033)
Instead of returning a signature verification error, pretend there
was no signature (i.e., this does not allow access to resources that
need a valid signature), so public resources can still be fetched

Fix #13011
2020-02-03 17:48:23 +01:00
ThibG 99cf92a3cf Change how unread announcements are handled (#13020)
* Change meaning of /api/v1/announcements/:id/dismiss to mark an announcement as read

* Change how unread announcements are counted in UI

* Add unread marker to announcements and mark announcements as unread as they are displayed

* Fixups
2020-02-03 01:53:09 +01:00
Eugen Rochko 9fb8165712 Add publish/unpublish controls to announcements in admin UI (#12967) 2020-01-27 11:05:33 +01:00
Eugen Rochko dd68e44cfa Add streaming API updates for announcements being modified or deleted (#12963)
Change `all_day` to be a visual client-side cue only

Publish immediately if `scheduled_at` is in the past

Add `published_at` and `updated_at` to announcements JSON
2020-01-26 20:07:26 +01:00
Eugen Rochko 669f1f5e7f Fix OEmbed leaking information about existence of non-public statuses (#12930) 2020-01-24 00:20:51 +01:00
Eugen Rochko e4aa4a1c28 Fix password change/reset not immediately invalidating other sessions (#12928)
While making browser requests in the other sessions after a password
change or reset does not allow you to be logged in and correctly
invalidates the session making the request, sessions have API tokens
associated with them, which can still be used until that session
is invalidated.

This is a security issue for accounts that were already compromised
some other way because it makes it harder to throw out the hijacker.
2020-01-24 00:20:38 +01:00
Eugen Rochko 36b1dd934d Fix relationships page not showing results in admin UI (#12934)
Follow-up to #12927
2020-01-24 00:20:23 +01:00
Eugen Rochko f95fd62bac Add announcements (#12662)
* Add announcements

Fix #11006

* Add reactions to announcements

* Add admin UI for announcements

* Add unit tests

* Fix issues

- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"

* Fix scheduler unpublishing announcements before they are due

* Fix filter params not being passed to announcements filter
2020-01-23 22:00:13 +01:00
Eugen Rochko 67172aa4f9 Change followers page to relationships page in admin UI (#12927)
Allow browsing and filtering all relationships instead of just
followers, unify the codebase with the user-facing relationship
manager, add ability to see who the user invited
2020-01-23 20:33:20 +01:00
Eugen Rochko 9ebfb23e87 Various fixes and improvements (#12878)
* Fix unused role routes being generated

* Remove unused JavaScript code

* Refactor filters code to be DRYer

* Fix `.count == 0` comparisons to `.empty?` in views

* Fix filters in views
2020-01-20 15:55:03 +01:00
Eugen Rochko 61320da638 Fix access to OEmbed endpoint in secure mode (#12864) 2020-01-14 08:52:32 +01:00
Eugen Rochko 413cb7e861 Fix base64-encoded file uploads not being possible (#12748)
Fix #3804, Fix #5776
2020-01-04 01:54:07 +01:00
Eugen Rochko e066de9a3c Fix missing authentication call in filters controller (#12746) 2020-01-03 05:29:08 +01:00
Eugen Rochko 91422d35ed Fix uncaught unknown format errors in host meta controller (#12747) 2020-01-03 05:28:56 +01:00
ThibG f0cca7a79f Hide blocked users from more places (#12733)
* Hide blocked, muted, and blocked-by users from toot favourite lists

* Hide blocked, muted, and blocked-by users from toot reblog lists

* Hide blocked, muted, and blocked-by users from followers/following (API)

* Fix tests

* Hide blocked, muted, and blocked-by users from followers/following on public pages
2019-12-31 00:55:32 +01:00
Eugen Rochko c3ef5d7628 Fix error when fetching followers/following from REST API when user has network hidden (#12716)
Fix #12510
2019-12-31 00:54:38 +01:00
ThibG 6b20921c88 Remove unused AccountRelationshipsPresenter call in public pages (#12734)
Those were used to show a “follow” or “unfollow” button on account grid on
public pages, but that got removed a while ago.
2019-12-30 19:13:02 +01:00
Eugen Rochko 49b6881379 Fix settings pages being cacheable by the browser (#12714)
Fix #12255
2019-12-30 04:38:30 +01:00
Eugen Rochko 7cca47919f Fix HTML error pages being returned when JSON is expected (#12713)
Fix #12509
See also #12214
2019-12-30 04:38:18 +01:00
Eugen Rochko af95252d4c Fix missing error templates for non-HTML requests (#12593) 2019-12-10 07:39:54 +01:00
Eugen Rochko 2eda06b768 Fix generic HTTP 500 error on duplicate records (#12563)
Fix #12551
Fix #12547
2019-12-06 22:40:06 +01:00
ThibG 60d26cabf0 Add follow_request notification type (#12198)
* Add follow_request notification type

The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.

Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.

Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.

* Show follow request link in single-column if there are pending requests, even if account isn't locked

* Push follow requests from notifications to the follow_requests list

* Offer to accept or reject follow request from the notification

* Redesign follow request notification
2019-12-01 17:25:29 +01:00
ThibG 4b0a6d79dd Add ability to filter reports by target account domain (#12154)
* Add ability to filter reports by target account domain

* Reword by_target_domain label
2019-11-30 19:53:58 +01:00
Eugen Rochko 5761622a1e Fix proofs API being inaccessible in secure mode (#12495) 2019-11-28 04:07:49 +01:00
Gomasy d13e680f74 Support min_id-based pagination for bookmarks (#12381)
* Support min_id-based pagination for bookmarks

* Fix spec
2019-11-17 17:09:41 +01:00
Jennifer Glauche 0aae35e310 make it not return http 400 when passing and empty source argument (#12259)
* make it not return http 400 when passing and empty source argument

* create a spec for the empty source hash bug

* compact checks for nil, empty? parameters

* use nil.blank? instead checking for nil
2019-11-16 19:02:09 +01:00
ThibG 517c1cd062 Add bookmarks (#7107)
* Add backend support for bookmarks

Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.

* Add spec for bookmark endpoints

* Add front-end support for bookmarks

* Introduce OAuth scopes for bookmarks

* Add bookmarks to archive takeout

* Fix migration

* Coding style fixes

* Fix rebase issue

* Update bookmarked_statuses to latest UI changes

* Update bookmark actions to properly reflect status changes in state

* Add bookmarks item to single-column layout

* Make active bookmarks red
2019-11-13 23:02:10 +01:00
Yamagishi Kazutoshi 2ede35e843 Change to always returns html document in error pages (#12214) 2019-11-13 22:53:05 +01:00
Faye Duxovni 6bca6dbc2d Add setting for whether to crop images in unexpanded toots (#12126) 2019-10-24 22:51:41 +02:00
ThibG 795144fbbb Fix incoming federation in whitelist mode (#12185)
… posting to the AP inbox required a logged-in local user…
2019-10-24 22:45:35 +02:00
Eugen Rochko 2e07a901c5 Fix attachment not being re-downloaded even if file is not stored (#12125)
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
2019-10-09 07:10:46 +02:00
Eugen Rochko 6b96a6dd15 Fix `GET /api/v1/instance` REST APIs being unavailable in secure mode (#12089) 2019-10-06 22:11:29 +02:00
Eugen Rochko c453888616 Fix performance of home feed regeneration (#12084)
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive

Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
2019-10-06 22:11:17 +02:00
Eugen Rochko 059287ad08 Add `reason` param to `POST /api/v1/accounts` REST API (#12064)
For approval-required registrations mode
2019-10-03 17:50:59 +02:00
ThibG be13fc919c Fix RSS caching (but disable localization) (#12054) 2019-10-02 18:30:33 +02:00
Eugen Rochko 28c19c66ba Fix featured hashtag URL being interpreted as media or with_replies (#12048)
Fix #12034
2019-10-02 04:53:17 +02:00
Eugen Rochko 51a72a802f Add a nodeinfo endpoint (#12002)
* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* Add some additional node info stuff

* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* expanding this to include federation info

* codeclimate feedback

* CC feedback

* using activeserializers seems like a good idea...

* get rid of draft 2.1 version

* Reimplement 2.1, also fix metaData -> metadata

* Fix metaData -> metadata here too

* Fix nodeinfo 2.1 tests

* Implement cache for monthly user aggregate

* Useless

* Remove ostatus from the list of supported protocols

* Fix nodeinfo's open_registration reading obsolete setting variable

* Only serialize domain blocks with user-facing limitations

* Do not needlessly list noop severity in nodeinfo

* Only serialize domain blocks info in nodeinfo when they are set to be displayed to everyone

* Enable caching for nodeinfo endpoints

* Fix rendering nodeinfo

* CodeClimate fixes

* Please CodeClimate

* Change InstancePresenter#active_user_count_months for clarity

* Refactor NodeInfoSerializer#metadata

* Remove nodeinfo 2.1 support as the schema doesn't exist

* Clean-up
2019-09-29 21:31:51 +02:00
Eugen Rochko ae03161ad9 Fix account migration not affecting followers on origin server (#11980) 2019-09-29 16:23:13 +02:00
Eugen Rochko f5c71a6cd2 Add (back) option to set redirect notice on account without moving followers (#11994)
Fix #11913
2019-09-29 05:03:19 +02:00
Eugen Rochko 561b0509c3 Fix redirecting non-functional accounts on public pages (#11978)
Fix #11969
2019-09-28 01:33:27 +02:00
Eugen Rochko 66d3e13777 Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
2019-09-28 01:02:21 +02:00
ThibG 70990720c5 Change silences to always require approval on follow (#11975)
* Change silenced accounts to require approval on follow

* Also require approval for follows by people explicitly muted by target accounts

* Do not auto-accept silenced or muted accounts when switching from locked to unlocked

* Add `follow_requests_count` to verify_credentials

* Show “Follow requests” menu item if needed even if account is locked

* Add tests

* Correctly reflect that follow requests weren't auto-accepted when local account is silenced

* Accept follow requests from user-muted accounts to avoid leaking mutes
2019-09-27 21:13:51 +02:00
abcang 4c0da93bc7 Improve status pin query (#11972) 2019-09-27 15:23:30 +02:00
Eugen Rochko 67796a267d Fix relays UI being available in whitelist/secure mode (#11963)
Fix relays UI referencing relay that is not functional
2019-09-27 02:13:34 +02:00
Eugen Rochko 3773115066 Fix authentication before 2FA challenge (#11943)
Regression from #11831
2019-09-24 04:35:36 +02:00
Takeshi Umeda 45fe523a18 Addition of update activity distribution by alias, minor correction (#11905)
* Addition of update activity distribution by alias, minor correction

* Distribute Update activity after adding alias
* Add uniqueness verification to alias uri
* accept acct starting with @

* fix double-quoted to single-quoted
2019-09-21 09:11:21 +02:00
Takeshi Umeda 74e4abcebc Fixed an error in the aliases template of the aliases controller (#11902) 2019-09-21 02:59:37 +02:00
Eugen Rochko a2d3728c36 Add account migration UI (#11846)
Fix #10736

- Change data export to be available for non-functional accounts
- Change non-functional accounts to include redirecting accounts
2019-09-19 20:58:19 +02:00
Eugen Rochko 61442032a2 Add table of contents to about page (#11885)
Move public domain blocks information to about page
2019-09-19 11:09:05 +02:00
Eugen Rochko 1781358bd9 Add password challenge to 2FA settings, e-mail notifications (#11878)
Fix #3961
2019-09-18 16:37:27 +02:00
Eugen Rochko 8904a4eb36 Fix TOTP codes not being filtered from logs during enabling/disabling (#11877)
Not a serious issue because they are meaningless past single use
2019-09-18 02:48:40 +02:00
Eugen Rochko ae443cb6ea Fix webfinger response not returning 410 when account is suspended (#11869) 2019-09-17 14:58:02 +02:00
mayaeh 566e85716f Add search and sort functions to hashtag admin UI (#11829)
* Add search and sort functions to hashtag admin UI

* Move scope processing from tags_controller to tag_filter

* Fix based on method naming conventions

* Fixed not to get 500 errors for invalid requests
2019-09-16 14:27:29 +02:00
Eugen Rochko 8eb0d880cb Fix 2FA challenge and password challenge for non-database users (#11831)
* Fix 2FA challenge not appearing for non-database users

Fix #11685

* Fix account deletion not working when using external login

Fix #11691
2019-09-15 21:08:39 +02:00
Eugen Rochko 8824964836 Remove deprecated `GET /api/v1/search` API (#11823)
Use `GET /api/v2/search` instead
2019-09-13 16:11:13 +02:00
ThibG 7d2fceae9b Change /api/v1/timelines/public to require auth when public preview is off (#11802)
Fixes #11289
2019-09-13 16:03:46 +02:00
Eugen Rochko ec20fd2112 Change unlisted custom emoji to not appear in autosuggestions (#11818)
Fix #11669
2019-09-13 16:01:09 +02:00
Eugen Rochko 605a4e654a Fix uncaught errors in media proxy controller (#11811) 2019-09-12 01:51:12 +02:00
Eugen Rochko 4291b74031 Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
ThibG 875d2e2b59 Add updated relationship to follow request API responses (#11800)
Fixes #11747
2019-09-10 20:56:42 +02:00
Eugen Rochko 25fb124ee6 Add batch actions and categories to admin UI for custom emojis (#11793) 2019-09-09 22:44:17 +02:00
Eugen Rochko 7799c7c75f Add batch approve/reject for pending hashtags in admin UI (#11791) 2019-09-09 12:50:09 +02:00
Takeshi Umeda 50c2f0dcc1 Add featured tags API (#11778)
* Add featured tags API

* Remove show and update, change scope, fix code style
2019-09-09 10:50:33 +02:00
Yamagishi Kazutoshi 7544167f3c Add healthcheck endpoint for web (#11770) 2019-09-07 02:47:51 +02:00
Eugen Rochko 89989b6255 Add timeline read markers API (#11762)
Fix #4093
2019-09-06 13:55:51 +02:00
Eugen Rochko 09fe562dd6 Fix wrong variable regression from #11753 (#11763) 2019-09-05 06:13:50 +02:00
Eugen Rochko 7ef93513f7 Change account deletion page to have better explanations (#11753)
Fix deletion of unconfirmed account not freeing up the username

Add prefill of logged-in user's email in the reconfirmation form
2019-09-04 04:13:54 +02:00
Eugen Rochko 436266ca27 Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
Eugen Rochko 76bdc01aee Fix wrong percentages in admin UI for hashtag usage breakdown (#11714) 2019-09-01 19:44:05 +02:00
Eugen Rochko b47119eb61 Change layout of public profile directory to be the same as in web UI (#11705) 2019-08-30 07:41:16 +02:00
Eugen Rochko 73d8f314a9 Fix error in REST API for an account's statuses (#11700) 2019-08-30 02:49:44 +02:00
Eugen Rochko a2a65f43b2 Fix uncaught error when resource param is missing in Webfinger request (#11701) 2019-08-30 02:49:33 +02:00
Eugen Rochko ee9d0c0f38 Fix uncaught domain normalization error in remote follow (#11703) 2019-08-30 02:19:17 +02:00
Eugen Rochko f21e27e914 Fix uncaught parameter missing exceptions and missing error templates (#11702) 2019-08-30 01:34:47 +02:00
Eugen Rochko 79922ae20f Add profile directory to web UI (#11688)
* Add profile directory to web UI

* Add a line of bio to the directory
2019-08-30 00:14:36 +02:00
Eugen Rochko 9959cdc79f Add option to include reported statuses in warning e-mail (#11639) 2019-08-23 22:37:23 +02:00
Eugen Rochko 32c781ec6b Add soft delete for statuses for instant deletes through API (#11623)
* Add soft delete for statuses to allow them to appear instant

* Allow reporting soft-deleted statuses and show them in the admin UI

* Change index for getting an account's statuses
2019-08-22 21:55:56 +02:00
Eugen Rochko e4605bbb4a Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
2019-08-22 04:17:12 +02:00
ThibG f17b0ff712 Add invite comments (#10465) 2019-08-19 11:40:42 +02:00
ThibG 867b9f3e9c Add public blocks to /about/blocks (#11298)
* Add automatic blocklist display in /about/blocks

Inspired by https://github.com/Gargron/mastodon.social-misc

* Add admin option to set who can see instance blocks

* Normalize locales files

* Rename “Sandbox” to “Silence” for consistency

* Disable /about/blocks when in whitelist mode

* Optionally display rationale for domain blocks

* Only display domain blocks that have user-facing limitations, and order them

* Redesign table of blocked domains to better handle long domain names and rationales

* Change domain blocks ordering now that rationales aren't displayed right away

* Only show explanation for block severities actually in use

* Reword instance block explanations and add disclaimer for public fetch mode
2019-08-19 11:35:48 +02:00
Takeshi Umeda 363c6e8f52 Add an RSS feed tagged to a public profile page (#10502)
* Add featured tag support to rss feed on public account page

* fix codeing style
2019-08-18 20:54:36 +02:00
Eugen Rochko dde2c17e12 Fix uncaught 422 and 500 errors (#11590) 2019-08-18 18:04:18 +02:00
Eugen Rochko dc20e1af71 Add HTTP signature keyId to request log (#11591) 2019-08-18 18:03:56 +02:00
Eugen Rochko 8e76c759c1 Add explanation to featured hashtags page and profile (#11586) 2019-08-17 18:07:52 +02:00
ThibG 404b64f5ca Fix multiple issues with replies collection for pages further than self-replies (#11582)
* Fix the replies collection returning snowflakes ids rather than URIs

Fixes #11568

* Fix min_id in replies queries once self-replies are exhausted

* Fix `next` attribute of replies collection being nil when there are no self-replies

* Rename other_accounts param to only_other_accounts
2019-08-16 22:58:51 +02:00
Eugen Rochko a8d983934e Fix blurhash and autoplay not working on public pages (#11585) 2019-08-16 19:15:05 +02:00
Eugen Rochko 88ef061da1 Fix 422 being returned instead of 404 when POSTing (#11574) 2019-08-16 02:08:35 +02:00
ThibG cfc3fc6bcd Fix reverse-proxy caching of instance actor object (#11561) 2019-08-13 15:30:37 +02:00
ThibG 93e6461e66 Fix more ActivityPub queries setting cookies and preventing caching (#11557) 2019-08-12 22:26:07 +02:00
ThibG bfa0b76a6c Fix ActivityPub and REST API queries setting cookies and preventing caching (#11539)
Regression from #8657
2019-08-11 22:59:40 +02:00
Eugen Rochko 24a5f99d10 Fix pinned statuses API returning pagination headers (#11526)
Fix #10227
2019-08-08 23:04:04 +02:00
Eugen Rochko 85466a2b9d Fix weekly usage not being displayed correctly in hashtag admin UI (#11524)
Fix percentages in usage breakdown having too many digits

Change trending hashtags to only ask for review if a hashtag enters
the top 3 position in the set, since it's the only items shown in
the default web UI
2019-08-08 23:03:28 +02:00
Eugen Rochko 10b486f627 Fix acct URIs with IDN domains not being resolved (#11520)
Fix #11494
2019-08-07 21:14:08 +02:00
Eugen Rochko cc1e24ddc7 Add breakdown of usage by source to admin UI for hashtags (#11517)
Allows determining where the majority of posts in a hashtag come
from on a given day at a glance.
2019-08-07 20:20:39 +02:00
ThibG 072e696aa1 Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Eugen Rochko 25f549557b Fix trending tags returning less items than requested sometimes (#11513)
Add better sorting defaults to the hashtags admin UI

Add "not reviewed" filter to hashtags admin UI
2019-08-07 17:08:30 +02:00
Eugen Rochko bfb43d7104 Add number of pending accounts and pending hashtags to admin dashboard (#11514) 2019-08-07 16:13:34 +02:00
Eugen Rochko fd352ec288 Fix account tags not being saved correctly (#11507)
* Fix account tags not being saved correctly

Regression from b838607

Fix Tag#discoverable not returning tags where listable is nil instead of true

Add notice when saving hashtags in admin UI

Change public hashtag and directory pages to return 404 for forbidden tags

* Remove unused locale string
2019-08-07 10:01:55 +02:00
Eugen Rochko 6362ac6617 Fix admin dashboard missing latest features (#11505)
Fix redis-namespace deprecation warning about administrative commands
2019-08-06 19:40:06 +02:00
Eugen Rochko 191392e5c3 Add trends UI with admin and user settings (#11502) 2019-08-06 17:57:52 +02:00
Eugen Rochko 05e9cd13eb Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
2019-08-05 19:54:29 +02:00
Eugen Rochko 0ed96e9aee Remove XML version of Webfinger and remove links to Atom feeds (#11460)
Fix #11453
2019-08-01 19:14:02 +02:00
Eugen Rochko 825dc3ca22 Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
ThibG 52cda33b82 Fix invites not being disabled upon account suspension (#11412)
* Disable invite links from disabled/suspended users

* Add has_many invites relationship to users

* Destroy unused invites when suspending an account
2019-07-26 18:55:33 +02:00
dependabot-preview[bot] 884e20c028 Bump active_model_serializers from 0.10.9 to 0.10.10 (#11311)
* Bump active_model_serializers from 0.10.9 to 0.10.10

Bumps [active_model_serializers](https://github.com/rails-api/active_model_serializers) from 0.10.9 to 0.10.10.
- [Release notes](https://github.com/rails-api/active_model_serializers/releases)
- [Changelog](https://github.com/rails-api/active_model_serializers/blob/v0.10.10/CHANGELOG.md)
- [Commits](https://github.com/rails-api/active_model_serializers/compare/v0.10.9...v0.10.10)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Add root option to render method
2019-07-23 11:10:42 +02:00
Eugen Rochko 6be7b414e2 Change unconfirmed user login behaviour (#11375)
Allow access to account settings, 2FA, authorized applications, and
account deletions to unconfirmed and pending users, as well as
users who had their accounts disabled. Suspended users cannot update
their e-mail or password or delete their account.

Display account status on account settings page, for example, when
an account is frozen, limited, unconfirmed or pending review.

After sign up, login users straight away and show a simple page that
tells them the status of their account with links to account settings
and logout, to reduce onboarding friction and allow users to correct
wrongly typed e-mail addresses.

Move the final sign-up step of SSO integrations to be the same
as above to reduce code duplication.
2019-07-22 10:48:50 +02:00
Eugen Rochko 79b9eee938 Add (back) rails-level JSON caching (#11333) 2019-07-21 22:32:16 +02:00
Eugen Rochko 2b4fa0d6fa Change locale detection to run once per session (#8657)
Fix #6462
2019-07-21 18:08:02 +02:00
ThibG 47dfcc601b Fix some flash notices/alerts staying on unrelated pages (#11364) 2019-07-19 23:13:21 +02:00
ThibG c2126e3f98 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
2019-07-19 01:44:42 +02:00
ThibG 9015b19e09 Add setting to disable the anti-spam (#11296)
* Add environment variable to disable the anti-spam

* Move antispam setting to admin settings

* Fix typo

* antispam → spam_check
2019-07-17 21:09:15 +02:00
Eugen Rochko 50caff4dc0 Change terms and privacy policy pages to always be accessible (#11334)
Fix #11328
2019-07-17 19:29:37 +02:00
ThibG 978792a112 Fix custom CSS controller (#11336) 2019-07-17 17:14:25 +02:00
ThibG a4b1083795 Fix caching headers in ActivityPub endpoints (#11331)
* Fix reverse-proxy caching in public fetch mode

* Fix caching in ActivityPub-specific controllers
2019-07-17 00:00:39 +02:00
Eugen Rochko 9dead08055 Add option to disable real-time updates in web UI (#9984)
Fix #9031
Fix #7913
2019-07-16 06:30:47 +02:00
Eugen Rochko 39719ae981 Add ActivityPub secure mode (#11269)
* Add HTTP signature requirement for served ActivityPub resources

* Change `SECURE_MODE` to `AUTHORIZED_FETCH`

* Add 'Signature' to 'Vary' header and improve code style

* Improve code style by adding `public_fetch_mode?` method
2019-07-11 20:11:09 +02:00
Eugen Rochko d0b0b63b1a Refactor domain block checks (#11268) 2019-07-09 03:27:35 +02:00
Eugen Rochko e17c937f65 Remove unused remote unfollow controller (#11250) 2019-07-08 12:04:06 +02:00
Eugen Rochko 56f0203c66 Refactor controllers for statuses, accounts, and more (#11249) 2019-07-08 12:03:45 +02:00
Eugen Rochko fbbcbd940d Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247) 2019-07-07 16:16:51 +02:00
Eugen Rochko 4931208dd8 Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Eugen Rochko d9f42ba34b Remove deprecated REST API `GET /api/v1/statuses/:id/card` (#11213) 2019-07-05 02:15:24 +02:00
Eugen Rochko 6cae1b40bf Remove deprecated REST API `GET /api/v1/timelines/direct` (#11212) 2019-07-05 02:14:56 +02:00
Eugen Rochko 1c612f24e4 Add categories for custom emojis (#11196)
Fix #7940
2019-06-28 15:54:10 +02:00