Commit Graph

1639 Commits (9ea7cc70a3611395f875d54b9f4fe24c3ca5b55f)

Author SHA1 Message Date
Thibaut Girka 7a84a84fc1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
  Upstream changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/models/user.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc not preventing moved accounts from logging
  in.
  Ported upstream changes while keeping the ability for moved accounts to log
  in.
- `app/policies/status_policy.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's ability  to hide followers count.
  Ported upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `package.json`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
2020-09-28 14:13:30 +02:00
Eugen Rochko 02c4c91faa Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
kawaguchi f13d563cb7 Fix validates :sign_count of WebauthnCredential (#14806) 2020-09-16 20:16:46 +02:00
Eugen Rochko e514304a76 Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
ThibG 739f7815ef Do not serve account actors at all in limited federation mode (#14800)
* Do not serve account actors at all in limited federation mode

When an account is fetched without a signature from an allowed instance,
return an error.

This isn't really an improvement in security, as the only information that was
previously returned was required protocol-level info, and the only personal bit
was the existence of the account. The existence of the account can still be
checked by issuing a webfinger query, as those are accepted without signatures.

However, this change makes it so that unallowed instances won't create account
records on their end when they find a reference to an unknown account.

The previous behavior of rendering a limited list of fields, instead of not
rendering the actor at all, was in order to prevent situations in which two
instances in Authorized Fetch mode or Limited Federation mode would fail to
reach each other because resolving an account would require a signed query…
from an account which can only be fetched with a signed query itself. However,
this should now be fine as fetching accounts is done by signing on behalf of
the special instance actor, which does not require any kind of valid signature
to be fetched.

* Fix tests
2020-09-14 13:04:29 +02:00
Eugen Rochko 403aeeb7e0 Refactor settings controllers (#14767)
- Disallow suspended accounts from revoking sessions and apps
- Allow suspended accounts to access exports
2020-09-11 20:56:35 +02:00
Thibaut Girka 04f2c25e5d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb

All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
2020-09-08 16:26:47 +02:00
Eugen Rochko 38aa7a9a7f Refactor feed manager (#14761) 2020-09-08 03:41:16 +02:00
ThibG e8aa85dc03 Fix multiple boosts of a same toot erroneously appearing in TL (#14759)
* Check for and record reblog info atomically

Instead of using ZREVRANK to determine whether a reblog is a new reblog or not,
use ZADD's NX option to perform the check/addition option atomically.

* Replace ZREVRANK call with ZSCORE key which is more efficient

* Make tests a bit stricter

* Fix off-by-one
2020-09-07 18:00:15 +02:00
Eugen Rochko 78ceda1af6 Refactor how public and tag timelines are queried (#14728) 2020-09-07 11:02:04 +02:00
Thibaut Girka 612158f4f6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict caused because we have additional code to make sure pinned
  local-only toots don't get rendered on the ActivityPub endpoints.
  Ported upstream changes.
2020-09-07 09:21:38 +02:00
Eugen Rochko 89fd56627c Fix various warnings in rspec (#14729) 2020-09-04 20:22:26 +02:00
ThibG 50d0c1e95f Add configuration option to filter replies in lists (#9205)
* Add database support for list show-reply preferences

* Add backend support to read and update list-specific show_replies settings

* Add basic UI to set list replies setting

* Add specs for list replies policy

* Switch "cycling" reply policy link to a set of radio inputs

* Capitalize replies_policy strings

* Change radio button design to be consistent with that of the directory explorer
2020-09-01 13:31:28 +02:00
Thibaut Girka 37e746eab6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/accounts_controller.rb`:
  Upstream change too close to a glitch-soc change related to
  instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Minor conflict due to glitch-soc's handling of Direct Messages,
  merged upstream changes.
- `yarn.lock`:
  Not really a conflict, caused by glitch-soc-only dependencies
  being textually too close to updated upstream dependencies.
  Merged upstream changes.
2020-08-30 16:13:08 +02:00
Eugen Rochko e8d1656bd6 Add support for dereferencing objects through bearcaps (#14683) 2020-08-30 12:34:20 +02:00
santiagorodriguez96 1a79d5c68e refactor: add email previews for WebAuthn emails (#14658)
This is a leftover for the work done in #14466.
2020-08-25 01:21:11 +02:00
santiagorodriguez96 f142983484 Add WebAuthn as an alternative 2FA method (#14466)
* feat: add possibility of adding WebAuthn security keys to use as 2FA

This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add request for WebAuthn as second factor at login if enabled

This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add possibility of deleting WebAuthn Credentials

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: disable WebAuthn when an Admin disables 2FA for a user

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA

Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.

* We had to change a little the flow for setting up TOTP, given that now
  it's possible to setting up again if you already had TOTP, in order to
  let users modify their authenticator app – given that now it's not
  possible for them to disable TOTP and set it up again with another
  authenticator app.
  So, basically, now instead of storing the new `otp_secret` in the
  user, we store it in the session until the process of set up is
  finished.
  This was because, as it was before, when users clicked on 'Edit' in
  the new two-factor methods lists page, but then went back without
  finishing the flow, their `otp_secret` had been changed therefore
  invalidating their previous authenticator app, making them unable to
  log in again using TOTP.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* refactor: fix eslint errors

The PR build was failing given that linting returning some errors.
This commit attempts to fix them.

* refactor: normalize i18n translations

The build was failing given that i18n translations files were not
normalized.
This commits fixes that.

* refactor: avoid having the webauthn gem locked to a specific version

* refactor: use symbols for routes without '/'

* refactor: avoid sending webauthn disabled email when 2FA is disabled

When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.

* refactor: avoid creating new env variable for webauthn_origin config

* refactor: improve flash error messages for webauthn pages

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
2020-08-24 16:46:27 +02:00
ThibG 4468fce63c Add support for inlined objects in activity audience (#14514)
* Add support for inlined objects in activity audience

* Add tests
2020-08-24 14:11:47 +02:00
Thibaut Girka f800c99b60 Merge branch 'master' into glitch-soc/merge-upstream 2020-08-19 19:16:41 +02:00
ThibG 4e292c23d6 Fix not being able to unbookmark toots when blocked by their author (#14604)
* Fix not being able to unbookmark toots when blocked by their author

* Add tests
2020-08-19 19:02:06 +02:00
Thibaut Girka c32625495f Merge branch 'master' into glitch-soc/merge-upstream 2020-08-13 22:17:29 +02:00
ThibG 8baad22f68 Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
Fire Demon 5171ba9a81 Add tests for direct message assertions 2020-08-04 19:43:19 +02:00
Thibaut Girka c2eb2a9d16 Merge branch 'master' into glitch-soc/merge-upstream 2020-08-02 14:30:28 +02:00
ThibG 0dd641dcea Change content-type to be always computed from file data (#14452)
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
2020-08-02 11:21:10 +02:00
Thibaut Girka 0afe3c1562 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `yarn.lock`:
  No real conflict, just a glitch-soc-only dependency textually too close to
  one updated upstream. Ported upstream changes.
2020-08-01 20:24:23 +02:00
ThibG d6f2d1d3f2 Fix handling of Reject Follow when a matching follow relationship exists (#14479)
* Add tests

* Fix handling of Reject Follow when a matching follow relationship exists

Regression from #12199
2020-08-01 18:20:37 +02:00
Thibaut Girka 714a3fcb3b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
  Not a “real” conflict, but change too close to a change we made to
  fix the vanilla WebUI locally pushing authored local-only toots in the
  public TL view.
2020-07-22 13:38:17 +02:00
ThibG e041b8b3f8 Fix not handling Undo on some activity types when they aren't inlined (#14346)
* Fix not handling Undo on some activity types when they aren't inlined

When receiving an Undo for a non-inlined activity, try looking it up in
database using the URI. The queries are ad-hoc because we don't have a global
index of object URIs, and not all activity types are stored in database with
an index on their URI.

Announces are just statuses, and have an index on URIs, so this check can
be done efficiently.

Accepts cannot be handled at all because we don't record their URI at any
point.

Follows don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

Likes don't have an index on URI, they have an index on the issuing account,
but the number of favs per account may be very high, so I decided not to
handle that.

Blocks don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

In all cases, if an Undo could not be handled properly, we call `delete_later!`
because that does not require us to know more than the URI of the undone
property.

* Add tests

* Make newer blocks overwrite older ones

Allows re-synchronizing block info by re-blocking and un-blocking again
when the original Undo Block has been lost.
2020-07-22 11:45:35 +02:00
ThibG 7792bab1cd Fix RSS feeds not being cachable (#14368)
* Add tests for some cachable responses

This only covers responses that we should have managed to make cachable
so far. It's not the case of all responses that should be cachable in
the end.

* Fix RSS feeds not being cachable
2020-07-22 11:44:02 +02:00
ThibG a9e58aa8c6 Fix boosted toots from blocked account not being retroactively removed from TL (#14339)
* Fix boosted toots from blocked account not being retroactively removed from TL

Fixes #14301

* Add test for clear_from_timeline
2020-07-17 07:07:54 +02:00
Thibaut Girka 67657d8361 Merge branch 'master' into glitch-soc/merge-upstream 2020-07-16 14:43:50 +02:00
ThibG 109ea6f860 Fix removing allowed domains being done synchronously (#14302)
* Fix removing allowed domains being done synchronously

* Add tests
2020-07-15 21:08:19 +02:00
Thibaut Girka 654fd69903 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `config/routes.rb`:
  Upstream disabled E2EE routes, which we did earlier, but slightly
  differently. Took upstream's version.
2020-07-15 15:32:40 +02:00
ThibG c9fbf47a23 Fix being unable to unboost when blocked by their author (#14308)
Fixes #14307
2020-07-15 14:43:19 +02:00
Thibaut Girka c2347f6cf6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, just some glitch-soc-specific dependency
  too close to an upstream-updated one.
2020-07-07 15:58:45 +02:00
Eugen Rochko 0edde9d6c8 Fix media attachments enumeration (#14254)
* Fix media attachment enumeration

* Switch media_attachments id to snowflake ids

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-07-07 15:26:51 +02:00
ThibG bbfafea10a Add test for removing endorsed accounts on account deletion/suspension (#14241) 2020-07-07 02:01:13 +02:00
Thibaut Girka 928a3a9fd5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/public.js`:
  Conflict because part of that file has been split to
  `app/javascript/core/settings.js`. Ported those changes
  there.
2020-07-01 19:23:14 +02:00
ThibG 3e51b04d40 Change move handler to carry blocks over (#14144)
* Change move handler to carry blocks and mutes over

When user A blocks user B and B moves to a new account C, make A block C
accordingly.

Note that it only works if A's instance is aware of the Move, that is,
if B is on A's instance or has followers there.

* Also notify instances with known people blocking you when moving

* Add automatic account notes when blocking/muting an account that had no note
2020-07-01 13:51:15 +02:00
Thibaut Girka c28f80802a Add support for “start” and “reversed” attributes to ol element
Fixes #1367
2020-07-01 01:02:27 +02:00
Eugen Rochko 36e5fd547b Fix remote files not using Content-Type header, streaming (#14184) 2020-06-30 23:58:02 +02:00
Thibaut Girka 2a2af880b8 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream deleted it, I decided to keep it.
- `package.json`:
  Upstream updated a dependency textually too close to wavesurfer.js
  which has been deleted from upstream but is kept in glitch-soc for now.
2020-06-30 23:00:20 +02:00
ThibG 3afc7977b1 Add user notes on accounts (#14148)
* Add UserNote model

* Add UI for user notes

* Put comment in relationships entity

* Add API to create user notes

* Copy user notes to new account when receiving a Move activity

* Address some of the review remarks

* Replace modal by inline edition

* Please CodeClimate

* Button design changes

* Change design again

* Cancel note edition when pressing Escape

* Fixes

* Tweak design again

* Move “Add note” item, and allow users to add notes to themselves

* Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”
2020-06-30 19:19:50 +02:00
Eugen Rochko 16a8db0958 Fix trying to write non-existent image remote URL attribute on preview cards (#14181)
Regression from #14145
2020-06-29 17:59:04 +02:00
Eugen Rochko a3ce01a102 Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
Thibaut Girka 83dc54c487 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict due to glitch-soc having to take care of local-only
  pinned toots in that controller.
  Took upstream's changes and restored the local-only special
  handling.
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflicts due to the theming system, applied upstream
  changes, adapted the following two files for glitch-soc's
  theming system:
  - `app/controllers/concerns/sign_in_token_authentication_concern.rb`
  - `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
  Minor conflict due to glitch-soc having to handle local-only
  toots specially. Applied upstream changes and restored
  the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
  Minor conflict due to the theming system.
- `package.json`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
- `yarn.lock`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
2020-06-09 10:39:20 +02:00
ThibG a7400a4f97 Make domain block/silence/reject-media code more robust (#13424)
* Split media cleanup from reject-media domain blocks to its own service

* Slightly improve ClearDomainMediaService error handling

* Lower DomainClearMediaWorker to lowest-priority queue

* Do not catch ActiveRecord::RecordNotFound in domain block workers

* Fix DomainBlockWorker spec labels

* Add some specs

* Change domain blocks to immediately mark accounts as suspended

Rather than doing so sequentially, account after account, while cleaning
their data. This doesn't change much about the time the block takes to
complete, but it immediately prevents interaction with the blocked domain,
while up to now, it would only be guaranteed when the process ends.
2020-06-09 10:32:00 +02:00
Eugen Rochko 2dbf6bc5ad Add e-mail-based sign in challenge for users with disabled 2FA (#14013) 2020-06-09 10:23:06 +02:00
ThibG e67a55ee49 Fix serialization of replies when some of them are URIs (#13957)
* Fix serialization of replies when some of them are URIs

Fixes #13956

* Add test
2020-06-04 19:03:31 +02:00
Eugen Rochko b864e78db8 Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
Ben Lubar 0584eaeaa1 Fix exception when trying to serialize posts with <a> tags in them without hrefs (#1334)
* fix exception when trying to serialize posts with <a> tags in them without hrefs

* Add tests

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-05-28 12:47:40 +02:00
Thibaut Girka 365ccfb96a Merge branch 'master' into glitch-soc/merge-upstream 2020-05-20 18:22:42 +02:00
ThibG bb3aff5c8c Add support for `summary` field for media description (#13763) 2020-05-15 17:08:59 +02:00
ThibG 063940f489 Fix hashtag search performing account search as well (#13758) 2020-05-14 23:37:37 +02:00
ThibG 60bb6ac6f4 Fix webfinger returning wrong status code on malformed or missing param (#13759)
Fixes #13757
2020-05-14 23:28:06 +02:00
Thibaut Girka 4853a87df8 Merge branch 'master' into glitch-soc/merge-upstream 2020-05-13 22:11:49 +02:00
dependabot-preview[bot] e7e09d3b7b Bump doorkeeper from 5.3.3 to 5.4.0 (#13733)
* Bump doorkeeper from 5.3.3 to 5.4.0

Bumps [doorkeeper](https://github.com/doorkeeper-gem/doorkeeper) from 5.3.3 to 5.4.0.
- [Release notes](https://github.com/doorkeeper-gem/doorkeeper/releases)
- [Changelog](https://github.com/doorkeeper-gem/doorkeeper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/doorkeeper-gem/doorkeeper/compare/v5.3.3...v5.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix tests

* Fix use of Doorkeeper::AccessToken.find_or_create_for

* Fix tests?

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-05-12 15:25:33 +02:00
ThibG f48e191e8e Fix sr locale being selected over sr-Latn (#13693)
* Fix sr locale being selected over sr-Latn

* Update tests
2020-05-11 01:09:21 +02:00
Thibaut Girka 4e4e5316c1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, just a glitch-soc-only dependency too close to a
  dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
  Not a real conflict, just a change too close to glitch-soc-changed code
  for optionally showing boosts in public timelines.
  Applied upstream changes.
- `app/views/layouts/application.html.haml`:
  Upstream a new, static CSS file, conflict due to glitch-soc's theming
  system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
  Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
  both files are very different. Removed 'unsafe-inline' as well.
2020-05-10 16:19:56 +02:00
Eugen Rochko 3144f8dbbc Refactor monkey-patching of Goldfinger (#12561) 2020-05-10 11:41:43 +02:00
Eugen Rochko fc7cad8b2d Add ability to remove identity proofs from account (#13682)
Fix #12613
2020-05-10 11:21:10 +02:00
Takeshi Umeda 861661100c Add remote only to public timeline (#13504)
* Add remote only to public timeline

* Fix code style
2020-05-10 10:36:18 +02:00
ThibG 41536557be Improve RSS entries for statuses (#13592)
* Improve RSS entries for statuses

- Render polls in both accounts and tags serializers
- Refactor RSS serializers
- Change title preview to include ellipsis when truncated
- Change title preview to show CW instead of toot text
- Add tests

* Remove title from OEmbed serialization

Twitter doesn't serialize title either, and tihs allows us to move the
title formatting code to the RSS serializers.
2020-05-10 09:50:54 +02:00
ThibG 5479d33843 Fix own following/followers not showing muted users (#13614)
Fixes #13612
2020-05-08 20:36:34 +02:00
Taras Gogol 1a03de8592 Fix followings list order | Issue #13538 (#13676) 2020-05-08 20:17:16 +02:00
Eugen Rochko e80d4479c7 Add more ActivityPub controller tests (#13590) 2020-05-03 22:19:24 +02:00
Thibaut Girka dfa13deaa1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/statuses_controller.rb`:
  Upstream disabled the embed controller for reblogs.
  Not a real conflict, but glitch-soc has an extra line to deal
  with its theming system.
  Ported upstream changes.
- `app/javascript/packs/public.js`:
  Upstream made changes to get rid of most inline CSS, this changes
  javascript for public pages, which in glitch are split between
  different files. Ported those changes.
- `app/models/status.rb`:
  Upstream changed the block check in `Status#permitted_for` to
  include domain-block checks. Not a real conflict with glitch-soc,
  but our scope is slightly different, as our scope for
  unauthenticated access do not include instance-local toots.
  Ported upstream changes.
- `app/serializers/rest/instance_serializer.rb`:
  Not a real conflict, upstream added a new field to the instance
  serializer, the conflict is one line above since we added more of
  that.
  Ported upstream changes.
- `app/views/settings/profiles/show.html.haml`:
  Upstream got rid of most inline CSS and moved hidden elements
  to data attributes in the process, in fields were we have
  different values.
  Ported upstream changes while keeping our glitch-specific
  values.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream got rid of inline CSS on an HAML line we treat
  differently, stripping empty text nodes.
  Ported upstream changes to the style attribute, keeping
  the empty text node stripping behavior.
2020-05-03 21:23:49 +02:00
Yamagishi Kazutoshi cf55546c41 Revert "improve status title (#8596)" (#13591)
This reverts commit dff05cd398.
2020-05-03 18:48:13 +02:00
Eugen Rochko 0ca06d0ba9 Add more tests for ActivityPub controllers (#13585) 2020-05-03 16:30:36 +02:00
Eugen Rochko 96920a7b8e Fix not being able to resolve public resources in development environment (#13505) 2020-04-25 22:01:08 +02:00
Thibaut Girka 43a9ed3de8 Merge branch 'master' into glitch-soc/merge-upstream 2020-04-17 20:16:24 +02:00
Eugen Rochko 2d3219549b Change delivery failure tracking to work with hostnames instead of URLs (#13437) 2020-04-15 20:33:24 +02:00
Thibaut Girka d5b5225614 Merge branch 'master' into glitch-soc/merge-upstream 2020-04-04 21:47:37 +02:00
Eugen Rochko 6932e0e2af Add ability to filter audit log in admin UI (#13381) 2020-04-03 13:06:34 +02:00
Thibaut Girka 631dacf1d7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
  Upstream bumped poll option character limit, but we already had
  a higher one, kept ours.
- `app/validators/poll_validator.rb`:
  Upstream bumped poll option character limit, but we already had
  a higher one, kept ours.
- `config/initializers/content_security_policy.rb`:
  Upstream added a rule, the way we compute ours is different, but
  that added rule has been ported.
- `package.json`:
  No real conflict, dependency update. Performed the same update.
- `yarn.lock`:
  No real conflict, dependency update. Performed the same update.
2020-04-02 20:32:00 +02:00
Eugen Rochko b1c596e8f3 Fix background jobs not using locks like they are supposed to (#13361)
Also:

- Fix locks not being removed when jobs go to the dead job queue
- Add UI for managing locks to the Sidekiq dashboard
- Remove unused Sidekiq workers

Fix #13349
2020-03-31 21:59:03 +02:00
Thibaut Girka 3788f37c30 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/services/post_status_service.rb`:
  CW/sensitive logic fixed upstream, but different in glitch-soc.
  Ported the changes accordingly.
2020-03-27 21:54:44 +01:00
ThibG 54da8c4337 Fix media not being marked sensitive when client sets a CW but no text (#13277)
Mastodon enforces the “sensitive” flag on media attachments whenever a toot
is posted with a Content Warning. However, it does so *after* potentially
converting the Content Warning to toot text (when there is no toot text),
which leads to inconsistent and surprising behavior for API clients.
This commit fixes this inconsistency.
2020-03-25 22:40:58 +01:00
Thibaut Girka 6b5c7ee662 Make sanitizer *not* add no-referrer etc. in local markdown toots if the link is “safe” 2020-03-24 12:07:47 +01:00
Thibaut Girka 909b2257ab Fix glitch-soc marking every link in toots as a tag
Fixes #1281
2020-03-24 12:07:47 +01:00
Thibaut Girka 99fc46d023 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Our README.md files are completely different. Discarded upstream changes.
- `app/javascript/core/admin.js`:
  Updating rails-ujs, no real conflict, but a comment to close to changed
  code. Various glitch-soc-only files have been updated to match those changes,
  though.
- `package.json`:
  No real conflict, just an additional dependency in glitch-soc that was too
  close to something updated upstream. Took upstream's changes.
2020-03-22 16:10:44 +01:00
dependabot-preview[bot] 75ba543aa5 Bump sidekiq from 5.2.7 to 6.0.4 (#11727)
* Bump sidekiq from 5.2.7 to 6.0.0

Bumps [sidekiq](https://github.com/mperham/sidekiq) from 5.2.7 to 6.0.0.
- [Release notes](https://github.com/mperham/sidekiq/releases)
- [Changelog](https://github.com/mperham/sidekiq/blob/master/Changes.md)
- [Commits](https://github.com/mperham/sidekiq/compare/v5.2.7...v6.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Sidekiq::Logger.logger -> Sidekiq.logger

* Drop support Ruby 2.4

* update

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-03-21 12:04:54 +09:00
ThibG 977b7ef6d0 Fix reported accounts not being whitelisted when resolving a spamcheck report (#13289) 2020-03-21 03:08:09 +01:00
Thibaut Girka 0fd3f87ea2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
  Conflict due to upstream adding a new parameter (with_rate_limit),
  too close to glitch-soc's own additional parameter (content_type).
  Added upstream's parameter.
- `app/services/post_status_service.rb`:
  Conflict due to upstream adding a new parameter (rate_limit),
  too close to glitch-soc's own additional parameter (content_type).
  Added upstream's parameter.
- `app/views/settings/preferences/appearance/show.html.haml`:
  Conflict due to us not exposing theme settings here (as we have
  a different flavour/skin menu).
  Took upstream change, while still not exposing theme settings.
- `config/webpack/shared.js`:
  Coding style fixes for a part we have rewritten.
  Discarded upstream changes.
2020-03-08 19:38:53 +01:00
guigeekz eb525ef602 Add submit button to the top of preferences pages (#13068)
* Move submit button to the top of the edit page

* Duplicate save button on long form

* Fix click submit on profile spec
2020-03-08 16:04:03 +01:00
Eugen Rochko a4668d994b Add specific rate limits for posting and following (#13172) 2020-03-08 15:17:39 +01:00
Thibaut Girka 07adbc487b Merge branch 'master' into glitch-soc/merge-upstream 2020-02-27 12:37:15 +01:00
Eugen Rochko e0db9f37f5 Fix leak of arbitrary statuses through unfavourite action in REST API (#13161) 2020-02-27 12:32:54 +01:00
Thibaut Girka 8e8cce20c7 Merge branch 'master' into glitch-soc/merge-upstream 2020-02-19 23:04:18 +01:00
ThibG ddd9bad7f1 Fix sign-ups without checked user agreement being accepted through the web form (#13088)
* Fix user agreement not being verified

* Fix tests

* Fix up agreement field being dismissed
2020-02-16 12:56:53 +01:00
Thibaut Girka 7e23ce3cfc Add test for health_check endpoint 2020-02-10 20:04:41 +01:00
Thibaut Girka da4e91ad73 Fix tests
Well, kinda, it's actually a bug from glitch-soc, but we will have to address it separately.
2020-02-09 12:32:33 +01:00
Thibaut Girka b89d5a7408 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile`:
  We updated httplog in a separate commit.
  Took upstream's change which updated it further.
- `Gemfile.lock`:
  We updated httplog in a separate commit.
  Took upstream's change which updated it further.
- `app/lib/sanitize_config.rb`:
  Upstream added better unsupported link stripping,
  while we had different sanitizing configs.
  Took only upstream's link stripping code.
- `config/locales/simple_form.pl.yml`:
  Strings unused in glitch-soc had been removed from
  glitch-soc, reintroduced them even if they are not
  useful, to reduce the risk of later merge conflicts.
2020-02-09 12:15:55 +01:00
Eugen Rochko 4cf2ddc5df Fix rendering `<a>` without `href` when scheme unsupported (#13040)
- Disallow links with relative paths
- Disallow iframes with non-http protocols and relative paths

Close #13037
2020-02-08 21:22:38 +01:00
Thibaut Girka ba1a34ca89 Merge branch 'master' into glitch-soc/merge-upstream 2020-02-06 21:36:38 +01:00
Eugen Rochko 5c4b1db54a Change signature verification to ignore signatures with invalid host (#13033)
Instead of returning a signature verification error, pretend there
was no signature (i.e., this does not allow access to resources that
need a valid signature), so public resources can still be fetched

Fix #13011
2020-02-03 17:48:23 +01:00
Thibaut Girka e76a010414 Merge branch 'master' into glitch-soc/merge-upstream 2020-02-03 09:22:58 +01:00
abcang 94bc823eb6 Search account domain in lowercase (#13016)
* Search account domain in lowercase

* fix rubocop error

* fix spec/models/account_spec.rb
2020-02-01 15:42:24 +01:00
Thibaut Girka 2447918316 Merge branch 'master' into glitch-soc/merge-upstream 2020-01-25 16:50:24 +01:00
Eugen Rochko c058ffd67b Fix validations of reactions limit (#12955) 2020-01-25 16:00:29 +01:00
Thibaut Girka f1ceb471b2 Merge branch 'master' into glitch-soc/merge-upstream 2020-01-25 10:04:41 +01:00
Eugen Rochko fb1df3d5d5 Fix user disabling changing activity timestamps, fix nil error (#12943) 2020-01-25 05:22:35 +01:00
Thibaut Girka a8e43875a5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/statuses_controller.rb`:
  Minor conflict due to theming system
2020-01-24 14:37:06 +01:00
Eugen Rochko 40dd82a857 Remove bad encoding middleware (#12931)
Revert #12741
2020-01-24 00:20:03 +01:00
Eugen Rochko f95fd62bac Add announcements (#12662)
* Add announcements

Fix #11006

* Add reactions to announcements

* Add admin UI for announcements

* Add unit tests

* Fix issues

- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"

* Fix scheduler unpublishing announcements before they are due

* Fix filter params not being passed to announcements filter
2020-01-23 22:00:13 +01:00
Eugen Rochko 930ba0630f Fix media attachments without file being uploadable (#12562)
Fix #12554
2020-01-23 21:40:03 +01:00
ThibG 25e7aa913c Add support for magnet: URIs (#12905) 2020-01-23 21:27:26 +01:00
Thibaut Girka 82e197787f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  No real conflict, glitch-soc-only dependency (redcarpet) too close to an
  upstream one (rdf-normalize)
- `README.md`:
  we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
  No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
  Upstream added code to keep bookmarked statuses, we were already doing so
  with slightly different code. Discarded upstream's changes.
- `package.json`:
  No real conflict, glitch-soc-only dependency (favico.js) too close to
  an upstream one
2020-01-12 15:57:34 +01:00
ThibG 96b43ad187 Remove dependency on OStatus2 gem (#12822) 2020-01-11 21:36:53 +01:00
ThibG b7927b397d Add support for linking XMPP URIs in toots (#12709)
* Fix wrong grouping in Twitter valid_url regex

* Add support for xmpp URIs

Fixes #9776

The difficult part is autolinking, because Twitter-text's extractor does
some pretty ad-hoc stuff to find things that “look like” URLs, and XMPP
URIs do not really match the assumptions of that lib, so it doesn't sound
wise to try to shoehorn it into the existing regex.

This is why I used a specific regex (very close, although slightly more
permissive than the RFC), and a specific scan function (a simplified version
of the generalized one from Twitter).

* Remove leading “xmpp:” from auto-linked text
2020-01-11 02:15:25 +01:00
Alexander dff05cd398 improve status title (#8596)
* improve shown status title, useful for atom/rss

* use single quotes to satisfy codeclimate

* fix tests, make message more pretty

* fix tests

* fix codestyle

* fix codestyle

* remove atom_serializer_spec

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-01-11 06:58:16 +09:00
ThibG f52ef9b007 Allow blocking TLDs, and fix TLD blocks not being editable (#12805)
Fixes #12795

It was already possible to create domain blocks for TLDs, but those
weren't enforced, nor editable. This commit changes it so that they
are enforced and editable.
2020-01-08 22:42:05 +01:00
Thibaut Girka 2591cedcad Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/application_controller.rb`:
  Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
  Conflict due to theming system.
2020-01-04 23:04:42 +01:00
Eugen Rochko 413cb7e861 Fix base64-encoded file uploads not being possible (#12748)
Fix #3804, Fix #5776
2020-01-04 01:54:07 +01:00
Bèr Kessels dfb57c699c Add feature test that tests behaviour of profile name and bio (#12658)
* Add feature test that tests behaviour of profile name and bio

* Fix rubocop style errors in Login Spec.

* DRY log_in_spec by reusing the stories helper

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2020-01-03 02:44:06 +01:00
ThibG 4fcd1d59a9 Fix RefollowWorker not keeping show_reblogs setting (#12707)
* Fix RefollowWorker not keeping show_reblogs setting

* Fix RefollowWorker
2020-01-02 20:52:39 +01:00
Eugen Rochko e8b07657c2 Fix uncaught query param encoding errors (#12741) 2020-01-02 17:14:58 +01:00
ThibG f0cca7a79f Hide blocked users from more places (#12733)
* Hide blocked, muted, and blocked-by users from toot favourite lists

* Hide blocked, muted, and blocked-by users from toot reblog lists

* Hide blocked, muted, and blocked-by users from followers/following (API)

* Fix tests

* Hide blocked, muted, and blocked-by users from followers/following on public pages
2019-12-31 00:55:32 +01:00
Eugen Rochko c72c27756f Fix IDN mentions not being processed, IDN domains not being rendered (#12715)
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.

Fix #7812, fix #12246
2019-12-30 19:20:43 +01:00
Thibaut Girka fa4a92e340 Merge branch 'master' into glitch-soc/master
Conflicts:
- `config/locales/en.yml`
  No real conflict, upstream added a translatable string “too close” to
  one specific to glitch-soc
- `lib/mastodon/statuses_cli.rb`
  Fixes made upstream, while changed in glitch-soc to keep bookmarked statuses
- `package.json`
  No real conflict, additional dependency in glitch-soc
2019-12-19 13:52:54 +01:00
Bèr Kessels 19ffe5f196 Fix typo in login feature-test. (#12651) 2019-12-18 16:54:29 +01:00
ThibG 357bee5ab9 Clean up OStatus-related codepaths (#12173)
* Remove “protocol” argument and return value, as only ActivityPub is supported

* Remove FetchRemoteAccountService, only use ActivityPub::FetchRemoteAccountService

* Fix tests
2019-12-17 13:32:57 +01:00
ThibG f5ac6f7801 Fixes featured hashtag setting page erroring out instead of rejecting invalid tags (#12436)
* Revert "Fix ignoring whole status because of one invalid hashtag (#11621)"

This reverts commit 123ffd0a45.

* Fix statuses being rejected because of invalid hashtag names

* Add spec for invalid hashtag names in statuses

* Add test for featured tags controller
2019-12-17 13:31:56 +01:00
Thomas Citharel fce806216b Add Event activity-type support (#12637)
This adds support for Event AP type in Mastodon. Events are converted
into toots by taking their title (AS name) and their URL (AP ID). Event
picture is also brought in if available.

Testable by fetching event content from https://test.mobilizon.org

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2019-12-16 23:55:28 +01:00
Thibaut Girka 2cc60e8b09 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- package.json
  Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
  Not really a conflict, caused by an additional dependency in glitch-soc.
2019-12-07 12:28:59 +01:00
Yamagishi Kazutoshi ca7447ca96 Move rspec examples to tmp dir (#12539) 2019-12-02 19:55:08 +01:00
ThibG 60d26cabf0 Add follow_request notification type (#12198)
* Add follow_request notification type

The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.

Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.

Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.

* Show follow request link in single-column if there are pending requests, even if account isn't locked

* Push follow requests from notifications to the follow_requests list

* Offer to accept or reject follow request from the notification

* Redesign follow request notification
2019-12-01 17:25:29 +01:00
Thibaut Girka 5aa79af7cb Merge branch 'master' into glitch-soc/merge-upstream 2019-11-23 19:03:01 +01:00
ThibG 32ef133fa6 Fix FetchLinkCardServices crashing on a tags without a target (#12159)
* Add test for links without targets

* Fix FetchLinkCardServices crashing on a tags without a target
2019-11-21 16:04:52 +01:00
Thibaut Girka a96375d2ef Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
  discarded upstream changes
- app/controllers/api/v1/bookmarks_controller.rb
  finally merged upstream, some code style fixes
  and slightly changed pagination code
- app/controllers/application_controller.rb
  changed upstream to always return HTML error pages
  slight conflict caused by theming code
- app/models/bookmark.rb
  finally merged upstream, no real conflict
- spec/controllers/api/v1/bookmarks_controller_spec.rb
  finally merged upstream, slightly changed pagination code
2019-11-20 15:36:09 +01:00
Dimitri Merejkowsky ac14dc7df3 Store rspec statuses in .cache/rspec (#12427)
This allows using rspec with `--only-failures`
2019-11-19 17:22:40 +01:00
Eugen Rochko 357a2e5564 Add cache for OEmbed endpoints to avoid extra HTTP requests (#12403)
* add youtube oembed endpoint

* add check for oembed endpoint

* change unless for a more readable if

* clear blank lines

* endpoint via https

* Fix string literal in condition

* use cache for endpoints

* use cache for endpoints

* clean up and adding check

* clean up and remove redundant return

* add html check

* add false to return

* use double quotes

* use double quotes

* Clean up
2019-11-17 18:40:33 +01:00
Gomasy d13e680f74 Support min_id-based pagination for bookmarks (#12381)
* Support min_id-based pagination for bookmarks

* Fix spec
2019-11-17 17:09:41 +01:00
Jennifer Glauche 0aae35e310 make it not return http 400 when passing and empty source argument (#12259)
* make it not return http 400 when passing and empty source argument

* create a spec for the empty source hash bug

* compact checks for nil, empty? parameters

* use nil.blank? instead checking for nil
2019-11-16 19:02:09 +01:00
Eugen Rochko d144122718 Fix localization test failing due to order of locale definitions (#12393) 2019-11-15 21:00:09 +01:00
ThibG 517c1cd062 Add bookmarks (#7107)
* Add backend support for bookmarks

Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.

* Add spec for bookmark endpoints

* Add front-end support for bookmarks

* Introduce OAuth scopes for bookmarks

* Add bookmarks to archive takeout

* Fix migration

* Coding style fixes

* Fix rebase issue

* Update bookmarked_statuses to latest UI changes

* Update bookmark actions to properly reflect status changes in state

* Add bookmarks item to single-column layout

* Make active bookmarks red
2019-11-13 23:02:10 +01:00
Yamagishi Kazutoshi 2ede35e843 Change to always returns html document in error pages (#12214) 2019-11-13 22:53:05 +01:00
Thibaut Girka 0fc784f8d0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`
  conflict because of the poll option limit being different than upstream's
2019-11-07 13:43:56 +01:00
ThibG 28b68ec875 Fix various issues with account migration (#12301)
* Fix being able to follow oneself by moving to an account that was following the old one

* Add specs

* Add spec to catch MoveWorker issue with local followers following both accounts

* Fix move worker breaking when a local account follows both source and target accounts

* Fix migration from remote to local account not sending Undo Follow

* Fix show_reblogs not being preserved for moved account's followers
2019-11-07 09:05:07 +02:00
Thibaut Girka e7655951d7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`
2019-11-06 14:38:28 +01:00
ThibG 479d8ff47a Fix remote media descriptions being cut off at 420 chars (#12262)
* Fix remote media descriptions being cut off at 420 chars

Fixes #12258

* Fix tests
2019-11-04 13:00:16 +01:00
Thibaut Girka e76f4056e5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/helpers/statuses_helper.rb
  Upstream moved account helpers to their own file, we had extra
  helpers there, moved too.
- app/lib/sanitize_config.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- config/locales/simple_form.en.yml
- spec/lib/sanitize_config_spec.rb
2019-10-27 12:14:01 +01:00
Takeshi Umeda 1dc1d01f95 Split AccountsHelper from StatusesHelper (#12078) 2019-10-24 22:50:09 +02:00
BSKY 483cf4d52a Add noopener and/or noreferrer (#12202) 2019-10-24 22:44:42 +02:00
Thibaut Girka 9506d9cafc Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/javascript/styles/mastodon/components.scss
  conflicts caused by image URLs being different
- app/models/status.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
- app/views/statuses/_simple_status.html.haml
- config/locales/en.yml
  some strings were changed upstream
- spec/models/status_spec.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
2019-10-10 17:26:08 +02:00
Eugen Rochko d99a6145de Remove auto-silence behaviour from spam check (#12117)
Fix #12113
2019-10-09 07:11:23 +02:00
Eugen Rochko 2e07a901c5 Fix attachment not being re-downloaded even if file is not stored (#12125)
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
2019-10-09 07:10:46 +02:00
Eugen Rochko c453888616 Fix performance of home feed regeneration (#12084)
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive

Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
2019-10-06 22:11:17 +02:00
Thibaut Girka 07abea7ba8 Merge tag 'v3.0.0' into glitch-soc/merge-upstream 2019-10-04 19:49:43 +02:00
Eugen Rochko 059287ad08 Add `reason` param to `POST /api/v1/accounts` REST API (#12064)
For approval-required registrations mode
2019-10-03 17:50:59 +02:00
Thibaut Girka c2d6ef1b64 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream raised max image size from 8MB to 10MB while our limit is
  configurable. Raised the default to 10MB.
2019-10-03 11:10:12 +02:00
Eugen Rochko ced431f02f Fix account counters being overwritten by parallel writes (#12045) 2019-10-02 04:59:37 +02:00
ThibG 081cae4c22 Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid (#12037)
* Add test to handle suspended and missing users in BootstrapTimelineService

* Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid
2019-10-01 15:10:00 +02:00
Eugen Rochko 67713f8b08 Remove HEAD request from fetching link previews (#12028)
It is not really necessary and we need to reduce requests
2019-10-01 04:54:10 +02:00
Eugen Rochko f77337daaf Fix records not being indexed sometimes (#12024)
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
2019-10-01 01:19:11 +02:00
Thibaut Girka cc94c90bf1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- Gemfile.lock
- app/controllers/about_controller.rb
- app/controllers/auth/sessions_controller.rb
2019-09-30 12:23:57 +02:00
Eugen Rochko 51a72a802f Add a nodeinfo endpoint (#12002)
* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* Add some additional node info stuff

* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* expanding this to include federation info

* codeclimate feedback

* CC feedback

* using activeserializers seems like a good idea...

* get rid of draft 2.1 version

* Reimplement 2.1, also fix metaData -> metadata

* Fix metaData -> metadata here too

* Fix nodeinfo 2.1 tests

* Implement cache for monthly user aggregate

* Useless

* Remove ostatus from the list of supported protocols

* Fix nodeinfo's open_registration reading obsolete setting variable

* Only serialize domain blocks with user-facing limitations

* Do not needlessly list noop severity in nodeinfo

* Only serialize domain blocks info in nodeinfo when they are set to be displayed to everyone

* Enable caching for nodeinfo endpoints

* Fix rendering nodeinfo

* CodeClimate fixes

* Please CodeClimate

* Change InstancePresenter#active_user_count_months for clarity

* Refactor NodeInfoSerializer#metadata

* Remove nodeinfo 2.1 support as the schema doesn't exist

* Clean-up
2019-09-29 21:31:51 +02:00
Eugen Rochko 66d3e13777 Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
2019-09-28 01:02:21 +02:00
ThibG 70990720c5 Change silences to always require approval on follow (#11975)
* Change silenced accounts to require approval on follow

* Also require approval for follows by people explicitly muted by target accounts

* Do not auto-accept silenced or muted accounts when switching from locked to unlocked

* Add `follow_requests_count` to verify_credentials

* Show “Follow requests” menu item if needed even if account is locked

* Add tests

* Correctly reflect that follow requests weren't auto-accepted when local account is silenced

* Accept follow requests from user-muted accounts to avoid leaking mutes
2019-09-27 21:13:51 +02:00
Eugen Rochko a2d3728c36 Add account migration UI (#11846)
Fix #10736

- Change data export to be available for non-functional accounts
- Change non-functional accounts to include redirecting accounts
2019-09-19 20:58:19 +02:00
Thibaut Girka 4837d6275a Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb
  Minor conflict due to glitch-soc's theming code
2019-09-18 17:25:56 +02:00
Eugen Rochko 1781358bd9 Add password challenge to 2FA settings, e-mail notifications (#11878)
Fix #3961
2019-09-18 16:37:27 +02:00
Eugen Rochko 195ea5bce5 Change spam check to apply to local accounts and add a threshold (#11806)
Instead of detecting spam on first duplicate message, add a
threshold of 5 such messages to reduce false positives
2019-09-18 12:53:13 +02:00
Eugen Rochko 8904a4eb36 Fix TOTP codes not being filtered from logs during enabling/disabling (#11877)
Not a serious issue because they are meaningless past single use
2019-09-18 02:48:40 +02:00
Eugen Rochko 8eb0d880cb Fix 2FA challenge and password challenge for non-database users (#11831)
* Fix 2FA challenge not appearing for non-database users

Fix #11685

* Fix account deletion not working when using external login

Fix #11691
2019-09-15 21:08:39 +02:00
Thibaut Girka e143653d3a Fix tests being broken because of different default setting 2019-09-13 20:28:22 +02:00
Thibaut Girka 2e38d889a3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
  Conflict because we changed the number of default results to be
  configurable
- app/lib/settings/scoped_settings.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
2019-09-13 18:13:43 +02:00
Eugen Rochko 8824964836 Remove deprecated `GET /api/v1/search` API (#11823)
Use `GET /api/v2/search` instead
2019-09-13 16:11:13 +02:00
Eugen Rochko 45b759a2ca Fix hashtags being split by ZWNJ character (#11821)
Fix #11761
2019-09-13 16:01:26 +02:00
Eugen Rochko 4291b74031 Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Tao Bror Bojlén 0430cbec29 add admin setting for default search engine indexing (fix #11750) (#11804) 2019-09-11 08:44:58 +02:00
ThibG 875d2e2b59 Add updated relationship to follow request API responses (#11800)
Fixes #11747
2019-09-10 20:56:42 +02:00
Eugen Rochko 25fb124ee6 Add batch actions and categories to admin UI for custom emojis (#11793) 2019-09-09 22:44:17 +02:00
Eugen Rochko 89989b6255 Add timeline read markers API (#11762)
Fix #4093
2019-09-06 13:55:51 +02:00
Thibaut Girka eac7c66d2a Merge branch 'master' into glitch-soc/merge-upstream 2019-09-05 11:36:41 +02:00
ThibG 8b271d6d25 Fix ActivityPub context not being dynamically computed (#11746)
* Fix contexts not being dynamically included

Fixes #11649

* Refactor Note context in serializer

* Refactor Actor serializer
2019-09-03 22:52:32 +02:00
Eugen Rochko 436266ca27 Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
Thibaut Girka 741b53651e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/directories/index.html.haml
  Upstream has redesigned the profile directory, and we
  had a glitch-soc-specific change to hide follower counts.
  Ported that change to the new design.
2019-08-30 13:38:41 +02:00
Eugen Rochko ee9d0c0f38 Fix uncaught domain normalization error in remote follow (#11703) 2019-08-30 02:19:17 +02:00
Eugen Rochko f21e27e914 Fix uncaught parameter missing exceptions and missing error templates (#11702) 2019-08-30 01:34:47 +02:00
Thibaut Girka 449a0727a7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb

All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
2019-08-29 12:07:50 +02:00
Eugen Rochko 9959cdc79f Add option to include reported statuses in warning e-mail (#11639) 2019-08-23 22:37:23 +02:00
Eugen Rochko e4605bbb4a Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
2019-08-22 04:17:12 +02:00
Thibaut Girka c060523dc2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/home_controller.rb
- app/controllers/shares_controller.rb
- app/javascript/packs/public.js
- app/models/status.rb
- app/serializers/initial_state_serializer.rb
- app/views/home/index.html.haml
- app/views/layouts/public.html.haml
- app/views/public_timelines/show.html.haml
- app/views/shares/show.html.haml
- app/views/tags/show.html.haml
- config/initializers/content_security_policy.rb
- config/locales/en.yml
- config/webpack/shared.js
- package.json
2019-08-19 21:49:35 +02:00
Eugen Rochko 848cb2a697 Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
2019-08-18 03:45:51 +02:00
Eugen Rochko a8d983934e Fix blurhash and autoplay not working on public pages (#11585) 2019-08-16 19:15:05 +02:00
Eugen Rochko 6393a094c6 Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
2019-08-16 01:24:03 +02:00
Thibaut Girka 81c6f2fd0e Merge branch 'master' into glitch-soc/merge-upstream 2019-08-08 08:44:33 +02:00
ThibG 072e696aa1 Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Thibaut Girka ff76f6de4f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/views/admin/dashboard/index.html.haml
- config/locales/simple_form.en.yml
2019-08-07 13:56:48 +02:00
Yusuke Nakamura ed9245d6b2 Bump faker from 1.9.6 to 2.1.0 and update faker api (#11489)
* Bump faker from 1.9.6 to 2.1.0

Bumps [faker](https://github.com/stympy/faker) from 1.9.6 to 2.1.0.
- [Release notes](https://github.com/stympy/faker/releases)
- [Changelog](https://github.com/stympy/faker/blob/master/CHANGELOG.md)
- [Commits](https://github.com/stympy/faker/compare/1.9.6...v2.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Use faker api v2

https://github.com/stympy/faker/releases/tag/2.0
2019-08-06 15:33:03 +02:00
Thibaut Girka 7a55d93244 Merge branch 'master' into glitch-soc/merge-upstream 2019-08-06 13:16:53 +02:00
Eugen Rochko 05e9cd13eb Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
2019-08-05 19:54:29 +02:00
Thibaut Girka 77416866a7 Merge branch 'master' into glitch-soc/merge-upstream 2019-08-05 13:13:28 +02:00
Eugen Rochko 0ed96e9aee Remove XML version of Webfinger and remove links to Atom feeds (#11460)
Fix #11453
2019-08-01 19:14:02 +02:00
Eugen Rochko b92e18080a Change hashtag search to only return results that have trended in the past (#11448)
* Change hashtag search to only return results that have trended in the past

A way to eliminate typos and other one-off "junk" results

* Fix excluding exact matches that don't have a score

* Fix tests
2019-07-30 20:29:50 +02:00
ThibG 1848f7fae4 Fix boosting & unboosting preventing a boost from appearing in the TL (#11405)
* Fix boosting & unboosting preventing a boost from appearing in the TL

* Add tests

* Avoids side effects when aggregate_reblogs isn't true
2019-07-30 13:18:23 +02:00
Thibaut Girka 4973ba2d1f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/directories_controller.rb
- package.json
- yarn.lock
2019-07-30 12:22:33 +02:00
Eugen Rochko 825dc3ca22 Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00