Commit Graph

1214 Commits (de27fa27b0338612ca64f6fb177fd34611014aee)

Author SHA1 Message Date
Claire da9237adb5 Fix “Authorized applications” inefficiently and incorrectly getting last use date (#25060) 2023-05-22 14:03:38 +02:00
Emelia Smith 8b4d7b22ed Allow reports with long comments from remote instances, but truncate (#25028) 2023-05-22 13:15:21 +02:00
Claire 0635b1c228 Change OpenGraph-based embeds to allow fullscreen (#25058) 2023-05-22 12:25:56 +02:00
Claire c8998a2678 Change profile updates to be sent to recently-mentioned servers (#24852) 2023-05-12 13:13:04 +02:00
Claire c752e939d5 Change AccessTokensVacuum to also delete expired tokens (#24868) 2023-05-11 04:40:03 +02:00
Claire 348aed03cf Merge commit '0d919f27beb6e4e7a562a6eed8f354415b5c217e' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Upstream made changes, but we had removed it.
  Discarded upstream changes.
- `.rubocop_todo.yml`:
  Upstream regenerated the file, we had some glitch-soc-specific ignores.
- `app/models/account_statuses_filter.rb`:
  Minor upstream code style change where glitch-soc had slightly different code
  due to handling of local-only posts.
  Updated to match upstream's code style.
- `app/models/status.rb`:
  Upstream moved ActiveRecord callback definitions, glitch-soc had an extra one.
  Moved the definitions as upstream did.
- `app/services/backup_service.rb`:
  Upstream rewrote a lot of the backup service, glitch-soc had changes because
  of exporting local-only posts.
  Took upstream changes and added back code to deal with local-only posts.
- `config/routes.rb`:
  Upstream split the file into different files, while glitch-soc had a few
  extra routes.
  Extra routes added to `config/routes/settings.rb`, `config/routes/api.rb`
  and `config/routes/admin.rb`
- `db/schema.rb`:
  Upstream has new migrations, while glitch-soc had an extra migration.
  Updated the expected serial number to match upstream's.
- `lib/mastodon/version.rb`:
  Upstream added support to set version tags from environment variables, while
  glitch-soc has an extra `+glitch` tag.
  Changed the code to support upstream's feature but prepending a `+glitch`.
- `spec/lib/activitypub/activity/create_spec.rb`:
  Minor code style change upstream, while glitch-soc has extra tests due to
  `directMessage` handling.
  Applied upstream's changes while keeping glitch-soc's extra tests.
- `spec/models/concerns/account_interactions_spec.rb`:
  Minor code style change upstream, while glitch-soc has extra tests.
  Applied upstream's changes while keeping glitch-soc's extra tests.
2023-05-08 19:28:21 +02:00
Claire 2aaf3528ca Merge commit 'd548faf69321260ab4c2a16759ca77773189126a' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/styles/mastodon/forms.scss`:
  Conflict because we ran eslint autofix on upstream files.
- `config/initializers/content_security_policy.rb`:
  Code style changes but we have a different version.
  Kept our version.
- `streaming/index.js`:
  Upstream fixed a typo close to glitch-soc-only code.
  Applied upstream's changes.
2023-05-08 15:28:36 +02:00
Nick Schonning f6fd5e4ad0 Autofix Rubocop Style/ConcatArrayLiterals (#23755) 2023-05-04 05:57:40 +02:00
Matt Jankowski 53682a7b4f Fix Performance/DeletePrefix cop (#24796) 2023-05-02 21:07:45 +02:00
Matt Jankowski 4d1f9d4278 Rename `with_lock` to `with_redis_lock` to avoid confusion with ActiveRecord's method (#24741) 2023-05-02 18:16:07 +02:00
Matt Jankowski 276a0f7ff9 Fix Performance/Count cop (#24793) 2023-05-02 18:06:43 +02:00
Matt Jankowski 621e53f8fc Fix Performance/Sum cop (#24788) 2023-05-02 16:10:40 +02:00
Matt Jankowski 182bb4595f Fix Lint/DuplicateBranch cop (#24766) 2023-05-02 12:57:11 +02:00
Claire abce7cf2b9 Rewrite import feature (#21054) 2023-05-02 12:08:48 +02:00
dependabot[bot] c943082b1b Bump connection_pool from 2.3.0 to 2.4.0 (#24290)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-04-30 15:45:02 +09:00
Claire 498de92bc8 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Updated upstream, removed in glitch-soc to disable noise.
  Kept removed.
- `CODE_OF_CONDUCT.md`:
  Upstream updated to a new version of the covenant, but I have not read it
  yet, so kept unchanged.
- `Gemfile.lock`:
  Not a real conflict, one upstream dependency updated textually too close to
  the glitch-soc only `hcaptcha` dependency.
  Applied upstream changes.
- `app/controllers/admin/base_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/application_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/disputes/base_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/relationships_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/statuses_cleanup_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/helpers/application_helper.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.jsx`:
  Upstream added a highlight animation for onboarding, while we changed the
  max character limit.
  Applied our local changes on top of upstream's new version.
- `app/views/layouts/application.html.haml`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `stylelint.config.js`:
  Upstream added ignore paths, glitch-soc had extra ignore paths.
  Added the same paths as upstream.
2023-04-29 10:44:56 +02:00
Nick Schonning 519ae8c0c6 Drop EOL Ruby 2.7 (#24237) 2023-04-27 01:46:18 +02:00
Matt Jankowski f01a643897 Remove instance variables from helper usage (#24203) 2023-04-23 22:35:54 +02:00
Claire 0cbf19affd Fix most rubocop issues (#2165)
* Run rubocop --autocorrect on app/, config/ and lib/, also manually fix some remaining style issues

* Run rubocop --autocorrect-all on db/

* Run rubocop --autocorrect-all on `spec/` and fix remaining issues
2023-04-09 11:25:30 +02:00
Claire 1a0c54554f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream added a link to the roadmap, but we have a completely different README.
  Kept ours.
- `app/models/media_attachment.rb`:
  Upstream upped media attachment limits.
  Updated the default according to upstream's.
- `db/migrate/20180831171112_create_bookmarks.rb`:
  Upstream changed the migration compatibility level.
  Did so too.
- `config/initializers/content_security_policy.rb`:
  Upstream refactored this file but we have a different version.
  Kept our version.
- `app/controllers/settings/preferences_controller.rb`:
  Upstream completely refactored user settings storage, and glitch-soc has a
  different set of settings.
  The file does not directly references individual settings anymore.
  Applied upstream changes.
- `app/lib/user_settings_decorator.rb`:
  Upstream completely refactored user settings storage, and glitch-soc has a
  different set of settings.
  The file got removed entirely.
  Removed it as well.
- `app/models/user.rb`:
  Upstream completely refactored user settings storage, and glitch-soc has a
  different set of settings.
  References to individual settings have been removed from the file.
  Removed them as well.
- `app/views/settings/preferences/appearance/show.html.haml`:
  Upstream completely refactored user settings storage, and glitch-soc has a
  different set of settings.
  Applied upstream's changes and ported ours back.
- `app/views/settings/preferences/notifications/show.html.haml`:
  Upstream completely refactored user settings storage, and glitch-soc has a
  different set of settings.
  Applied upstream's changes and ported ours back.
- `app/views/settings/preferences/other/show.html.haml`:
  Upstream completely refactored user settings storage, and glitch-soc has a
  different set of settings.
  Applied upstream's changes and ported ours back.
- `config/settings.yml`:
  Upstream completely refactored user settings storage, and glitch-soc has a
  different set of settings.
  In particular, upstream removed user-specific and unused settings.
  Did the same in glitch-soc.
- `spec/controllers/application_controller_spec.rb`:
  Conflicts due to glitch-soc's theming system.
  Mostly kept our version, as upstream messed up the tests.
2023-03-31 21:30:27 +02:00
Eugen Rochko c75fccf033 Change user settings to be stored in a more optimal way (#23630)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-03-30 14:44:00 +02:00
Matt Jankowski 4e6cf24ce9 Update retention model sql query (#24249) 2023-03-26 00:37:57 +01:00
Claire 488cd37f1a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream changed their README, we have our own.
  Kept ours.
- `app/helpers/application_helper.rb`:
  Minor code style fix upstream, on a line that is different in glitch-soc
  due to the different theming system.
  Applied the code style fix to our own code.
- `app/views/settings/preferences/appearance/show.html.haml`:
  Code style fix on a line next to lines exclusive to glitch-soc.
  Applied upstream changes.
- `yarn.lock`:
  Upstream updated a dependency textually close to a glitch-soc-only
  dependency.
  Updated the dependency like upstream did.
2023-03-16 23:12:59 +01:00
Claire 3b17f2f718 Add warning for object storage misconfiguration (#24137) 2023-03-16 22:47:01 +01:00
Christian Schmidt 7583523bf0 Replace `Status#translatable?` with language matrix in separate endpoint (#24037) 2023-03-16 11:07:24 +01:00
Eugen Rochko f0734513f4 Refactor monkey-patching of `PrivateAddressCheck` (#24122) 2023-03-16 04:33:38 +01:00
Claire f39d9ab256 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream switched to pushing to both DockerHub and GitHub Container
  Repository, while glitch-soc was already pushing to the latter only.
  Updated our configuration to be slightly more consistent with upstream's
  naming and styling, but kept our behavior.
- `Gemfile.lock`:
  Updated dependencies textually too close to glitch-soc only hcaptcha
  dependency.
  Updated dependencies as upstream did.
- `README.md`:
  Upstream updated its README, but we have a completely different one.
  Kept our README, though it probably should be reworked at some point.
- `app/views/auth/sessions/two_factor.html.haml`:
  Minor style fix upstream that's on a line glitch-soc removed because
  of its different theming system.
  Kept our file as is.
- `spec/controllers/health_controller_spec.rb`:
  This file apparently did not exist upstream, upstream created it with
  different contents but it is functionally the same.
  Switched to upstream's version of the file.
- `spec/presenters/instance_presenter_spec.rb`:
  Upstream changed the specs around `GITHUB_REPOSITORY`, while glitch-soc
  had its own code because it's a fork and does not have the same default
  source URL.
  Took upstream's change, but with glitch-soc's repo as the default case.
- `yarn.lock`:
  Upstream dependencies textually too close to a glitch-soc only one.
  Updated dependencies as upstream did.
2023-03-15 09:16:10 +01:00
Claire bde1c7aa96 Fix dashboard crash on ElasticSearch server error (#23751) 2023-03-15 04:01:49 +01:00
Claire 23bce96f0c Change sidekiq-bulk's batch size from 10,000 to 1,000 jobs in one Redis call (#24034) 2023-03-15 03:45:15 +01:00
Christian Schmidt 170d3a0bc8 Unescape HTML entities (#24019) 2023-03-08 19:56:41 +01:00
Matt Jankowski 8ea6707b2b Remove unused TOCGenerator (#23986) 2023-03-06 17:33:58 +01:00
Claire a6363c3a2a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream README has been changed, but we have a completely different one.
  Kept our `README.md`.
- `lib/sanitize_ext/sanitize_config.rb`:
  Upstream added support for more incoming HTML tags (a large subset of what
  glitch-soc accepts).
  Change the code style to match upstream's but otherwise do not change our
  code.
- `spec/lib/sanitize_config_spec.rb`:
  Upstream added support for more incoming HTML tags (a large subset of what
  glitch-soc accepts).
  Kept our version, since the tests are mostly glitch-soc's, except for cases
  which are purposefuly different.
2023-03-05 20:46:56 +01:00
Jean byroot Boussier 08c2938b4a Upgrade to latest redis-rb 4.x and fix deprecations (#23616)
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-03-04 16:38:28 +01:00
Christian Schmidt cc0c37cd69 Only offer translation for supported languages (#23879) 2023-03-03 21:06:31 +01:00
Claire f15ded319f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.prettierignore`:
  Upstream added a line at the end of the file, while glitch-soc had its own
  extra lines.
  Took upstream's change.
- `CONTRIBUTING.md`:
  We have our custom CONTRIBUTING.md quoting upstream. Upstream made changes.
  Ported upstream changes.
- `app/controllers/application_controller.rb`:
  Upstream made code style changes in a method that is entirely replaced
  in glitch-soc.
  Ignored the change.
- `app/models/account.rb`:
  Code style changes textually close to glitch-soc-specific changes.
  Ported upstream changes.
- `lib/sanitize_ext/sanitize_config.rb`:
  Upstream code style changes.
  Ignored them.
2023-02-25 14:00:40 +01:00
Nick Schonning 247bf6e812 Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning dd63a919f0 Autofix Rubocop Style/RescueStandardError (#23745) 2023-02-20 11:01:20 +01:00
Nick Schonning e6312104c7 Autofix Rubocop remaining Layout rules (#23679) 2023-02-20 06:58:28 +01:00
Nick Schonning b8f17330ea Autofix Rubocop Rails/Pluck (#23730) 2023-02-20 02:28:40 +01:00
Claire 63992c6900 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Upstream made changes while we have dropped this file.
  Keep the file deleted.
- `.prettierignore`:
  Upstream made changes at the end of the file, where we
  had our extra lines.
  Just moved our extra lines back at the end.
- `app/serializers/initial_state_serializer.rb`:
  Upstream code style changes.
  Applied them.
- `app/services/backup_service.rb`:
  Upstream code style changes.
  Applied them.
2023-02-19 10:42:55 +01:00
Nick Schonning be1ec56c6d Autofix Rubocop Style/StringLiterals (#23695) 2023-02-19 07:38:14 +09:00
Nick Schonning 49f1168050 Autofix Rubocop Style/RedundantBegin (#23703) 2023-02-19 07:09:40 +09:00
Nick Schonning 793f8c7dd5 Autofix Rubocop Style/IfUnlessModifier (#23697) 2023-02-18 12:37:47 +01:00
Nick Schonning 4553bc78ba Autofix Rubocop Style/PerlBackrefs (#23649) 2023-02-18 14:20:20 +09:00
Nick Schonning b7d4391c4d Autofix Rubocop Lint/AmbiguousOperatorPrecedence (#23681) 2023-02-18 04:30:23 +01:00
Nick Schonning a71a0802d6 Autofix Rubocop Lint/SymbolConversion (#23683) 2023-02-18 03:23:49 +01:00
Nick Schonning 31ca76fe2c Run rubocop formatting except line length (#23632) 2023-02-18 06:56:20 +09:00
Claire 6917c854d1 Merge branch 'main' into glitch-soc/merge-upstream 2023-02-10 22:49:59 +01:00
Claire ea4cffac37 Fix unbounded recursion in post discovery (#23506)
* Add a limit to how many posts can get fetched as a result of a single request

* Add tests

* Always pass `request_id` when processing `Announce` activities

---------

Co-authored-by: nametoolong <nametoolong@users.noreply.github.com>
2023-02-10 22:16:37 +01:00
Claire 5f26e6e7ca Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Minor upstream change, our README is completely different.
  Kept ours.
- `lib/tasks/assets.rake`:
  glitch-soc has extra code to deal with its theming system,
  upstream changed a line that exists in glitch-soc.
  Applied upstream changes.
2023-02-09 12:46:12 +01:00
Nick Schonning ae45c9ccd4 Apply Rubocop Rails/WhereNot (#23448)
* Apply Rubocop Rails/WhereNot

* Update spec for where.not
2023-02-08 10:39:57 +01:00
Nick Schonning 4a49c50e64 Apply Rubocop Performance/BlockGivenWithExplicitBlock (#23441)
* Apply Rubocop Performance/BlockGivenWithExplicitBlock

* Unprefix used block parameter
2023-02-08 10:36:23 +01:00
Nick Schonning 67a6f688ca Apply Rubocop Performance/StringIdentifierArgument (#23444) 2023-02-08 02:36:20 +01:00
Nick Schonning 6af127a98e Apply Rubocop Performance/DoubleStartEndWith (#23442) 2023-02-08 10:06:48 +09:00
Nick Schonning 38ec6d27d8 Autofix Rails/EagerEvaluationLogMessage (#23429)
* Autofix Rails/EagerEvaluationLogMessage

* Update spec for debug block syntax
2023-02-07 03:44:36 +01:00
Claire 91f66ece5a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream updated `docker/build-push-action`, and we a different config
  for `docker/metadata-action` so the lines directly above were different,
  but it's not a real conflict.
  Upgraded `docker/build-push-action` as upstream did.
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
  Upstream changed the codestyle near a line we had modified to accommodate
  configurable character count.
  Kept our change.
2023-02-03 19:23:27 +01:00
Eugen Rochko 502c919bfd Change number of stored items in home feed from 400 to 800 (#23349) 2023-02-01 11:24:22 +01:00
Vyr Cossont 06912c80a7 Advertise supported MIME types for statuses (#2090) 2023-01-20 13:57:32 +01:00
Claire 5fbe5fed1e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `config/i18n-tasks.yml`:
  Upstream added new ignored strings, glitch-soc has extra ignored strings
  because of the theming system.
  Added upstream's changes.
2023-01-18 17:38:11 +01:00
Claire ae94782edb Fix spurious admin dashboard warning when using ElasticSearch 7.x (#23064)
Some 7.x ElasticSearch versions support some 6.x nodes, thus the version check
is inadequate. I am not sure there is a good way to check if a server
implements all the 7.x APIs, so check server version and minimum wire version
instead.
2023-01-18 16:21:48 +01:00
Claire 43b9c4c0a7 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/status.rb`:
  Minor upstream refactor moved hook definitions around,
  and glitch-soc has an extra `before_create`.
  Moved the `before_create` accordingly.
- `app/services/batched_remove_status_service.rb`:
  Minor upstream refactor changed a block in which glitch-soc
  had one extra call to handle direct timelines.
  Adapted changes to keep glitch-soc's extra call.
2023-01-12 10:15:46 +01:00
Markus Unterwaditzer b7f37e68ad fix: allow verification when page size exceeds 1MB (using HTML5 parser) (#22879)
* fix: allow verification when page size exceeds 1MB
Truncates the page after 1MB instead

Closes #15316

* switch to HTML5 parser, fix rubocop errors

* undo rubocop fixes

Co-authored-by: Chris Zubak-Skees <chriszs@gmail.com>
2023-01-11 21:59:13 +01:00
Kaspar V 1cd04e5d77 Fix/remove calling private method with send in model (#22951)
* fix(status): remove send usage for private unlink_from_conversations

- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service

* fix(feeds_vacuum): replace find_in_batches with in_batches

because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
2023-01-11 21:57:24 +01:00
Claire 341b653578 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream updated its README, while we have a completely different one.
  Kept our README.
- `app/controllers/concerns/web_app_controller_concern.rb`:
  Conflict because of glitch-soc's theming system.
  Additionally, glitch-soc has different behavior regarding moved accounts.
  Ported some of the changes, but kept our overall behavior.
- `app/javascript/packs/admin.js`:
  Code changes actually applied to `app/javascript/core/admin.js`
2023-01-05 14:16:25 +01:00
Vyr Cossont b2c7ca296c Check OpenSearch compatibility version instead of regular version (#22422)
Fixes #18535.
2023-01-05 13:44:31 +01:00
Jeong Arm 5180ae5bf6 Fix home TL could contain post from who blocked me (#22849)
* Fix home tl contains post from who blocked me

* Add test

* Fix feed_manager's build_crutches

blocked_by was not includes status' owner

* Add test for status from I blocked

* Fix typo
2023-01-05 13:30:38 +01:00
Claire f23d971cd8 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Discarded upstream changes: we have our own README
- `app/controllers/follower_accounts_controller.rb`:
  Port upstream's minor refactoring
2022-12-15 20:25:25 +01:00
Claire e1d8048bc4 Fix status cache hydration discrepancy (#19879) 2022-12-15 18:55:55 +01:00
Claire c83083eef5 Fix inability to use local LibreTranslate without setting ALLOWED_PRIVATE_ADDRESSES (#21926)
Fixes #20029
2022-12-15 17:04:38 +01:00
s0 289097866b Fix crash in elasticsearch_check.rb (#21006)
Nil unwrap causes the admin dashboard to crash/500 when the Chewy client info version number value is nil.
This occurs when running another ES-compatible backend such as MeiliSearch.
Obviously it would be good for chewy to recognise upstream but at least avoiding the crash would be fine.
2022-12-15 15:56:48 +01:00
Jeong Arm 53b6623136 Don't delivery a reply to domains which are blocked by author (#22117)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-15 15:52:06 +01:00
Claire aec7de494f Fix unbounded recursion in account discovery (#22025)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-07 00:15:24 +01:00
Claire afa828e3d4 Fix unbounded recursion in account discovery (#1994)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-04 21:23:19 +01:00
kibigo! 2fbfe4ad92 Use a tree‐based approach for advanced text formatting (#1907)
* Use a tree‐based approach for adv. text formatting

Sanitizing HTML/Markdown means parsing the content into an HTML tree
under‐the‐hood anyway, and it is more accurate to do mention/hashtag
replacement on the text nodes in that tree than it is to try to hack it
in with regexes et cetera.

This undoes the overrides of `#entities` and `#rewrite` on
`AdvancedTextFormatter` but also stops using them, instead keeping
track of the parsed Nokogiri tree itself and using that in the `#to_s`
method.

Internally, this tree uses `<mastodon-entity>` nodes to keep track of
hashtags, links, and mentions. Sanitization is moved to the beginning,
so it should be known that these do not appear in the input.

* Also disallow entities inside of `<code>`

I think this is generally expected behaviour, and people are annoyed
when their code gets turned into links/hashtags/mentions.

* Minor cleanup to AdvancedTextFormatter

* Change AdvancedTextFormatter to rewrite entities in one pass and sanitize at the end

Also, minor refactoring to better match how other formatters are organized.

* Add some tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-02 10:29:42 +01:00
Claire 3186d2f63e Merge branch 'main' into glitch-soc/merge-upstream 2022-12-01 11:08:52 +01:00
Claire 1bef772764 Fix some performance issues with /admin/instances (#21907)
/admin/instances?availability=failing remains wholly unefficient
2022-12-01 10:32:10 +01:00
Claire 02992b262a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Our README is completely different.
  Discarded upstream changes.
2022-11-28 11:33:34 +01:00
Kaspar V f2ec356cb6 refactor(vacuum statuses): reduce amount of db queries and load for each query - improve performance (#21487)
* refactor(statuses_vacuum): remove dead code - unused

Method is not called inside class and private.
Clean up dead code.

* refactor(statuses_vacuum): make retention_period present test explicit

This private method only hides functionality.
It is best practice to be as explicit as possible.

* refactor(statuses_vacuum): improve query performance

- fix statuses_scope having sub-select for Account.remote scope by
  `joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
  because it is inexplicit, bad practice and even slower than normal
  `.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
  active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
  because in_batches delivers a full blown active record query result,
  in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
  reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
  the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
  another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send

* refactor(status): simplify unlink_from_conversations

- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?

* refactor(status): searchable_by way less sub selects

These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.

This is how it was:

```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
  Status Load (1.6ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (1.5ms)  SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```

This is how it is with this change:

```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
  Status Load (1.7ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (0.7ms)  SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```
2022-11-27 20:41:18 +01:00
Claire 699db5f2af Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile`:
  Upstream removed blank lines.
2022-11-14 20:27:31 +01:00
Claire c9b53436d7 Fix emoji substitution not applying only to text nodes in backend code (#20641)
Signed-off-by: Claire <claire.github-309c@sitedethib.com>

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-11-14 20:26:21 +01:00
Claire 81d24609e5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/account.rb`:
  Conflict because we (glitch-soc) have disabled trending of posts without
  review.
  Discarded that upstream change.
- `app/views/admin/settings/discovery/show.html.haml`:
  Just an extra setting in glitch-soc.
  Kept that extra setting.
2022-11-12 13:28:03 +01:00
Bearice Ren 192fe87f51 fixes ArgumentError when proxy is used (#20420)
* fixes ArgumentError when proxy is used

* Update app/lib/request.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-11-11 21:31:03 +01:00
Eugen Rochko e6e139ac2b Change incoming activity processing to happen in `ingress` queue (#20264) 2022-11-10 14:21:51 +01:00
Claire 09f987bce1 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/custom_emoji.rb`:
  Not a real conflict, just upstream changing a line too close to
  a glitch-soc-specific validation.
  Applied upstream changes.
- `app/models/public_feed.rb`:
  Not a real conflict, just upstream changing a line too close to
  a glitch-soc-specific parameter documentation.
  Applied upstream changes.
2022-11-10 09:36:47 +01:00
James Tucker c4ff2640c1 Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Claire 18a11da3b5 Change Request connection logic to try both IPv6 and IPv4 when available (#20108)
Fixes #19751
2022-11-08 16:36:26 +01:00
Claire 0f9c70d5a2 Merge branch 'main' into glitch-soc/merge-upstream 2022-11-08 15:44:57 +01:00
James Tucker f48dcd025b Improve performance by avoiding method cache busts (#19957)
Switch to monkey-patching http.rb rather than a runtime extend of each
response, so as to avoid busting the global method cache. A guard is
included that will provide developer feedback in development and test
environments should the monkey patch ever collide.
2022-11-08 04:00:27 +01:00
Claire 99cdfec05e Merge branch 'main' into glitch-soc/merge-upstream 2022-11-07 19:53:30 +01:00
Claire 2ed7a4124b Fix filter handling in status cache hydration (#19963) 2022-11-07 19:47:48 +01:00
Claire 03aa732d5c Fix status cache hydration discrepancy 2022-11-06 17:29:04 +01:00
Claire 20fb459dee Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
  glitch-soc change because of having changed the default number of
  available poll options.
  Applied upstream's changes while keeping glitch-soc's default number of
  poll options.
- `public/oops.png`:
  We had a minor graphics change, probably not worth diverging from upstream.
  Took upstream version.
2022-11-06 09:50:41 +01:00
Claire 6543708c41 Fix users not being able to change their hide_followers_count setting (#1889) 2022-11-05 11:55:25 +01:00
Claire 0735081cc2 Fix additional issues with status cache hydration (#19747)
* Spare one SQL query when hydrating polls

* Improve tests

* Fix more discrepancies

* Fix possible crash when the status has no application set
2022-11-04 20:01:33 +01:00
Claire e97b0e0700 Fix various issues with store hydration (#19746)
- Improve tests
- Fix possible crash when application of a reblogged post isn't set
- Fix discrepancies around favourited and reblogged attributes
- Fix discrepancies around pinned attribute
- Fix polls not being hydrated
2022-11-04 19:33:16 +01:00
Claire 4b0e72b2b7 Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
Eugen Rochko 5b639f9e15 Add caching for payload serialization during fan-out (#19642) 2022-11-04 13:21:06 +01:00
Claire 0060eb5dcf Merge branch 'main' into glitch-soc/merge-upstream 2022-10-28 19:23:58 +02:00
Eugen Rochko 9788e248fb Fix improperly checking for blocked domain on followed hashtags (#19472)
Fix #19469
2022-10-26 20:40:56 +02:00
Eugen Rochko d53e1fc6d9 Fix vacuum scheduler missing lock, locks never expiring (#19458)
Remove vacuuming of orphaned preview cards
2022-10-26 12:10:48 +02:00