Commit Graph

670 Commits (a5beca28d52fafc5b879ccf549325c5f7bfd12e7)

Author SHA1 Message Date
Claire 852eb6ce7a Fix sidekiq jobs not triggering Elasticsearch index updates (#24046) 2023-03-12 23:47:55 +01:00
9p4 8ca1b0b6fc Add refreshing many accounts at once with "tootctl accounts refresh" (#23304) 2023-03-08 17:06:53 +01:00
Jean byroot Boussier 08c2938b4a Upgrade to latest redis-rb 4.x and fix deprecations (#23616)
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-03-04 16:38:28 +01:00
Claire cdc026c2fb Fix tootctl accounts migrate error due to typo (#23567) 2023-03-03 20:45:12 +01:00
Claire 16c8144b1c Add support for incoming rich text (#23913) 2023-03-03 20:19:29 +01:00
Nick Schonning eb6991cee1 Autofix Rubocop Rails/RootPathnameMethods (#23760) 2023-02-22 09:57:15 +09:00
Nick Schonning 247bf6e812 Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning dd63a919f0 Autofix Rubocop Style/RescueStandardError (#23745) 2023-02-20 11:01:20 +01:00
Nick Schonning 9b3be79cb5 Autofix Rubocop Style/FormatString (#23743) 2023-02-20 07:58:33 +01:00
Nick Schonning e6312104c7 Autofix Rubocop remaining Layout rules (#23679) 2023-02-20 06:58:28 +01:00
Nick Schonning c08d9e5e5c Autofix Rubocop Style/UnpackFirst (#23741) 2023-02-20 06:51:43 +01:00
Nick Schonning c0f0c42326 Audofix Rubocop Style/WordArray (#23739) 2023-02-20 06:14:10 +01:00
Nick Schonning be1ec56c6d Autofix Rubocop Style/StringLiterals (#23695) 2023-02-19 07:38:14 +09:00
Nick Schonning 49f1168050 Autofix Rubocop Style/RedundantBegin (#23703) 2023-02-19 07:09:40 +09:00
Nick Schonning 3f9d7419d1 Autofix Rubocop Style/Lambda (#23696) 2023-02-18 12:39:00 +01:00
Nick Schonning 793f8c7dd5 Autofix Rubocop Style/IfUnlessModifier (#23697) 2023-02-18 12:37:47 +01:00
Nick Schonning b7d4391c4d Autofix Rubocop Lint/AmbiguousOperatorPrecedence (#23681) 2023-02-18 04:30:23 +01:00
Nick Schonning 5973a58281 Enable Lint/RedundantCopDisableDirective (#23687) 2023-02-18 04:30:14 +01:00
Nick Schonning bc31a28361 Apply Rubocop Style/RedundantConstantBase (#23463) 2023-02-18 04:30:03 +01:00
Nick Schonning 8fa4163574 Enable Rubocop Style/NumericLiterals (#23647) 2023-02-18 11:05:57 +09:00
Nick Schonning 31ca76fe2c Run rubocop formatting except line length (#23632) 2023-02-18 06:56:20 +09:00
Nick Schonning 6417e06194 Upgrade to Stylelint 15 with Prettier (#23558) 2023-02-13 04:57:03 +01:00
Claire 20c6a153e4 Bump version to 4.1.0 (#23471)
* Bump version to 4.1.0

* Editorialize changelog some more and highlight API changes

* Update changelog
2023-02-10 22:21:23 +01:00
Nick Schonning 4eb077fc25 Apply Rubocop Rails/RootPublicPath (#23447) 2023-02-08 10:38:07 +01:00
Nick Schonning e0a02f8696 Apply Rubocop Style/NegatedIfElseCondition (#23451) 2023-02-08 07:07:36 +01:00
Nick Schonning 67a6f688ca Apply Rubocop Performance/StringIdentifierArgument (#23444) 2023-02-08 02:36:20 +01:00
Nick Schonning fbfa531daa Apply Rubocop Performance/BindCall (#23437) 2023-02-08 09:10:25 +09:00
Claire c9e86e57e8 Bump version to 4.1.0rc3 (#23384) 2023-02-03 16:39:38 +01:00
Claire b26ab9916d Bump version to 4.1.0rc2 (#23220) 2023-01-25 16:20:54 +01:00
Claire efa0acc165 Bump version to 4.1.0rc1 (#23112) 2023-01-20 14:19:12 +01:00
JT Olio 2cc9a94660 Add Storj DCS to cloud object storage options (#21929)
* Add Storj DCS to cloud object storage options

More explanation here: https://forum.storj.io/t/object-storage-provider-for-mastodon-instance/11464/37

* more help for which command to use
2023-01-18 17:47:49 +01:00
Claire 908f5f4c6e Add `tootctl domains purge` options to select subdomains and keep domain blocks (#22063)
* Add --include-subdomains option to tootctl domains purge

* Add support for '*.' subdomain wildcard patterns in `tootctl domains purge`

* Fix custom emojis deletion not following subdomain and URI options

* Change `tootctl domains purge` to not purge domain blocks unless --purge-domain-blocks is passed

* Refactor `tootctl domains purge`

* Add feedback on deleted domain blocks
2023-01-18 16:50:50 +01:00
Jeong Arm 4fd439e632 Implement tootctl accounts prune (#18397)
* Implement tootctl accounts prune

* Optimise query

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-01-13 22:34:16 +01:00
Claire 8cb9cfb0ab Add `tootctl accounts migrate` (#22330)
* Add tootctl accounts replay-migration

Fixes #22281

* Change `tootctl accounts replay-migration` to `tootctl accounts migrate`
2023-01-13 17:00:23 +01:00
Claire aadf68654a Fix incorrect env file generation in mastodon:setup (#23072)
Regression from #23012
2023-01-13 10:17:07 +01:00
Claire 614dd22095 Fix sanitizer parsing link text as HTML when stripping unsupported links (#22558) 2023-01-11 22:21:10 +01:00
Claire fdea93b6be Fix `$` not being escaped in `.env.production` file generated by `mastodon:setup` (#23012)
* Fix `$` not being escaped in `.env.production` file generated by `mastodon:setup`

* Improve robustness of dotenv escaping
2023-01-11 21:53:11 +01:00
Nick Schonning 675fff259a Expand Stylelint glob to include CSS files (#22469) 2023-01-05 13:42:13 +01:00
Dan Peterson 2a98aad36d Fix default S3_HOSTNAME used in mastodon:setup (#19932)
s3-us-east-1.amazonaws.com does not exist.

Co-authored-by: Effy Elden <effy@effy.space>
2022-12-15 16:38:51 +01:00
Claire 762efea31f Add --email and --dry-run options to `tootctl accounts delete` (#22328) 2022-12-15 14:52:50 +01:00
Evan 2d3b9084db Add command to remove avatar and header images of inactive remote accounts from the local database (#22149)
* Add tootctl subcommand media remove-profile-media

* Trigger workflows

* Correcting external linting

* External linting error

* External linting fix

* Merging with remove command

* Linting

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Remove saving a list of purged accounts

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-14 19:50:07 +01:00
Claire 053d3ba6ca Fix crash and incorrect behavior in tootctl domains crawl (#19004) 2022-12-13 20:02:32 +01:00
Claire 56e253553f Fix deprecation warning in `tootctl accounts rotate` (#22120) 2022-12-07 14:13:10 +01:00
Claire 360e6f7982 Fix pre-4.0 admin action logs (#22091)
* Fix BackfillAdminActionLogs post-deployment migration

* Improve migration tests

* Backfill admin action logs again
2022-12-06 23:38:03 +01:00
Claire daf52f6c69 Remove support for Ruby 2.6 (#21477)
As pointed out by https://github.com/mastodon/mastodon/pull/21297#discussion_r1028372193
at least one of our dependencies already dropped support for Ruby 2.6, and we
had removed Ruby 2.6 tests from the CI over a year ago (#16861).

So stop advertising Ruby 2.6 support, bump targeted version, and drop some
compatibility code.
2022-11-27 20:41:39 +01:00
Claire c863f76a62 Add logging for Rails cache timeouts (#21667)
* Reduce redis cache store connect timeout from default 20 seconds to 5 seconds

* Log cache store errors
2022-11-27 20:37:37 +01:00
Eugen Rochko a4d1745ffa Bump version to 4.0.2 (#20725) 2022-11-15 03:57:18 +01:00
Eugen Rochko eb1cb5b6d9 Bump version to 4.0.1 (#20696) 2022-11-14 22:21:14 +01:00
Eugen Rochko 4a307df02f Bump version to 4.0.0 (#20636) 2022-11-14 20:27:12 +01:00
Eugen Rochko c5da82cada Bump version to 4.0.0rc4 (#20634) 2022-11-14 08:50:14 +01:00
Claire abc0ea76e7 Fix index name in fix-duplicates task (#20632) 2022-11-14 08:33:48 +01:00
Eugen Rochko 8b98bebc17 Fix error when passing unknown filter param in REST API (#20626)
Fix #19156
2022-11-14 08:06:06 +01:00
Claire ac0ae9c83d Update fix-duplicates (#20502)
Fixes #19133
2022-11-13 21:01:38 +01:00
Arthur Isac bca22dc131 Copied Spaces support from packer .rake (#20573) 2022-11-13 20:58:40 +01:00
Eugen Rochko 791592c961 Bump version to 4.0.0rc3 (#20378) 2022-11-11 08:39:38 +01:00
Pierre Bourdon 4db2969701 blurhash_transcoder: prevent out-of-bound reads with <8bpp images (#20388)
The Blurhash library used by Mastodon requires an input encoded as 24
bits raw RGB data. The conversion to raw RGB using Imagemagick did not
previously specify the desired bit depth. In some situations, this leads
Imagemagick to output in a pixel format using less bpp than expected.
This then manifested as segfaults of the Sidekiq process due to
out-of-bounds read, or potentially a (highly noisy) memory infoleak.

Fixes #19235.
2022-11-11 07:45:16 +01:00
Yamagishi Kazutoshi 51e199a7c0 Fix `ENV` (#20377) 2022-11-11 01:33:32 +01:00
F 24af2c05bc Make enable_starttls configurable by envvars (#20321)
ENABLE_STARTTLS is designed to replace ENABLE_STARTTLS_AUTO by accepting
three values: 'auto' (the default), 'always', and 'never'. If
ENABLE_STARTTLS isn't provided, we fall back to ENABLE_STARTTLS_AUTO. In
this way, this change should be fully backwards compatible.

Resolves #20311
2022-11-10 21:06:21 +01:00
Eugen Rochko cdbd1a8bb3 Bump version to 4.0.0rc2 (#19831) 2022-11-06 06:59:56 +01:00
Eugen Rochko e75cab5d37 Add assets from Twemoji 14.0 (#19733) 2022-11-04 16:08:41 +01:00
Claire 05bb841077 Fix admin action logs page (#19649)
* Add tests

* Fix crash when trying to display orphaned action logs

* Add migration for older admin action logs
2022-11-03 16:06:42 +01:00
Claire 6d0c85d6a8 Fix mastodon:setup not setting the admin's role properly (#19670)
* Fix mastodon:setup not setting the admin's role properly

* Set contact username when creating admin account in mastodon:setup
2022-11-02 16:35:21 +01:00
pea-sys 066fcb0d41 png optimization(loss less) (#19630) 2022-11-01 15:06:52 +01:00
Eugen Rochko 79f5e4d088 Bump version to 4.0.0rc1 (#19473) 2022-10-28 00:26:02 +02:00
Eugen Rochko c92e2658e0 Add `--remove-role` option to `tootctl accounts modify` (#19477)
Fix #19152
2022-10-27 14:31:10 +02:00
Jeong Arm 32efaf0856 Fix Ambiguous SQL error on tootctl media refresh (#19206) 2022-09-20 23:50:19 +02:00
Claire b8853ddcb9 Improve error reporting and logging when processing remote accounts (#15605)
* Add a more descriptive PrivateNetworkAddressError exception class

* Remove unnecessary exception class to rescue clause

* Remove unnecessary include to JsonLdHelper

* Give more neutral error message when too many webfinger redirects

* Remove unnecessary guard condition

* Rework how “ActivityPub::FetchRemoteAccountService” handles errors

Add “suppress_errors” keyword argument to avoid raising errors in
ActivityPub::FetchRemoteAccountService#call (default/previous behavior).

* Rework how “ActivityPub::FetchRemoteKeyService” handles errors

Add “suppress_errors” keyword argument to avoid raising errors in
ActivityPub::FetchRemoteKeyService#call (default/previous behavior).

* Fix Webfinger::RedirectError not being a subclass of Webfinger::Error

* Add suppress_errors option to ResolveAccountService

Defaults to true (to preserve previous behavior). If set to false,
errors will be raised instead of caught, allowing the caller to be
informed of what went wrong.

* Return more precise error when failing to fetch account signing AP payloads

* Add tests

* Fixes

* Refactor error handling a bit

* Fix various issues

* Add specific error when provided Digest is not 256 bits of base64-encoded data

* Please CodeClimate

* Improve webfinger error reporting
2022-09-20 23:30:26 +02:00
luzpaz 45ddea2f0c Fix typos (#18604)
* Fix typos

Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,pixelx,ro`

* Follow-up typo fix
2022-08-28 17:44:34 +02:00
Eugen Rochko a0bb5674d9 Add admin API for managing canonical e-mail blocks (#19067) 2022-08-28 03:31:54 +02:00
Jeong Arm 8c1ea0976b Add '--days' option to tootctl media refresh (#18425)
* Add '--days' option to tootctl media refresh

* Fix undefined scope
2022-08-25 04:40:17 +02:00
Brayd 71f41ca363 Minimal adjustments to the short description (#18001)
Minimal adjustments have been made to the short description so that it logically follows the long description
2022-08-13 15:41:12 +02:00
Eugen Rochko e164d6a687 Add customizable user roles (#18641)
* Add customizable user roles

* Various fixes and improvements

* Add migration for old settings and fix tootctl role management
2022-07-05 02:41:40 +02:00
Claire 90e5a9bd98 Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Eugen Rochko c3f7041715 Fix wrong aspect ratio of logo in icons (#18639) 2022-06-11 20:32:02 +02:00
Eugen Rochko 03d2cb5146 Change brand color and logotypes (#18592)
- Add rake task for generating Apple/Android icons and favicons from SVG
- Add rake task for generating PNG icons and logos for e-mails from SVG
- Remove obsolete Microsoft icons and configuration
- Remove PWA shortcut icons
2022-06-09 22:25:23 +02:00
Claire a0556ea3f1 Remove dependency on running Redis server for db:setup (#18560) 2022-06-01 19:23:31 +02:00
Eugen Rochko 5b1c0d21ab Bump version to 3.5.3 (#18530) 2022-05-26 23:26:15 +02:00
Eugen Rochko eda9c41ed8 Change algorithm of `tootctl search deploy` to improve performance (#18463) 2022-05-22 22:16:43 +02:00
Eugen Rochko f4cab3ebad Change search indexing to use batches to minimize resource usage (#18451) 2022-05-18 23:29:14 +02:00
Claire 5cc4d2ed8b Fix redis configuration not being changed by mastodon:setup (#18383)
Fixes #18342
2022-05-09 23:19:11 +02:00
Claire ea99b93364 Bump version to 3.5.2 (#18295)
* Bump version to 3.5.2

* Change some entries to be more clear

* Add some extra notes

* Fix line wrap

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-05-04 00:57:42 +02:00
Claire a6e431baca Add ability to set approval-based registration through tootctl (#18248)
Fixes #18235

Add `tootctl settings registrations approved` with
optional `--require-reason` switch.
2022-05-02 17:41:34 +02:00
Eugen Rochko a8e694233c Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Gaelan Steele 7a00957cbb Fix light-mode emoji borders. (#18131) 2022-04-29 19:23:03 +02:00
Eugen Rochko 553889bc7c Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire ef902a72cf Fix instance actor being incorrectly created when running migrations (#18109)
* Add migration test about instance actor key

* Fix old migration

* Work around incorrect database state
2022-04-26 21:22:09 +02:00
Eugen Rochko 23b61a753a Bump version to 3.5.1 (#18000) 2022-04-08 21:57:24 +02:00
0x2019 e681e8d14c Fix error resposes for `from` search prefix (#17963)
* Fix error responses in `from` search prefix (addresses mastodon/mastodon#17941)

Using unsupported prefixes now reports a 422; searching for posts from an
account the instance is not aware of reports a 404. TODO: The UI for this
on the front end is abysmal.

Searching `from:username@domain` now succeeds when `domain` is the local
domain; searching `from:@username(@domain)?` now works as expected.

* Remove unused methods on new Error classes as they are not being used

Currently when `raise`d there are error messages being supplied, but
this is not actually being used. The associated `raise`s have been
edited accordingly.

* Remove needless comments

* Satisfy rubocop

* Try fixing tests being unable to find AccountFindingConcern methods

* Satisfy rubocop

* Simplify `from` prefix logic

This incorporates @ClearlyClaire's suggestion (see
https://github.com/mastodon/mastodon/pull/17963#pullrequestreview-933986737).

Accepctable account strings in `from:` clauses are more lenient than
before this commit; for example, `from:@user@example.org@asnteo +cat`
will not error, and return posts by @user@example.org containing the
word "cat". This is more consistent with how Mastodon matches mentions
in statuses. In addition, `from` clauses will not be checked for
syntatically invalid usernames or domain names, simply 404ing when
`Account.find_remote!` raises ActiveRecord::NotFound.

New code for this PR that is no longer used has been removed.
2022-04-08 21:21:49 +02:00
Eugen Rochko b7a6bba0b7 Fix cookies secure flag being set when served over Tor (#17992) 2022-04-08 12:47:18 +02:00
Claire c1b85d1f99 Fix migration error handling (#17991) 2022-04-07 20:46:30 +02:00
Claire 2b7d558807 Fix error re-running some migrations if they get interrupted at the wrong moment (#17989) 2022-04-07 20:17:49 +02:00
Eugen Rochko f982d56b4e Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Holger fb3fc6cab4 fix: `s3_force_single_request` not parsed (#17922) 2022-04-01 23:56:23 +02:00
Eugen Rochko a7a8594f2e Bump version to 3.5.0 (#17911) 2022-03-30 14:52:37 +02:00
Eugen Rochko 91dd5ce8b7 Bump version to 3.5.0rc3 (#17876) 2022-03-26 04:29:36 +01:00
Eugen Rochko bd42dab872 Bump version to 3.5.0rc2 (#17855) 2022-03-26 02:54:11 +01:00
Claire ef643e9224 Fix tootctl email_domain_blocks add (#17842)
Fixes #17831
2022-03-21 19:10:09 +01:00
Claire b167133918 Fix wrong language code for Kurdish languages (#17812) 2022-03-17 01:37:03 +01:00
Eugen Rochko de14ccc13b Bump version to 3.5.0rc1 (#17618)
* Bump version to 3.5.0rc1

* Various fixes and improvements

* Update AUTHORS.md

* Various fixes and improvements

* Update README.md
2022-03-15 08:16:45 +01:00
Claire 503f16a333 Update fix-duplicates maintenance task (#17731)
* Update fix-duplicates task to 2022_02_10_153119

Also add support for Appeal to AccountMerging#merge_with!

* Update fix-duplicates task to 2022_03_07_094650

* Update fix-duplicates task to 2022_03_09_213005

* Update fix-duplicates task to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060626

* Update fix-duplicates script to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060706

* Update fix-duplicates task to 2022_03_10_060959

* Silence CodeClimate
2022-03-12 08:33:11 +01:00