Commit Graph

1321 Commits (92d606bc98ae8d59b2e59ef91ee355044f7174bc)

Author SHA1 Message Date
Claire 2fe47bc7db Remove dead code and refactor status threading code (#20357)
* Remove dead code

* Remove unneeded/broken parameters and refactor descendant computation
2022-11-10 22:30:00 +01:00
Eugen Rochko faaa47c421 Change verification to only work for https links (#20304)
Fix #20242
2022-11-10 21:09:03 +01:00
Eugen Rochko ea7cc10811 Change link verification to ignore IDN domains (#20295)
Fix #3833
2022-11-10 06:27:45 +01:00
James Tucker c4ff2640c1 Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Eugen Rochko 17881148e4 Revert filtering public timelines by locale by default (#20294) 2022-11-10 05:34:42 +01:00
Eugen Rochko 3c80a6e445 Fix being able to spoof link verification (#20217)
- Change verification to happen in `default` queue
- Change verification worker to only be queued if there's something to do
- Add `link` tags from metadata fields to page header of profiles
2022-11-09 08:24:21 +01:00
Postmodern 97ee940b1a Micro-optimization: use `if`/`else` instead of `Array#compact` and `Array#min` (#19906)
* Technically `if`/`else` is faster than using `[value1, value2].compact.min` to find the lesser of two values, one of which may be `nil`.
2022-11-08 03:50:47 +01:00
Claire 5778ba1028 Fix validation error in SynchronizeFeaturedTagsCollectionWorker (#20018)
* Fix followers count not being updated when migrating follows

Fixes #19900

* Fix validation error in SynchronizeFeaturedTagsCollectionWorker

Also saves remote user's chosen case for hashtags

* Limit remote featured tags before validation
2022-11-07 22:35:53 +01:00
Postmodern 41d43a1e1f Micro-optimization: only split `acct` into two Strings (#19901)
* Since `acct` is split by `@` and assigned to `username` and `domain`, we only need to split `acct` into two Strings.
2022-11-07 16:17:55 +01:00
nightpool 149a8246d2 Skip Webfinger cache during migrations as well (#19883) 2022-11-07 03:31:38 +01:00
Eugen Rochko b2cf46ee21 Fix not using GIN index for account search queries (#19830) 2022-11-06 06:16:34 +01:00
Claire 4b0e72b2b7 Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
Eugen Rochko 6b2ae9e5a5 Fix featured tags not saving preferred casing (#19732) 2022-11-04 16:08:29 +01:00
Claire a5e1f56347 Fix inaccurate admin log entry for re-sending confirmation e-mails (#19674)
Fixes #19593
2022-11-02 18:50:21 +01:00
txt-file a2703b8852 Add support for AVIF uploads (#19647) 2022-11-01 22:08:41 +01:00
Eugen Rochko ca1dec3831 Add support for HEIC uploads (#19618) 2022-11-01 16:26:25 +01:00
Eugen Rochko 529fe25add Change post-processing to be deferred only for large media types (#19617) 2022-11-01 15:27:58 +01:00
Eugen Rochko 0c0daf7142 Change max. thumbnail dimensions to 640x360px (360p) (#19619) 2022-11-01 13:01:39 +01:00
Eugen Rochko fdfe1eae4a Remove language filtering from hashtag timelines (#19563) 2022-10-30 21:29:23 +01:00
Eugen Rochko 3455b9001f Add reputation and followers score boost to SQL-only account search (#19251) 2022-10-30 13:23:05 +01:00
Eugen Rochko eedf46b243 Fix account action type validation (#19476)
* Fix account action type validation

Fix #19143

* Fix #19145

* Fix code style issues
2022-10-30 02:44:32 +02:00
Eugen Rochko c129275c25 Fix not being able to input featured tag with `#` (#19535) 2022-10-30 02:43:20 +02:00
Eugen Rochko 6099eecf57 Fix account migration form ever using outdated account data (#18429) 2022-10-29 01:31:45 +02:00
Claire 9cb523d3be Change admin announcement edition interface to use datetime-local (#18321)
* Change admin announcement edition interface to use datetime-local

* Dynamically set announcement stop date as required if start date is set, set minimum date for stop date

* Change `all_day` to not be bound to presence of time-range

* Add pattern and placeholder as minimal fallback for browsers not supporting datetime-local

* Display datetime-local inputs as local time

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-10-28 12:56:32 +02:00
Eugen Rochko bf1bc15575 Fix using wrong policy on status-related actions in admin UI (#19490) 2022-10-28 00:48:30 +02:00
Eugen Rochko c71443ef1d Remove unused method `searchable?` on accounts (#19489)
It called the wrong methods, but nothing uses it
2022-10-27 19:30:08 +02:00
Eugen Rochko 35b5120c5d Fix notifications about deleted reports not being also deleted (#19475)
* Fix notifications about deleted reports not being also deleted

* Fix notification with empty report crashing web UI

Fix #18909
2022-10-27 02:10:54 +02:00
zunda 3697b51abc Store integer settings as integer (#19478) 2022-10-26 22:14:07 +02:00
Eugen Rochko 3e5588bf70 Add ability to view previous edits of a status in admin UI (#19462)
* Add ability to view previous edits of a status in admin UI

* Change moderator access to posts to be controlled by a separate policy
2022-10-26 13:42:29 +02:00
Eugen Rochko 7b0b8221a5 Fix IP blocks not having a unique index (#19456) 2022-10-25 21:43:44 +02:00
Takeshi Umeda 64b46b9ae5 Change featured tag updates to add/remove activity (#19409)
* Change featured tag updates to add/remove activity

* Fix to check for the existence of feature tag

* Rename service and worker

* Merge AddHashtagSerializer with AddSerializer

* Undo removal of sidekiq_options
2022-10-22 18:30:55 +02:00
Takeshi Umeda 17648d9bf6 Fix duplicate featured tags (#19403)
* Fix duplicate featured tags

* Add unique tag name validator

* Fix error message
2022-10-22 14:30:59 +02:00
Takeshi Umeda 46755e25a2 Add featured tag add/remove activity handler (#19408) 2022-10-22 11:49:41 +02:00
Eugen Rochko 2130895196 Change settings area to be separated into categories in admin UI (#19407)
And update all descriptions
2022-10-22 11:44:41 +02:00
Eugen Rochko 0fdfbe555e Change public accounts pages to mount the web UI (#19319)
* Change public accounts pages to mount the web UI

* Fix handling of remote usernames in routes

- When logged in, serve web app
- When logged out, redirect to permalink
- Fix `app-body` class not being set sometimes due to name conflict

* Fix missing `multiColumn` prop

* Fix failing test

* Use `discoverable` attribute to control indexing directives

* Fix `<ColumnLoading />` not using `multiColumn`

* Add `noindex` to accounts in REST API

* Change noindex directive to not be rendered by default before a route is mounted

* Add loading indicator for detailed status in web UI

* Fix missing indicator appearing while account is loading in web UI
2022-10-20 14:35:29 +02:00
Takeshi Umeda fd61882f1a Add synchronization of remote featured tags (#19380)
* Add LIMIT of featured tag to instance API response

* Add featured_tags_collection_url to Account

* Add synchronization of remote featured tags

* Deliver update activity when updating featured tag

* Remove featured_tags_collection_url

* Revert "Add featured_tags_collection_url to Account"

This reverts commit cff349fc27b104ded2df6bb5665132dc24dab09c.

* Add hashtag sync from featured collections

* Fix tag name normalize

* Add target option to fetch featured collection

* Refactor fetch_featured_tags_collection_service

* Add LIMIT of featured tag to v1/instance API response
2022-10-20 09:15:52 +02:00
Eugen Rochko ada6590031 Fix trending statuses returning more than one post by the same author (#19349) 2022-10-14 01:44:23 +02:00
Eugen Rochko e376f33207 Change about page to be mounted in the web UI (#19345) 2022-10-13 14:42:37 +02:00
Eugen Rochko dd1dc1743c Add image processing and generate blurhash for server thumbnail (#19348)
Remove separate server hero setting
2022-10-13 11:29:19 +02:00
Yamagishi Kazutoshi c6fb0a7555 Fix missing `skip_review?` (#19335) 2022-10-10 08:03:19 +02:00
Eugen Rochko b24b85b63f Add support for language preferences for trending statuses and links (#18288) 2022-10-08 16:45:40 +02:00
Eugen Rochko b1d0cb83eb Fix privacy policy being empty if custom setting exists but is empty (#19318) 2022-10-08 08:34:00 +02:00
Eugen Rochko 378e00b59c Change privacy policy to be rendered in web UI, add REST API (#19310)
Source string no longer localized, Markdown instead of raw HTML
2022-10-08 06:01:11 +02:00
Eugen Rochko ce7d058a91 Remove setting that disables account deletes (#17683) 2022-10-06 10:16:47 +02:00
Eugen Rochko 85a10f182e Change public timelines to be filtered by current locale by default (#19291)
In the absence of an opt-in to multiple specific languages in the
preferences, it makes more sense to filter by the user's presumed
language only (interface language or `lang` override)
2022-10-05 03:48:06 +02:00
Eugen Rochko 58c367fb47 Remove code for rendering public and hashtag timelines outside the web UI (#19257) 2022-10-04 20:13:46 +02:00
Eugen Rochko 195560b03a Fix content retention policy settings not accepting a blank value (#19248) 2022-09-29 01:15:09 +02:00
Eugen Rochko f6f08aa610 Fix language dropdown sometimes not appearing in web UI (#19246)
When user has no locale preference saved (such as never changing it
from the default), the preferred posting language is nil, and
the dropdown is not visible
2022-09-28 01:02:15 +02:00
Eugen Rochko 7a482567f8 Add retention policy for cached content and media (#19232) 2022-09-27 03:08:19 +02:00
Eugen Rochko f4e91041b2 Add ability to filter followed accounts' posts by language (#19095) 2022-09-20 23:51:21 +02:00
Eugen Rochko ecddc06474 Change "Allow trends without prior review" setting to include statuses (#17977)
* Change "Allow trends without prior review" setting to include posts

* Fix i18n-tasks
2022-08-28 04:00:39 +02:00
Eugen Rochko e7cc67d505 Add admin API for managing e-mail domain blocks (#19066) 2022-08-28 03:37:55 +02:00
Eugen Rochko a0bb5674d9 Add admin API for managing canonical e-mail blocks (#19067) 2022-08-28 03:31:54 +02:00
Eugen Rochko 71f59e78b8 Add admin API for managing IP blocks (#19065) 2022-08-27 20:56:47 +02:00
Eugen Rochko 7300beadd9 Remove digest e-mails (#17985)
* Remove digest e-mails

* Remove digest-related code
2022-08-25 23:38:22 +02:00
Eugen Rochko 1206e691bd Add ability to select all accounts matching search for batch actions (#19053) 2022-08-25 23:33:34 +02:00
Eugen Rochko f9ebc320ed Add audit log entries for user roles (#19040)
* Refactor audit log schema

* Add audit log entries for user roles
2022-08-25 20:39:40 +02:00
Claire 9d14683f86 Add option for EMAIL_DOMAIN_DENYLIST/EMAIL_DOMAIN_ALLOWLIST to apply after confirmation (#18642)
Fixes #18620
2022-08-25 04:31:10 +02:00
Claire 522fbf9d13 Add ability to filter individual posts (#18945)
* Add database table for status-specific filters

* Add REST endpoints, entities and attributes

* Show status filters in /filters interface

* Perform server-side filtering for individual posts filters

* Fix filtering on context mismatch

* Refactor `toServerSideType` by moving it to its own module

* Move loupe and delete icons to their own module

* Add ability to filter individual posts from WebUI

* Replace keyword list by warnings (expired, context mismatch)

* Refactor server-side filtering code

* Add tests
2022-08-25 04:27:47 +02:00
Eugen Rochko e642516264 Change e-mail domain blocks to match subdomains of blocked domains (#18979) 2022-08-24 19:00:55 +02:00
Eugen Rochko b2e1224baa Add ability to block sign-ups from IP (#19037) 2022-08-24 19:00:37 +02:00
Eugen Rochko abb4dca69b Fix unicode regression in #18809 (#18863) 2022-07-22 03:17:56 +02:00
Eugen Rochko 265c09fef1 Add ability to follow hashtags (#18809) 2022-07-17 13:49:29 +02:00
Claire 0e504e67f3 Add support for editing labelling of one's own role (#18812)
Still disallow edition of rank or permissions
2022-07-17 13:37:30 +02:00
Eugen Rochko 57cf1c5a98 Fix nil error when rendering featured hashtags on profile (#18808)
Regression from #18795
2022-07-14 01:23:10 +02:00
Eugen Rochko 38d04135bf Change how hashtags are normalized (#18795)
* Change how hashtags are normalized

* Fix tests
2022-07-13 15:03:28 +02:00
Eugen Rochko 89ef936126 Change custom emoji file size limit from 50 KB to 256 KB (#18788) 2022-07-09 22:07:17 +02:00
Eugen Rochko e164d6a687 Add customizable user roles (#18641)
* Add customizable user roles

* Various fixes and improvements

* Add migration for old settings and fix tootctl role management
2022-07-05 02:41:40 +02:00
Jeong Arm 386980c196 Support audio/vnd.wave (#18737)
See: https://datatracker.ietf.org/doc/html/rfc2361
And Misskey uses this mime type for wav file.
2022-06-28 19:49:35 +02:00
Claire 90e5a9bd98 Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Eugen Rochko e6eb0a29b9 Add notifications for new reports (#18697) 2022-06-27 09:30:15 +02:00
Claire d0cc795d0f Add /api/v1/admin/domain_allows (#18668)
- `GET /api/v1/admin/domain_allows` lists allowed domains
- `GET /api/v1/admin/domain_allows/:id` shows one by ID
- `DELETE /api/v1/admin/domain_allows/:id` deletes a given domain from the list
  of allowed domains
- `POST /api/v1/admin/domain_allows` to allow a new domain:
  if that domain is already allowed, the existing DomainAllow will be returned
2022-06-23 23:12:01 +02:00
Eugen Rochko 0eb2db6b52 Add administrative webhooks (#18510)
* Add administrative webhooks

* Fix error when webhook is deleted before delivery worker runs
2022-06-09 21:57:36 +02:00
Claire a0556ea3f1 Remove dependency on running Redis server for db:setup (#18560) 2022-06-01 19:23:31 +02:00
Claire 9f3bf956ad Add /api/v1/admin/domain_blocks (#18247)
* Add /api/v1/admin/domain_blocks

Fixes #18140

- `GET /api/v1/admin/domain_blocks` lists domain blocks
- `GET /api/v1/admin/domain_blocks/:id` shows one by ID
- `DELETE /api/v1/admin/domain_blocks/:id` deletes a given domain block
- `POST /api/v1/admin/domain_blocks` to create a new domain block:
  if it conflicts with an existing one, returns an error with
  an attribute `existing_domain_block` with the rendered domain block

* Simplify conflict handling as suggested in review
2022-06-01 17:31:36 +02:00
Alexander Ivanov a59e8245f5 Add support for webp uploads (#18506) 2022-05-27 20:06:40 +02:00
Claire 73c8032178 Remove unused `filtered_languages` column (#18533)
* Remove unused `filtered_languages` column

Fixes #18522

* Fix tests
2022-05-27 20:05:22 +02:00
Eugen Rochko 4bb50e32e4 Fix being able to report otherwise inaccessible statuses (#18528) 2022-05-26 22:08:02 +02:00
Eugen Rochko 35ebb5571e Fix follower and other counters being able to go negative (#18517) 2022-05-26 20:32:48 +02:00
Eugen Rochko 5be3d071f2 Fix regression in `tootctl search deploy` caused by unloaded attribute (#18514) 2022-05-26 18:05:47 +02:00
Claire f5e6c776c1 Change unapproved and unconfirmed account to not be accessible in the REST API (#17530)
* Change unapproved and unconfirmed account to not be accessible in the REST API

* Change Account#searchable? to reject unconfirmed and unapproved users

* Disable search for unapproved and unconfirmed users in Account.search_for

* Disable search for unapproved and unconfirmed users in Account.advanced_search_for

* Remove unconfirmed and unapproved accounts from Account.searchable scope

* Prevent mentions to unapproved/unconfirmed accounts

* Fix some old tests for Account.advanced_search_for

* Add some Account.advanced_search_for tests for existing behaviors

* Add some tests for Account.search_for

* Add Account.advanced_search_for tests unconfirmed and unapproved accounts

* Add Account.searchable tests

* Fix Account.without_unapproved scope potentially messing with previously-applied scopes

* Allow lookup of unconfirmed/unapproved accounts through /api/v1/accounts/lookup

This is so that the API can still be used to check whether an username is free
to use.
2022-05-26 15:50:33 +02:00
Claire e220fcf1d4 Fix unnecessary query on status creation (#17901) 2022-05-26 00:20:30 +02:00
Claire 2baba513d9 Fix warning an account outside of a report closing all reports for that account (#18387)
* Fix warning an account outside of a report closing all reports for that account

* Make it clear what actions solve other reports

* Revert "Make it clear what actions solve other reports"

This reverts commit ad006de821f72e75480701298d13f0945b509059.
2022-05-23 20:38:29 +02:00
Eugen Rochko eda9c41ed8 Change algorithm of `tootctl search deploy` to improve performance (#18463) 2022-05-22 22:16:43 +02:00
Eugen Rochko 9d160dae82 Fix preferred posting language returning unusable value in REST API (#18428) 2022-05-16 19:13:36 +02:00
Eugen Rochko f1707c6d50 Refactor how Redis locks are created (#18400)
* Refactor how Redis locks are created

* Fix autorelease duration on account deletion lock
2022-05-13 00:02:35 +02:00
Claire 0e0b96b8e7 Fix account warnings not being recorded in audit log (#18338)
* Fix account warnings not being recorded in audit log

Fixes #18334

* Only record warnings if they are not associated to another action
2022-05-06 21:40:20 +02:00
Claire a31975d369 Fix possible crash when a post references an invalid media attachment (#18211) 2022-05-01 00:55:26 +02:00
Eugen Rochko 1cd4518c29 Remove IP matching from e-mail domain blocks (#18190)
Clear out e-mail domain blocks created from automatically resolved DNS records
2022-04-29 23:27:03 +02:00
Eugen Rochko a8e694233c Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Eugen Rochko 2fb1c02eb7 Change trending statuses to only show one status from each account (#18181)
Calculate trends in temporary sets to avoid having to manage items
that go below the decay threshold while not having any moments
where a half-processed set is accessible to end-users
2022-04-29 22:42:42 +02:00
Eugen Rochko 4543f42811 Change half-life of trending status scores from 6 hours to 2 hours (#18182) 2022-04-29 11:46:59 +02:00
Eugen Rochko 553889bc7c Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire ef902a72cf Fix instance actor being incorrectly created when running migrations (#18109)
* Add migration test about instance actor key

* Fix old migration

* Work around incorrect database state
2022-04-26 21:22:09 +02:00
Jeong Arm 4af30ac16b Let votes statuses are also searchable (#18070) 2022-04-23 21:47:27 +02:00
Claire 5e13634c6b Fix crash in alias settings page (#18004) 2022-04-09 20:11:06 +02:00
Eugen Rochko d2265cab26 Fix dangling language-specific trends (#17997)
- Change score half-life for trending statuses from 2 to 6 hours
- Change score threshold for trimming old items from 1 to 0.3
2022-04-08 19:35:31 +02:00
Eugen Rochko 4ba66d256a Change e-mail notifications to only be sent when recipient is offline (#17984)
* Change e-mail notifications to only be sent when recipient is offline

Change the default for follow and mention notifications back on

* Add preference to always send e-mail notifications

* Change wording
2022-04-08 18:03:31 +02:00
Eugen Rochko e3be46c743 Fix trends returning less results per page when filtered in REST API (#17996)
- Change filtering and pagination to occur in SQL instead of Redis
- Change rank/score displayed on trends in admin UI to be locale-specific
2022-04-08 17:10:53 +02:00
Eugen Rochko aa6bc541d3 Fix pagination header on empty trends responses in REST API (#17986) 2022-04-07 18:06:15 +02:00