Commit Graph

177 Commits (65646a3e891a212e66afec45cf93a75554d6bc26)

Author SHA1 Message Date
Claire 6b89ab368c Merge commit '4ae247954387b530b532eaa89ca017a17cbe63bd' into glitch-soc/merge-upstream
Conflicts:
- `streaming/index.js`:
  Upstream has added a parameter to `streamFrom`, while glitch-soc had an
  extra parameter.
  Ported upstream changes.
2023-09-19 12:59:23 +02:00
Emelia Smith c1b1cbb58a Add additional metrics for streaming (#26945) 2023-09-19 12:25:30 +02:00
Claire 63969b15b3 Merge commit '8bffce5f015ace5b73d5a8aa746c02e374d927bb' into glitch-soc/merge-upstream
Conflicts:
- `app/models/form/admin_settings.rb`:
  Upstream added the notion of overriden settings, while we had extra code for
  pseudo-settings (only used to combine flavour and skin in a single select
  field).
  Ported upstream changes.
- `config/i18n-tasks.yml`:
  Upstream added `simple_form.overridden` to `ignore_unused`,
  we had `simple_form.glitch_only`.
  Added `simple_form.glitch_only` as well.
2023-09-02 14:44:47 +02:00
Gabriel Simmer 8bffce5f01 Move to ioredis for streaming (#26581)
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
2023-09-01 17:44:28 +02:00
Claire 6fba17f6f3 Merge commit 'b4a95b46cd4399420670c8edf1807ad3a6e979ec' into glitch-soc/merge-upstream 2023-08-24 20:55:28 +02:00
Emelia Smith c2ef995baa Fix: support both DATABASE_URL and DB_PASS (#26295) 2023-08-18 15:05:35 +02:00
Claire 33024cca51 Merge commit '13a4a81f022a1b944dd65acb1041dc76093b0901' into glitch-soc/main
Conflicts:
- `.rubocop_todo.yml`:
  Upstream regenerated this file, glitch-soc had a specific ignore.
2023-08-13 18:47:15 +02:00
Emelia Smith b92b2b1298 Change streaming `/metrics` to include additional metrics (#26299) 2023-08-04 16:11:30 +02:00
Claire 84bc751433 Merge commit '6a19d5ce9942686e0cda1a9decec349d0a1f4e26' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/authorize_interactions_controller.rb`:
  Small conflict due to our theming system.
- `streaming/index.js`:
  Upstream refactored part of the streaming server.
  We had some extra logic for handling local-only posts.
  Applied the refactor.
2023-07-30 16:11:55 +02:00
Claire 68d080177a Merge commit 'eb1139fd42203ad966810f7f7ed6b55949ac7e8a' into glitch-soc/merge-upstream 2023-07-30 15:54:34 +02:00
Renaud Chaput 6a19d5ce99 Fix missing return values in streaming (#26233) 2023-07-28 19:11:58 +02:00
Claire d21fa55246 Swap debug statements in streaming server (#26231) 2023-07-28 17:59:31 +02:00
Emelia Smith 813acdb431 Fix: Streaming server memory leak in HTTP EventSource cleanup (#26228) 2023-07-28 12:06:29 +02:00
Emelia Smith 0ee5d7a804 Refactor streaming's filtering logic & improve documentation (#26213) 2023-07-27 15:38:18 +02:00
Claire eb1139fd42 Fix wrong filters sometimes applying in streaming (#26159) 2023-07-27 15:12:10 +02:00
Claire f101b67a45 Merge commit '239ee4f5d10aaa30b52cc59a58b2dc33fea5615f' into glitch-soc/merge-upstream 2023-06-18 10:36:14 +02:00
Claire 1c25f35ddf Merge commit '23c3e0985346406f312b88c659cd9fad63cb6060' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Conflict because glitch-soc does not have a theme selector here.
2023-06-10 18:42:29 +02:00
Emelia Smith 34c9ee4a66 Add logging of websocket send errors (#25280) 2023-06-10 18:35:57 +02:00
Emelia Smith 90070920bf Fix logging of messages that are binary before closing their connection (#25361) 2023-06-10 18:24:37 +02:00
Claire c51e8de4ad Merge commit 'bc7e7185ac0f0c10ebf354b738b8dfb4e8e43dcb' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Upstream fixed a translation bug in the theme selector that is absent from
  glitch-soc due to our different theming system.
  Discarded upstream changes.
- `streaming/index.js`:
  Upstream changed the signature of a function to change its return type.
  This is not a real conflict, the conflict being caused by an extra
  argument in glitch-soc's code.
  Applied upstream's change while keeping our extra argument.
2023-06-10 17:18:36 +02:00
Emelia Smith bc7e7185ac Fix performance of streaming by parsing message JSON once (#25278) 2023-06-09 19:29:16 +02:00
Claire fa16de4a4e Merge commit '7b5a2748360753a903108a6e57caaee3194a5750' into glitch-soc/merge-upstream
Conflicts:
- `.eslintrc.js`:
  Upstream moved a configuration block in which we had added a glitch-only
  path.
  Moved the configuration block as upstream did.
- other files:
  Upstream reordered imports, and those files had different ones.
  Kept our version and reordered imports using the same rules.
2023-05-28 15:54:31 +02:00
Renaud Chaput 7b5a274836 Enforce import order with ESLint (#25096) 2023-05-23 17:15:17 +02:00
Claire 2aaf3528ca Merge commit 'd548faf69321260ab4c2a16759ca77773189126a' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/styles/mastodon/forms.scss`:
  Conflict because we ran eslint autofix on upstream files.
- `config/initializers/content_security_policy.rb`:
  Code style changes but we have a different version.
  Kept our version.
- `streaming/index.js`:
  Upstream fixed a typo close to glitch-soc-only code.
  Applied upstream's changes.
2023-05-08 15:28:36 +02:00
Claire e390c26429 Fix some of the Javascript linting issues, as well as bugs and unneeded divergences with upstream (#2208)
* Run eslint --fix

* Fix linting issues in video player and reduce divergences with upstream

This includes a behavior change of not auto-looping videos anymore. I don't
remember loops being ever intended, and they have been removed from upstream
a while ago, but we somehow missed the change.

* Fix lint issues in `app/javascript/flavours/glitch/selectors/index.js`

Those were basically caused by dead code that isn't present upstream, so
that brings us closer to upstream as well.

* Fix linting issue and bug in streaming/index.js

* Fix linting issues in config/webpack/shared.js

* Fix unused import in flavours/glitch/features/ui/index.js

* Fix linting issues and reduce divergences from upstream in flavours/glitch/features/ui/components/video_modal.jsx

* Fix linting issues in flavours/glitch/reducers

* Fix linting issues in glitch-soc onboarding modal

* Fix linting issues in flavours/glitch/features/ui/components/navigation_panel.jsx

* Remove dead code for unused local setting navbar_under

* Fix various linting issues

* Fix linting issues in flavours/glitch/components/scrollable_list.jsx and reduce divergences with upstream
2023-05-07 18:22:25 +02:00
Nick Schonning 0224c3c0d9 Add eslint-plugin-jsdoc (#24719) 2023-04-30 02:29:54 +02:00
Claire 498de92bc8 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Updated upstream, removed in glitch-soc to disable noise.
  Kept removed.
- `CODE_OF_CONDUCT.md`:
  Upstream updated to a new version of the covenant, but I have not read it
  yet, so kept unchanged.
- `Gemfile.lock`:
  Not a real conflict, one upstream dependency updated textually too close to
  the glitch-soc only `hcaptcha` dependency.
  Applied upstream changes.
- `app/controllers/admin/base_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/application_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/disputes/base_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/relationships_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/statuses_cleanup_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/helpers/application_helper.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.jsx`:
  Upstream added a highlight animation for onboarding, while we changed the
  max character limit.
  Applied our local changes on top of upstream's new version.
- `app/views/layouts/application.html.haml`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `stylelint.config.js`:
  Upstream added ignore paths, glitch-soc had extra ignore paths.
  Added the same paths as upstream.
2023-04-29 10:44:56 +02:00
Emelia Smith dca2892276 Remove clustering from streaming API (#24655) 2023-04-26 11:37:51 +02:00
Claire f39d9ab256 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream switched to pushing to both DockerHub and GitHub Container
  Repository, while glitch-soc was already pushing to the latter only.
  Updated our configuration to be slightly more consistent with upstream's
  naming and styling, but kept our behavior.
- `Gemfile.lock`:
  Updated dependencies textually too close to glitch-soc only hcaptcha
  dependency.
  Updated dependencies as upstream did.
- `README.md`:
  Upstream updated its README, but we have a completely different one.
  Kept our README, though it probably should be reworked at some point.
- `app/views/auth/sessions/two_factor.html.haml`:
  Minor style fix upstream that's on a line glitch-soc removed because
  of its different theming system.
  Kept our file as is.
- `spec/controllers/health_controller_spec.rb`:
  This file apparently did not exist upstream, upstream created it with
  different contents but it is functionally the same.
  Switched to upstream's version of the file.
- `spec/presenters/instance_presenter_spec.rb`:
  Upstream changed the specs around `GITHUB_REPOSITORY`, while glitch-soc
  had its own code because it's a fork and does not have the same default
  source URL.
  Took upstream's change, but with glitch-soc's repo as the default case.
- `yarn.lock`:
  Upstream dependencies textually too close to a glitch-soc only one.
  Updated dependencies as upstream did.
2023-03-15 09:16:10 +01:00
Claire dcde901b4d Disable anonymous access to the streaming API (#23989) 2023-03-06 21:00:10 +01:00
Claire a6363c3a2a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream README has been changed, but we have a completely different one.
  Kept our `README.md`.
- `lib/sanitize_ext/sanitize_config.rb`:
  Upstream added support for more incoming HTML tags (a large subset of what
  glitch-soc accepts).
  Change the code style to match upstream's but otherwise do not change our
  code.
- `spec/lib/sanitize_config_spec.rb`:
  Upstream added support for more incoming HTML tags (a large subset of what
  glitch-soc accepts).
  Kept our version, since the tests are mostly glitch-soc's, except for cases
  which are purposefuly different.
2023-03-05 20:46:56 +01:00
Eugen Rochko 8f9f5dd620 Fix streaming API not being usable without `DATABASE_URL` (#23960) 2023-03-05 01:52:12 +01:00
Ramūns Usovs c92301e942 Allow streaming to connect to postgress with self-signed certs (#21431) 2023-03-03 21:01:18 +01:00
Claire 92d1e05c3e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.prettierignore`:
  Upstream added a line at the end, glitch-soc had extra entries at the end.
  Added upstream's new line before glitch-soc's.
- `Gemfile.lock`:
  Upstream updated dependencies while glitch-soc has an extra one (hcaptcha).
  Updated dependencies like upstream did.
- `app/controllers/api/v1/statuses_controller.rb`:
  Not a real conflict, upstream added a parameter (`allowed_mentions`) where
  glitch-soc already had an extra one (`content_type`).
  Added upstream's new parameter.
- `app/javascript/styles/fonts/roboto-mono.scss`:
  A lot of lines were changed upstream due to code style changes, and a lot
  of those lines had path changes to accomodate glitch-soc's theming system.
  Applied upstream's style changes.
- `app/javascript/styles/fonts/roboto.scss`:
  A lot of lines were changed upstream due to code style changes, and a lot
  of those lines had path changes to accomodate glitch-soc's theming system.
  Applied upstream's style changes.
2023-02-13 19:35:35 +01:00
emilweth 1f777e2436 fix metrics format (#23520) 2023-02-11 03:00:44 +01:00
emilweth 7099c057d4 dot is not allowed (#23519) 2023-02-11 02:15:07 +01:00
Claire 5f26e6e7ca Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Minor upstream change, our README is completely different.
  Kept ours.
- `lib/tasks/assets.rake`:
  glitch-soc has extra code to deal with its theming system,
  upstream changed a line that exists in glitch-soc.
  Applied upstream changes.
2023-02-09 12:46:12 +01:00
Eugen Rochko 6716afb8f8 Add information about database pool to metrics endpoint in streaming API (#23469) 2023-02-09 11:24:16 +01:00
Eugen Rochko 83eba2d6cf Add database pool configuration over `DB_POOL` to streaming API (#23470)
Add connection timeout of 15s
2023-02-09 11:20:59 +01:00
Eugen Rochko 2332ee61c0 Add metrics endpoint to streaming API (#23388) 2023-02-03 21:29:23 +01:00
Claire 54920be04d Merge branch 'main' into glitch-soc/merge-upstream 2022-12-21 15:59:39 +01:00
Nick Schonning 507f2b4c1e Enable eslint:recommended ruleset (#22433)
* Enable ESLint recommended ruleset

* Disable failing ESLint recommended rules

* Remove rules shadowed by eslint:recommended
2022-12-19 00:51:37 +09:00
Claire 6ace18fdf6 Merge branch 'main' into glitch-soc/merge-upstream 2022-11-15 02:10:37 +01:00
Claire 6645b57d39 Fix filters from other users being used in the streaming service (#20719) 2022-11-15 02:09:58 +01:00
Claire 997553a99e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/announcements/edit.html.haml`:
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
- `app/views/admin/announcements/new.html.haml`
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
2022-11-14 08:35:55 +01:00
Claire 45f808956c Fix public/local timeline posts not being properly filtered (#20567)
* Fix streaming server using wrong property name for matching filters

Late in the PR, the `filter_results` property has been renamed to `filtered`,
but the change has not been reflected in the streaming server code.

* Fix filter_action attribute being an integer instead of a string
2022-11-13 20:59:49 +01:00
Claire b50ab06371 Merge branch 'main' into glitch-soc/merge-upstream
- `.env.production.sample`:
  Our sample config file is very different from upstream since it is much more
  complete. Upstream added documentation for a few env variables.
  Copied the new variables/documentation from upstream.
- `app/lib/feed_manager.rb`:
  Upstream added a timeline type (hashtags), while glitch-soc already had an
  extra one (direct messages). Not really a conflict but textually close
  changes.
  Ported upstream's changes.
- `app/models/custom_emoji.rb`:
  Upstream upped the custom emoji size limit, while glitch-soc had configurable
  limits.
  Upped the default limits accordingly.
- `streaming/index.js`:
  Upstream reworked how hastags were normalized. Minor conflict due to
  glitch-soc's handling of instance-local posts.
  Ported upstream's changes.
2022-07-17 22:07:20 +02:00
Eugen Rochko 38d04135bf Change how hashtags are normalized (#18795)
* Change how hashtags are normalized

* Fix tests
2022-07-13 15:03:28 +02:00
Claire 490417762f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Fix erroneous deletion in a previous merge.
- `Gemfile`:
  Conflict caused by glitch-soc-only hCaptcha dependency
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/controllers/filters_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/serializers/rest/status_serializer.rb`:
  Minor conflict due to glitch-soc having an extra `local_only` property
2022-06-28 11:11:18 +02:00
Claire 90e5a9bd98 Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00