Commit Graph

252 Commits (9f05a06dee264a80827a03569664931f8d443588)

Author SHA1 Message Date
Emelia Smith 1335083bed
Streaming: replace npmlog with pino & pino-http (#27828) 2024-01-18 18:40:25 +00:00
Claire 2f521bc84a Merge commit 'a2f02a07758c32f0dcc6388b4f30ca5a84e762f3' into glitch-soc/merge-upstream 2024-01-15 20:27:41 +01:00
Emelia Smith 58830be943
Streaming: Rework websocket server initialisation & authentication code (#28631) 2024-01-15 10:36:30 +00:00
Claire a3b6a85521 Merge commit 'cd37048439f9b324f48e6af1f426a7894c54b4da' into glitch-soc/merge-upstream
Conflicts:
- `.prettierignore`:
  Conflict due to glitch-soc-only files.
  Kept those at the end of the file.
2024-01-13 12:11:29 +01:00
Renaud Chaput 923fc83190
Improve `lint-staged` config (#28704) 2024-01-12 09:09:57 +00:00
Claire 0b2c7cdb02 Merge commit 'fe2667bb0d3487a32b9da5250402a90482a85fe2' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/content_security_policy.rb`:
  Conflict caused by glitch-soc's support for the extra `EXTRA_DATA_HOSTS`
  environment variable.
  Ported upstream's changes while keeping support for `EXTRA_DATA_HOSTS`.
2024-01-10 17:51:12 +01:00
Emelia Smith 9826b7780a
Streaming: use standard cors package instead of custom implementation (#28523) 2024-01-04 09:18:03 +00:00
Claire 35b6583371 Merge commit '6374358357ee81b5f137f471f028a56974109155' into glitch-soc/merge-upstream 2023-12-29 17:40:57 +01:00
Christian Winther 01df9f1abb
install "wget" in streaming Dockerfile (#28497) 2023-12-29 10:15:10 +00:00
Claire b8209c3b96 Merge commit '85662a5a57531af5402a6777d0b1089e78c56815' into glitch-soc/merge-upstream
Conflicts:
- `config/initializers/content_security_policy.rb`:
  Upstream reworked the CSP, we kept our version for now.
- `spec/requests/content_security_policy_spec.rb`:
  Upstream reworked the CSP, we kept our version for now.
2023-12-20 20:10:45 +01:00
Claire b135b6ba8f Merge commit '8ebc94dd22a18c28c4c9763b909e92e6ba64e242' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/custom_emojis/new.html.haml`:
  Conflict caused by glitch-soc having a different file size limit constant
  name.
  Updated like upstream did while keeping glitch-soc's constant name.
2023-12-19 21:24:01 +01:00
Claire 3ad92e8d0d Merge commit 'bd575a1dd69d87ca0f69873f7badf28d38e8b9ed' into glitch-soc/merge-upstream 2023-12-19 20:17:23 +01:00
Claire 8afd313807 Merge commit '15b2d7eec59c745b418debf63907d8bd08c4a730' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Upstream split some dependencies, including `express`, to
  `streaming/package.json`.
  However, glitch-soc had extra dependencies textually close to `express`.
  Kept the extra dependencies and moved `express` to streaming.
- `yarn.lock`:
  Upstream split some dependencies, including `express`, to
  `streaming/package.json`.
  However, glitch-soc had extra dependencies textually close to `express`.
  Kept the extra dependencies and moved `express` to streaming.
2023-12-19 20:06:32 +01:00
Michael Stanclift c40cfc5d09
Fix apt cache not being properly utilized in Dockerfile (#28115) 2023-11-28 22:23:44 +00:00
Emelia Smith a4de0e364b
Refactor streaming to simplify for logging change (#28056) 2023-11-28 14:24:41 +00:00
Emelia Smith 4949b6da58
Fix streaming eslint configuration (#28055) 2023-11-28 14:09:21 +00:00
Michael Stanclift a80530d1df
Dockerfile rewrite based on Ruby image with performance optimizations and size reduction, dedicated Streaming image (#26850)
Co-authored-by: “Michael <“mx@vmstan.com>
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
2023-11-28 10:04:40 +00:00
renovate[bot] 73ab080608
Update dependency jsdom to v23 (#28075)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-27 10:06:40 +00:00
renovate[bot] 1ee3314f58
Update dependency @types/npmlog to v7 (#28048)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-23 10:28:52 +00:00
renovate[bot] 998f068499
Update Yarn to v4.0.2 (#27857)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-14 17:52:34 +00:00
Emelia Smith 15b2d7eec5
Split streaming server from web server (#24702) 2023-11-14 17:43:20 +00:00
Claire 4a9cdfb928 Merge commit '774e1189d26fffd914107a4236f6287043c988f8' into glitch-soc/merge-upstream 2023-10-09 21:12:28 +02:00
Renaud Chaput 774e1189d2
Change `eslint` config to autofix missing comma and indentation in JS files (#26711) 2023-10-09 13:38:29 +02:00
Claire e9a181c52c Merge commit 'e95d25e1013b6328457b81bd98e8d6a841d45ec2' into glitch-soc/merge-upstream 2023-10-08 16:53:05 +02:00
Emelia Smith f68d540271
Fix websocket connections being incorrectly decremented twice on errors (#27238) 2023-10-02 13:21:43 +02:00
Claire 22ee4e057d Merge commit 'a83615edc9fc1ce3363ddcf1fc676806421a5f65' into glitch-soc/merge-upstream
Conflicts:
- `streaming/index.js`:
  Upstream has added a parameter to `streamFrom`, while glitch-soc had an
  extra parameter.
  Ported upstream changes.
2023-09-19 12:59:23 +02:00
Emelia Smith fbfceb9c77
Add additional metrics for streaming (#26945) 2023-09-19 12:25:30 +02:00
Claire 4ed629cd7e Merge commit 'be991f1d18006a4820c1e9ca6625bf2bd2bfedac' into glitch-soc/merge-upstream
Conflicts:
- `app/models/form/admin_settings.rb`:
  Upstream added the notion of overriden settings, while we had extra code for
  pseudo-settings (only used to combine flavour and skin in a single select
  field).
  Ported upstream changes.
- `config/i18n-tasks.yml`:
  Upstream added `simple_form.overridden` to `ignore_unused`,
  we had `simple_form.glitch_only`.
  Added `simple_form.glitch_only` as well.
2023-09-02 14:44:47 +02:00
Gabriel Simmer be991f1d18
Move to ioredis for streaming (#26581)
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
2023-09-01 17:44:28 +02:00
Claire 1b5d3fdc5e Merge commit 'ee702e36e58d638bcf75b2eae2ca86499693465e' into glitch-soc/merge-upstream 2023-08-24 20:55:28 +02:00
Emelia Smith 6375e390af
Fix: support both DATABASE_URL and DB_PASS (#26295) 2023-08-18 15:05:35 +02:00
Claire 6530f9709c Merge commit '121443c0fca383268b8022c048dd137994785aff' into glitch-soc/main
Conflicts:
- `.rubocop_todo.yml`:
  Upstream regenerated this file, glitch-soc had a specific ignore.
2023-08-13 18:47:15 +02:00
Emelia Smith 65096c1ccd
Change streaming `/metrics` to include additional metrics (#26299) 2023-08-04 16:11:30 +02:00
Claire 92fa9d34b0 Merge commit '3554c527954441fd924586a49c7d99a89101ac7e' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/authorize_interactions_controller.rb`:
  Small conflict due to our theming system.
- `streaming/index.js`:
  Upstream refactored part of the streaming server.
  We had some extra logic for handling local-only posts.
  Applied the refactor.
2023-07-30 16:11:55 +02:00
Claire 454d8567d4 Merge commit '1814990a3d117555153321216fa593e4d9e84de3' into glitch-soc/merge-upstream 2023-07-30 15:54:34 +02:00
Renaud Chaput 3554c52795
Fix missing return values in streaming (#26233) 2023-07-28 19:11:58 +02:00
Claire 446821a0dc
Swap debug statements in streaming server (#26231) 2023-07-28 17:59:31 +02:00
Emelia Smith 81cdc0f972
Fix: Streaming server memory leak in HTTP EventSource cleanup (#26228) 2023-07-28 12:06:29 +02:00
Emelia Smith ddaf200c78
Refactor streaming's filtering logic & improve documentation (#26213) 2023-07-27 15:38:18 +02:00
Claire 1814990a3d
Fix wrong filters sometimes applying in streaming (#26159) 2023-07-27 15:12:10 +02:00
Claire 65cbcce997 Merge commit '39110d1d0af5e3d9cf452ae47496a52797249fd0' into glitch-soc/merge-upstream 2023-06-18 10:36:14 +02:00
Claire 42a0898f16 Merge commit '3a2a15c6ea4d4603469861ed9be09da12a122e45' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Conflict because glitch-soc does not have a theme selector here.
2023-06-10 18:42:29 +02:00
Emelia Smith 215081240f
Add logging of websocket send errors (#25280) 2023-06-10 18:35:57 +02:00
Emelia Smith c81f59583c
Fix logging of messages that are binary before closing their connection (#25361) 2023-06-10 18:24:37 +02:00
Claire 8c26d49cbd Merge commit '16dd3f08c1e5396d5f9ff3f13417901bc4e4b8b9' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Upstream fixed a translation bug in the theme selector that is absent from
  glitch-soc due to our different theming system.
  Discarded upstream changes.
- `streaming/index.js`:
  Upstream changed the signature of a function to change its return type.
  This is not a real conflict, the conflict being caused by an extra
  argument in glitch-soc's code.
  Applied upstream's change while keeping our extra argument.
2023-06-10 17:18:36 +02:00
Emelia Smith 16dd3f08c1
Fix performance of streaming by parsing message JSON once (#25278) 2023-06-09 19:29:16 +02:00
Claire 1cd7f5ff17 Merge commit 'd27216dc4616d80659c0cc5d2a55394e0e1ae874' into glitch-soc/merge-upstream
Conflicts:
- `.eslintrc.js`:
  Upstream moved a configuration block in which we had added a glitch-only
  path.
  Moved the configuration block as upstream did.
- other files:
  Upstream reordered imports, and those files had different ones.
  Kept our version and reordered imports using the same rules.
2023-05-28 15:54:31 +02:00
Renaud Chaput d27216dc46
Enforce import order with ESLint (#25096) 2023-05-23 17:15:17 +02:00
Claire 2d029dedd9 Merge commit '0ad2413b35287958f59073a5b63aecc659a64d98' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/styles/mastodon/forms.scss`:
  Conflict because we ran eslint autofix on upstream files.
- `config/initializers/content_security_policy.rb`:
  Code style changes but we have a different version.
  Kept our version.
- `streaming/index.js`:
  Upstream fixed a typo close to glitch-soc-only code.
  Applied upstream's changes.
2023-05-08 15:28:36 +02:00
Claire 3fb7fe14c6
Fix some of the Javascript linting issues, as well as bugs and unneeded divergences with upstream (#2208)
* Run eslint --fix

* Fix linting issues in video player and reduce divergences with upstream

This includes a behavior change of not auto-looping videos anymore. I don't
remember loops being ever intended, and they have been removed from upstream
a while ago, but we somehow missed the change.

* Fix lint issues in `app/javascript/flavours/glitch/selectors/index.js`

Those were basically caused by dead code that isn't present upstream, so
that brings us closer to upstream as well.

* Fix linting issue and bug in streaming/index.js

* Fix linting issues in config/webpack/shared.js

* Fix unused import in flavours/glitch/features/ui/index.js

* Fix linting issues and reduce divergences from upstream in flavours/glitch/features/ui/components/video_modal.jsx

* Fix linting issues in flavours/glitch/reducers

* Fix linting issues in glitch-soc onboarding modal

* Fix linting issues in flavours/glitch/features/ui/components/navigation_panel.jsx

* Remove dead code for unused local setting navbar_under

* Fix various linting issues

* Fix linting issues in flavours/glitch/components/scrollable_list.jsx and reduce divergences with upstream
2023-05-07 18:22:25 +02:00
Nick Schonning f22b4e556c
Add eslint-plugin-jsdoc (#24719) 2023-04-30 02:29:54 +02:00
Claire 12b935fadf Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Updated upstream, removed in glitch-soc to disable noise.
  Kept removed.
- `CODE_OF_CONDUCT.md`:
  Upstream updated to a new version of the covenant, but I have not read it
  yet, so kept unchanged.
- `Gemfile.lock`:
  Not a real conflict, one upstream dependency updated textually too close to
  the glitch-soc only `hcaptcha` dependency.
  Applied upstream changes.
- `app/controllers/admin/base_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/application_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/disputes/base_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/relationships_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/controllers/statuses_cleanup_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/helpers/application_helper.rb`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.jsx`:
  Upstream added a highlight animation for onboarding, while we changed the
  max character limit.
  Applied our local changes on top of upstream's new version.
- `app/views/layouts/application.html.haml`:
  Minor conflict due to glitch-soc's theming system.
  Applied upstream changes.
- `stylelint.config.js`:
  Upstream added ignore paths, glitch-soc had extra ignore paths.
  Added the same paths as upstream.
2023-04-29 10:44:56 +02:00
Emelia Smith 301e03eb8c
Remove clustering from streaming API (#24655) 2023-04-26 11:37:51 +02:00
Claire 3ef5f62abf Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream switched to pushing to both DockerHub and GitHub Container
  Repository, while glitch-soc was already pushing to the latter only.
  Updated our configuration to be slightly more consistent with upstream's
  naming and styling, but kept our behavior.
- `Gemfile.lock`:
  Updated dependencies textually too close to glitch-soc only hcaptcha
  dependency.
  Updated dependencies as upstream did.
- `README.md`:
  Upstream updated its README, but we have a completely different one.
  Kept our README, though it probably should be reworked at some point.
- `app/views/auth/sessions/two_factor.html.haml`:
  Minor style fix upstream that's on a line glitch-soc removed because
  of its different theming system.
  Kept our file as is.
- `spec/controllers/health_controller_spec.rb`:
  This file apparently did not exist upstream, upstream created it with
  different contents but it is functionally the same.
  Switched to upstream's version of the file.
- `spec/presenters/instance_presenter_spec.rb`:
  Upstream changed the specs around `GITHUB_REPOSITORY`, while glitch-soc
  had its own code because it's a fork and does not have the same default
  source URL.
  Took upstream's change, but with glitch-soc's repo as the default case.
- `yarn.lock`:
  Upstream dependencies textually too close to a glitch-soc only one.
  Updated dependencies as upstream did.
2023-03-15 09:16:10 +01:00
Claire 174352716a
Disable anonymous access to the streaming API (#23989) 2023-03-06 21:00:10 +01:00
Claire 7623e18124 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream README has been changed, but we have a completely different one.
  Kept our `README.md`.
- `lib/sanitize_ext/sanitize_config.rb`:
  Upstream added support for more incoming HTML tags (a large subset of what
  glitch-soc accepts).
  Change the code style to match upstream's but otherwise do not change our
  code.
- `spec/lib/sanitize_config_spec.rb`:
  Upstream added support for more incoming HTML tags (a large subset of what
  glitch-soc accepts).
  Kept our version, since the tests are mostly glitch-soc's, except for cases
  which are purposefuly different.
2023-03-05 20:46:56 +01:00
Eugen Rochko be1792e1d4
Fix streaming API not being usable without `DATABASE_URL` (#23960) 2023-03-05 01:52:12 +01:00
Ramūns Usovs 0872f3e3d7
Allow streaming to connect to postgress with self-signed certs (#21431) 2023-03-03 21:01:18 +01:00
Claire ce84d163cc Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.prettierignore`:
  Upstream added a line at the end, glitch-soc had extra entries at the end.
  Added upstream's new line before glitch-soc's.
- `Gemfile.lock`:
  Upstream updated dependencies while glitch-soc has an extra one (hcaptcha).
  Updated dependencies like upstream did.
- `app/controllers/api/v1/statuses_controller.rb`:
  Not a real conflict, upstream added a parameter (`allowed_mentions`) where
  glitch-soc already had an extra one (`content_type`).
  Added upstream's new parameter.
- `app/javascript/styles/fonts/roboto-mono.scss`:
  A lot of lines were changed upstream due to code style changes, and a lot
  of those lines had path changes to accomodate glitch-soc's theming system.
  Applied upstream's style changes.
- `app/javascript/styles/fonts/roboto.scss`:
  A lot of lines were changed upstream due to code style changes, and a lot
  of those lines had path changes to accomodate glitch-soc's theming system.
  Applied upstream's style changes.
2023-02-13 19:35:35 +01:00
emilweth 7c45145430
fix metrics format (#23520) 2023-02-11 03:00:44 +01:00
emilweth c1b823368c
dot is not allowed (#23519) 2023-02-11 02:15:07 +01:00
Claire 85558a5e18 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Minor upstream change, our README is completely different.
  Kept ours.
- `lib/tasks/assets.rake`:
  glitch-soc has extra code to deal with its theming system,
  upstream changed a line that exists in glitch-soc.
  Applied upstream changes.
2023-02-09 12:46:12 +01:00
Eugen Rochko c5a4d8c82d
Add information about database pool to metrics endpoint in streaming API (#23469) 2023-02-09 11:24:16 +01:00
Eugen Rochko 4cb7647516
Add database pool configuration over `DB_POOL` to streaming API (#23470)
Add connection timeout of 15s
2023-02-09 11:20:59 +01:00
Eugen Rochko 01584f03e8
Add metrics endpoint to streaming API (#23388) 2023-02-03 21:29:23 +01:00
Claire b248aa4d3e Merge branch 'main' into glitch-soc/merge-upstream 2022-12-21 15:59:39 +01:00
Nick Schonning 06b68490d1
Enable eslint:recommended ruleset (#22433)
* Enable ESLint recommended ruleset

* Disable failing ESLint recommended rules

* Remove rules shadowed by eslint:recommended
2022-12-19 00:51:37 +09:00
Claire 66461ae79a Merge branch 'main' into glitch-soc/merge-upstream 2022-11-15 02:10:37 +01:00
Claire 8c3c6dcae9
Fix filters from other users being used in the streaming service (#20719) 2022-11-15 02:09:58 +01:00
Claire 32c70d2f09 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/announcements/edit.html.haml`:
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
- `app/views/admin/announcements/new.html.haml`
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
2022-11-14 08:35:55 +01:00
Claire cd5e98dbdb
Fix public/local timeline posts not being properly filtered (#20567)
* Fix streaming server using wrong property name for matching filters

Late in the PR, the `filter_results` property has been renamed to `filtered`,
but the change has not been reflected in the streaming server code.

* Fix filter_action attribute being an integer instead of a string
2022-11-13 20:59:49 +01:00
Claire cd87d7dcef Merge branch 'main' into glitch-soc/merge-upstream
- `.env.production.sample`:
  Our sample config file is very different from upstream since it is much more
  complete. Upstream added documentation for a few env variables.
  Copied the new variables/documentation from upstream.
- `app/lib/feed_manager.rb`:
  Upstream added a timeline type (hashtags), while glitch-soc already had an
  extra one (direct messages). Not really a conflict but textually close
  changes.
  Ported upstream's changes.
- `app/models/custom_emoji.rb`:
  Upstream upped the custom emoji size limit, while glitch-soc had configurable
  limits.
  Upped the default limits accordingly.
- `streaming/index.js`:
  Upstream reworked how hastags were normalized. Minor conflict due to
  glitch-soc's handling of instance-local posts.
  Ported upstream's changes.
2022-07-17 22:07:20 +02:00
Eugen Rochko e7aa2be828
Change how hashtags are normalized (#18795)
* Change how hashtags are normalized

* Fix tests
2022-07-13 15:03:28 +02:00
Claire fe5f6bc7ed Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Fix erroneous deletion in a previous merge.
- `Gemfile`:
  Conflict caused by glitch-soc-only hCaptcha dependency
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/controllers/filters_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/serializers/rest/status_serializer.rb`:
  Minor conflict due to glitch-soc having an extra `local_only` property
2022-06-28 11:11:18 +02:00
Claire 02851848e9
Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Claire 002d2729fb Merge branch 'main' into glitch-soc/merge-upstream 2022-04-25 17:09:32 +02:00
Yamagishi Kazutoshi 5781d1db84
Fix parsing `TRUSTED_PROXY_IP` (#18051) 2022-04-19 08:11:58 +01:00
Claire 59864ff495 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Upstream modified it, but we deleted it in glitch-soc.
  Keep it deleted.
2022-03-22 17:00:38 +01:00
Claire f29458da1d
Fix streaming server sometimes silently dropping subscriptions (#17841) 2022-03-21 19:08:29 +01:00
Claire f224237862 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `db/schema.rb`:
  Conflict due to glitch-soc adding the `content_type` column on status edits
  and thus having a different schema version number.
  Solved by taking upstream's schema version number, as it is higher than
  glitch-soc's.
2022-02-17 10:58:44 +01:00
Claire 6ea80ba2a2
Change streaming server error messages when failing to parse client input (#17559)
Fixes #17541

- prefix JSON parsing error message by “Error parsing message from …”
- output user id if a user is logged in, IP address otherwise
- reduce log level from error to warning when a user is logged in, and to silly
  otherwise
2022-02-16 14:37:26 +01:00
Claire fca50e4b50 Merge branch 'main' into glitch-soc/merge-upstream 2022-01-07 19:51:47 +01:00
Claire bc066d8936
Fix timeline streaming stopping for multiple sessions instead of one (#17259)
* Fix timeline streaming stopping for multiple sessions instead of one

Fixes #17256.

In updating the code for a newer version of node-redis, #17183 also broke
redis subscription management when multiple streaming clients subscribe to the
same channel.

This commit restores the redis subscription management code.

* Let node-redis actually handle the subscriptions
2022-01-07 19:50:12 +01:00
Claire 09c61e1376 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  We have completely different contents. Kept our version.
- `package.json`:
  Not a real conflict, just an upstream dependency udpated
  textually too close to a glitch-soc-only dependency.
  Updated dependencies like upstream.
- `streaming/index.js`:
  Conflict due to code style changes on parts that were
  modified in glitch-soc to handle local-only toots.
  Changed style according to upstream.
2021-12-27 15:16:18 +01:00
Lerk 4d1eaf3e6e
Finish update of node-redis (#17183)
* fix streaming redis client

* use console.error instead of console.log

* follow node-redis migration guide

https://github.com/redis/node-redis/blob/master/docs/v3-to-v4.md

* fix config options for node-redis

* keep indentation

* Update streaming/index.js

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2021-12-25 22:55:06 +01:00
Claire 1665b84285 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, just a glitch-soc-only dependency textually too close
  to an updated upstream one.
- `yarn.lock`:
  Not really a conflict, just a glitch-soc-only dependency textually too close
  to an updated upstream one.
2021-10-13 15:50:24 +02:00
Sasha Sorokin 6c88ebfd4b
fix(streaming): req.scopes can be nullable (#16823)
When checking for required OAuth scopes, an unexpected error could
happen due to missing (null-y) req.scopes. This commit fixes that by
checking if req.scopes are present before checking if any required
scopes are present, otherwise it skips that straight to rejection.
2021-10-13 05:02:55 +02:00
Claire 3622110778 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `streaming/index.js`:
  Filtering code for streaming notifications has been refactored upstream, but
  glitch-soc had similar code for local-only toots in the same places.
  Ported upstream changes, but did not refactor local-only filtering.
2021-09-26 18:28:59 +02:00
Eugen Rochko a0d4129893
Refactor notifications to go through a separate stream in streaming API (#16765)
Eliminate need to have custom notifications filtering logic in the
streaming API code by publishing notifications into a separate stream
and then simply using the multi-stream capability to subscribe to that
stream when necessary
2021-09-26 13:23:28 +02:00
Claire 50b430d9a2 Merge branch 'main' into glitch-soc/merge-upstream
- `app/views/statuses/_simple_status.html.haml`:
  Small markup change in glitch-soc, on a line that has been modified by
  upstream. Ported upstream changes.
2021-05-07 18:21:59 +02:00
Eugen Rochko c5c46dd6ee
Fix "cb is not a function" error in streaming API server (#16134)
Third argument of `ping` is the callback

Regression from #15932
2021-05-02 14:30:26 +02:00
Eugen Rochko aafe65a142
Change log level of worker start/end to warn in streaming API (#16110) 2021-05-01 23:19:18 +02:00
Claire f60c99a8fb Merge branch 'main' into glitch-soc/merge-upstream 2021-03-24 10:53:50 +01:00
Claire 49814d5799
Switch from deprecated ClusterWS/cws to ws package (#15932)
* Switch from deprecated ClusterWS/cws to ws package

Fixes #15184

Co-authored-by: Edho Arief <me@nanaya.pro>

* Make bufferutil and utf-8-validate optional dependencies

Co-authored-by: Edho Arief <me@nanaya.pro>
2021-03-24 09:37:41 +01:00
Thibaut Girka e71b16ae2b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  glitch-soc-only dependency textually too close to updated upstream
  dependencies. Updated to upsteam dependencies.
2020-11-25 15:31:45 +01:00
abcang a2da02626e
Fixed ESLint error (#15214)
* eslint --fix

* fix consistent-return

* fix promise/catch-or-return

* ignore import rule
2020-11-23 17:35:14 +01:00
Thibaut Girka de5cc20dd8 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Conflict caused because of glitch-soc's theming system.
  Took upstream's new code and applied the theming system changes on top
  of it.
- `app/controllers/concerns/two_factor_authentication_concern.rb`:
  Conflict caused because of glitch-soc's theming system.
  Took upstream's new code and applied the theming system changes on top
  of it.
2020-11-14 00:30:36 +01:00
Eugen Rochko aa10200e58
Fix streaming API allowing connections to persist after access token invalidation (#15111)
Fix #14816
2020-11-12 23:05:24 +01:00
Thibaut Girka a7aedebc31 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
  Upstream changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/models/user.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc not preventing moved accounts from logging
  in.
  Ported upstream changes while keeping the ability for moved accounts to log
  in.
- `app/policies/status_policy.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's ability  to hide followers count.
  Ported upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `package.json`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
2020-09-28 14:13:30 +02:00
fuyu e39d97f700
Fix crash streaming process when receive invalid json (#14859) 2020-09-22 15:30:41 +02:00