Commit Graph

572 Commits (20f7e2c452663be957a0500d84946c2df707354a)

Author SHA1 Message Date
Takeshi Umeda 4395de7360 Fix a bear check when the activity object is nil (#14981) 2020-10-13 22:41:00 +02:00
Eugen Rochko da4c730c47 Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Eugen Rochko adaf3f7b7a Remove dependency on goldfinger gem (#14919)
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.

Fix #14091
2020-10-08 00:34:57 +02:00
ThibG 79b2d30a19 Add option to disable swiping motions across the WebUI (#13885)
Fixes #13882
2020-09-30 19:31:03 +02:00
ThibG 1603dc87ac Add environment variable to allow requests to some private addresses (#14722) 2020-09-26 20:57:39 +02:00
Eugen Rochko 02c4c91faa Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
Eugen Rochko e514304a76 Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
Eugen Rochko 38aa7a9a7f Refactor feed manager (#14761) 2020-09-08 03:41:16 +02:00
ThibG e8aa85dc03 Fix multiple boosts of a same toot erroneously appearing in TL (#14759)
* Check for and record reblog info atomically

Instead of using ZREVRANK to determine whether a reblog is a new reblog or not,
use ZADD's NX option to perform the check/addition option atomically.

* Replace ZREVRANK call with ZSCORE key which is more efficient

* Make tests a bit stricter

* Fix off-by-one
2020-09-07 18:00:15 +02:00
Eugen Rochko c6ad9aae69 Add featured hashtags as an ActivityPub collection (#11595) 2020-09-02 02:11:12 +02:00
ThibG 50d0c1e95f Add configuration option to filter replies in lists (#9205)
* Add database support for list show-reply preferences

* Add backend support to read and update list-specific show_replies settings

* Add basic UI to set list replies setting

* Add specs for list replies policy

* Switch "cycling" reply policy link to a set of radio inputs

* Capitalize replies_policy strings

* Change radio button design to be consistent with that of the directory explorer
2020-09-01 13:31:28 +02:00
Eugen Rochko a143764c4c Change own direct-visibility statuses to be in the home feed again (#14711)
And remove highlighting in web UI

Full circle from #8940
2020-09-01 13:30:42 +02:00
dependabot[bot] 61b768572e Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
Eugen Rochko e8d1656bd6 Add support for dereferencing objects through bearcaps (#14683) 2020-08-30 12:34:20 +02:00
ThibG 46283b489e Fix dereferencing remote statuses not using the correct account (#14656)
Follow-up to #14359

In the case of limited toots, the receiver may not be explicitly part of the
audience. If a specific user's inbox URI was specified, it makes sense to
dereference the toot from the corresponding user, instead of trying to find
someone in the explicit audience.
2020-08-24 16:56:21 +02:00
ThibG 4468fce63c Add support for inlined objects in activity audience (#14514)
* Add support for inlined objects in activity audience

* Add tests
2020-08-24 14:11:47 +02:00
ThibG d6f2d1d3f2 Fix handling of Reject Follow when a matching follow relationship exists (#14479)
* Add tests

* Fix handling of Reject Follow when a matching follow relationship exists

Regression from #12199
2020-08-01 18:20:37 +02:00
ThibG e041b8b3f8 Fix not handling Undo on some activity types when they aren't inlined (#14346)
* Fix not handling Undo on some activity types when they aren't inlined

When receiving an Undo for a non-inlined activity, try looking it up in
database using the URI. The queries are ad-hoc because we don't have a global
index of object URIs, and not all activity types are stored in database with
an index on their URI.

Announces are just statuses, and have an index on URIs, so this check can
be done efficiently.

Accepts cannot be handled at all because we don't record their URI at any
point.

Follows don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

Likes don't have an index on URI, they have an index on the issuing account,
but the number of favs per account may be very high, so I decided not to
handle that.

Blocks don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

In all cases, if an Undo could not be handled properly, we call `delete_later!`
because that does not require us to know more than the URI of the undone
property.

* Add tests

* Make newer blocks overwrite older ones

Allows re-synchronizing block info by re-blocking and un-blocking again
when the original Undo Block has been lost.
2020-07-22 11:45:35 +02:00
ThibG 9c90f5347e Dereference object URIs in Create and Update messages (#14359)
* Dereference object URIs in Create and Update messages

Fixes #14353

Signed-off-by: Thibaut Girka <thib@sitedethib.com>

* Refactor, and perform origin check *before* attempting to fetch object

Co-authored-by: Fire Demon <firedemon@creature.cafe>
2020-07-22 11:43:17 +02:00
Takeshi Umeda ac0805ad10 Fix to add RedisLock to handle Announce activity (#14365) 2020-07-20 11:25:26 +02:00
Takeshi Umeda 622dcbeb8c Fix movie width and frame_rate returning nil (#14357)
* Fix movie width and frame_rate returning nil

* Add StreamValidationError and raise

* Fix code style
2020-07-19 22:28:27 +02:00
ThibG a9e58aa8c6 Fix boosted toots from blocked account not being retroactively removed from TL (#14339)
* Fix boosted toots from blocked account not being retroactively removed from TL

Fixes #14301

* Add test for clear_from_timeline
2020-07-17 07:07:54 +02:00
Takeshi Umeda 27c1146f37 Improve group processing (#14279)
* Fix the local group's followers collection

* Fix to accept followed relayed_through_account

* Add local delivery to the group's followers

* Fix code style

* Revert "Add local delivery to the group's followers"

This reverts commit 3237effc199772e4c4d30f19082cbc5633f56196.
2020-07-14 19:05:34 +02:00
abcang aca93fa882 Fix rubocop warning (#14288)
* Fix rubocop warning

* use limit variable

* use ContextCreatingMethods option
2020-07-14 19:05:07 +02:00
Eugen Rochko ca238c879d Change Redis#exists calls to Redis#exists? to avoid deprecation warning (#14191) 2020-07-01 19:05:21 +02:00
Eugen Rochko 36e5fd547b Fix remote files not using Content-Type header, streaming (#14184) 2020-06-30 23:58:02 +02:00
Eugen Rochko a3ce01a102 Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
Sasha Sorokin 8e9572112d Fix Thai being skipped from language detection (#13989)
Thai does not separate words by spaces, so I figured out it should be
in 'reliable characters regexp' that denotes languages that do the same.

Related #13891.
2020-06-25 22:45:01 +02:00
ThibG df13142cdd Fix crash in MergeWorker (#14129)
Similarly to #12324, the code is passing an Account object where an id
is expected.
2020-06-23 16:40:01 +02:00
Ben Lubar da59204700 FIX: filters ignore media descriptions (#13837)
* FIX: filters ignore media descriptions

* remove parentheses to make codeclimate happy

* combine the text and run the regular expression only once.

https://github.com/tootsuite/mastodon/pull/13837#discussion_r431752581

* Fix use of “filter” instead of “compact”, fix coding style issues

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-06-09 00:11:42 +02:00
Eugen Rochko b864e78db8 Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
ThibG bb3aff5c8c Add support for `summary` field for media description (#13763) 2020-05-15 17:08:59 +02:00
ThibG 60bb6ac6f4 Fix webfinger returning wrong status code on malformed or missing param (#13759)
Fixes #13757
2020-05-14 23:28:06 +02:00
Takeshi Umeda 25307798d7 Fix keybase config serializer (#13742) 2020-05-12 15:28:29 +02:00
Eugen Rochko 909c31a765 Fix error within error when limiting backtrace to 3 lines (#13120)
Fix #13086, close #13113
2020-05-10 10:30:27 +02:00
ThibG 41536557be Improve RSS entries for statuses (#13592)
* Improve RSS entries for statuses

- Render polls in both accounts and tags serializers
- Refactor RSS serializers
- Change title preview to include ellipsis when truncated
- Change title preview to show CW instead of toot text
- Add tests

* Remove title from OEmbed serialization

Twitter doesn't serialize title either, and tihs allows us to move the
title formatting code to the RSS serializers.
2020-05-10 09:50:54 +02:00
Eugen Rochko 2d3219549b Change delivery failure tracking to work with hostnames instead of URLs (#13437) 2020-04-15 20:33:24 +02:00
Eugen Rochko 50fa554899 Add rate limit for reporting (#13390) 2020-04-05 14:40:08 +02:00
ThibG ef48ffc127 Fix PostgreSQL load when linking in announcements (#13250)
* Fix PostgreSQL load when linking in announcements

Fixes #13245 by caching status lookups

Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.

Since it's only used by announcements so far, there won't
be much statuses to cache.

* Perform status lookup when saving announcements, not when rendering them

* Change EntityCache#status to fetch URLs instead of looking into the database

* Move announcement link lookup to publishing worker

* Address issues pointed out during review
2020-04-05 12:51:22 +02:00
Eugen Rochko 1cb4645638 Change `tootctl media remove-orphans` to work for all classes (#13316)
Change `tootctl media lookup` to not use an interactive prompt
2020-03-26 01:56:41 +01:00
dependabot-preview[bot] b4bc2817ba Bump cld3 from 3.2.6 to 3.3.0 (#13107)
* Bump cld3 from 3.2.6 to 3.3.0

Bumps [cld3](https://github.com/akihikodaki/cld3-ruby) from 3.2.6 to 3.3.0.
- [Release notes](https://github.com/akihikodaki/cld3-ruby/releases)
- [Commits](https://github.com/akihikodaki/cld3-ruby/compare/v3.2.6...v3.3.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix compatibility with cld3 3.3.0

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2020-03-09 00:12:52 +01:00
Eugen Rochko a4668d994b Add specific rate limits for posting and following (#13172) 2020-03-08 15:17:39 +01:00
Eugen Rochko 4cf2ddc5df Fix rendering `<a>` without `href` when scheme unsupported (#13040)
- Disallow links with relative paths
- Disallow iframes with non-http protocols and relative paths

Close #13037
2020-02-08 21:22:38 +01:00
Eugen Rochko bf4fb26b9d Fix malformed HTML causing uncaught error (#13042)
Fix OEmbed preview API leaking existence of private statuses (see #12930)
2020-02-07 15:24:22 +01:00
Eugen Rochko dd68e44cfa Add streaming API updates for announcements being modified or deleted (#12963)
Change `all_day` to be a visual client-side cue only

Publish immediately if `scheduled_at` is in the past

Add `published_at` and `updated_at` to announcements JSON
2020-01-26 20:07:26 +01:00
Eugen Rochko f95fd62bac Add announcements (#12662)
* Add announcements

Fix #11006

* Add reactions to announcements

* Add admin UI for announcements

* Add unit tests

* Fix issues

- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"

* Fix scheduler unpublishing announcements before they are due

* Fix filter params not being passed to announcements filter
2020-01-23 22:00:13 +01:00
ThibG 25e7aa913c Add support for magnet: URIs (#12905) 2020-01-23 21:27:26 +01:00
ThibG 4eaa481fae Exempt staff members from spam check (#12874)
Consider admins and moderators as trusted, for the purpose of the
spam checker.

Fixes #12872
2020-01-17 07:50:30 +01:00
ThibG c297a1e7a7 Fix Ruby 2.7 support (#12831)
- update http gem to avoid errors
- update blurhash gem to avoid shared object loading error
- update goldfinger gem so the http gem could be updated
- update json gem to avoid warnings
2020-01-12 15:49:30 +01:00
ThibG b7927b397d Add support for linking XMPP URIs in toots (#12709)
* Fix wrong grouping in Twitter valid_url regex

* Add support for xmpp URIs

Fixes #9776

The difficult part is autolinking, because Twitter-text's extractor does
some pretty ad-hoc stuff to find things that “look like” URLs, and XMPP
URIs do not really match the assumptions of that lib, so it doesn't sound
wise to try to shoehorn it into the existing regex.

This is why I used a specific regex (very close, although slightly more
permissive than the RFC), and a specific scan function (a simplified version
of the generalized one from Twitter).

* Remove leading “xmpp:” from auto-linked text
2020-01-11 02:15:25 +01:00