Commit Graph

80 Commits (ea1a221e2de95bf40aefaa34cab8b08716f941a5)

Author SHA1 Message Date
ThibG 89f40b6c3e
Make domain block/silence/reject-media code more robust (#13424)
* Split media cleanup from reject-media domain blocks to its own service

* Slightly improve ClearDomainMediaService error handling

* Lower DomainClearMediaWorker to lowest-priority queue

* Do not catch ActiveRecord::RecordNotFound in domain block workers

* Fix DomainBlockWorker spec labels

* Add some specs

* Change domain blocks to immediately mark accounts as suspended

Rather than doing so sequentially, account after account, while cleaning
their data. This doesn't change much about the time the block takes to
complete, but it immediately prevents interaction with the blocked domain,
while up to now, it would only be guaranteed when the process ends.
2020-06-09 10:32:00 +02:00
ThibG aa138ea350 Fix RefollowWorker not keeping show_reblogs setting (#12707)
* Fix RefollowWorker not keeping show_reblogs setting

* Fix RefollowWorker
2020-01-02 20:52:39 +01:00
ThibG 66c1fe0495 Fix various issues with account migration (#12301)
* Fix being able to follow oneself by moving to an account that was following the old one

* Add specs

* Add spec to catch MoveWorker issue with local followers following both accounts

* Fix move worker breaking when a local account follows both source and target accounts

* Fix migration from remote to local account not sending Undo Follow

* Fix show_reblogs not being preserved for moved account's followers
2019-11-07 09:05:07 +02:00
ThibG bced70469a Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Eugen Rochko 23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
ThibG 9a90ec3b3b Fix account URI in UpdatePollSerializer (#11194)
* Fix account URI in UpdatePollSerializer

Fixes #11185

* Add specs
2019-06-27 19:41:55 +02:00
ThibG 9d3c6f1849 Improved remote thread fetching (#10106)
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
2019-02-28 15:22:21 +01:00
Eugen Rochko a49d43d112
Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
Eugen Rochko 24cafd73a2
Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
aschmitz 554c2fd8af Clean up reblog tracking keys, related improvements (#5428)
* Clean up reblog-tracking sets from FeedManager

Builds on #5419, with a few minor optimizations and cleanup of sets
after they are no longer needed.

* Update tests, fix multiply-reblogged case

Previously, we would have lost the fact that a given status was
reblogged if the displayed reblog of it was removed, now we don't.

Also added tests to make sure FeedManager#trim cleans up our reblog
tracking keys, fixed up FeedCleanupScheduler to use the right loop,
and fixed the test for it.
2017-10-17 11:45:06 +02:00
Eugen Rochko e528114c53 Follow-up to #4582 and #5027, removing dead code (#5101) 2017-09-26 01:06:27 +02:00
Eugen Rochko cf615abbf9 Add configuration to disable private status federation over PuSH (#4582) 2017-08-24 17:51:32 +02:00
Eugen Rochko 412ea87306 Improve ActivityPub/OStatus compatibility (#4632)
*Note: OStatus URIs are invalid for ActivityPub. But we have them for
as long as we want to keep old OStatus-sourced content and as long as
we remain OStatus-compatible.*

- In Announce handling, if object URI is not a URL, fallback to object URL
- Do not use specialized ThreadResolveWorker, rely on generalized handling
- When serializing notes, if parent's URI is not a URL, use parent's URL
2017-08-19 18:44:48 +02:00
Eugen Rochko b7370ac8ba ActivityPub delivery (#4566)
* Deliver ActivityPub Like

* Deliver ActivityPub Undo-Like

* Deliver ActivityPub Create/Announce activities

* Deliver ActivityPub creates from mentions

* Deliver ActivityPub Block/Undo-Block

* Deliver ActivityPub Accept/Reject-Follow

* Deliver ActivityPub Undo-Follow

* Deliver ActivityPub Follow

* Deliver ActivityPub Delete activities

Incidentally fix #889

* Adjust BatchedRemoveStatusService for ActivityPub

* Add tests for ActivityPub workers

* Add tests for FollowService

* Add tests for FavouriteService, UnfollowService and PostStatusService

* Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService

* Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService

* Add tests for BatchedRemoveStatusService

* Deliver updates to a local account to ActivityPub followers

* Minor adjustments
2017-08-13 00:44:41 +02:00
Eugen Rochko a549d1ae6b Bump to 1.5.0rc1 (#4318) 2017-07-24 16:21:08 +02:00
Eugen Rochko 1fcdaafa6f Fix webfinger retries (#4275)
* Do not raise unretryable exceptions in ResolveRemoteAccountService

* Removed fatal exceptions from ResolveRemoteAccountService

Exceptions that cannot be retried should not be raised. New exception
class for those that can be retried (Mastodon::UnexpectedResponseError)
2017-07-20 01:59:07 +02:00
Eugen Rochko 1618b68bfa HTTP signatures (#4146)
* Add Request class with HTTP signature generator

Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06

* Add HTTP signature verification concern

* Add test for SignatureVerification concern

* Add basic test for Request class

* Make PuSH subscribe/unsubscribe requests use new Request class

Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day

* Make all PuSH workers use new Request class

* Make Salmon sender use new Request class

* Make FetchLinkService use new Request class

* Make FetchAtomService use the new Request class

* Make Remotable use the new Request class

* Make ResolveRemoteAccountService use the new Request class

* Add more tests

* Allow +-30 seconds window for signed request to remain valid

* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
2017-07-14 20:41:49 +02:00
Akihiko Odaki (@fn_aki@pawoo.net) 7d8e3721ae Overwrite old statuses with reblogs in PrecomputeFeedService (#3984) 2017-06-28 14:50:23 +02:00
Eugen Rochko e17c2e5da5 Batched remove status service (#3735)
* Make Pubsubhubbub::DistributionWorker handle both single stream entry
arguments, as well as arrays of stream entries

* Add BatchedRemoveStatusService, make SuspendAccountService use it

* Improve method names

* Add test

* Add more tests

* Use PuSH payloads of 100 to have a clear mapping of
1000 input statuses -> 10 PuSH payloads

It was nice while it lasted
2017-06-14 18:01:35 +02:00
Matt Jankowski ec34ec63b1 Specs for cleanup workers (#3235)
* Add spec files for feed and media cleanup workers

* Add coverage for feed and media cleanup schedulers

* Clean up feed and media cleanup workers
2017-05-22 19:36:21 +02:00
Matt Jankowski d2e0edd721 Add spec coverage for regeneration worker (#3143) 2017-05-19 16:55:09 +02:00
Matt Jankowski 8ec8410651 After remote follow worker specs (#3098)
* Add specs for the after remote follow workers

* Refactor the after remote follow workers
2017-05-18 21:10:41 +02:00
Eugen Rochko 657496b5a9 Do not cancel PuSH subscriptions after encountering "permanent" error… (#3046)
* Do not cancel PuSH subscriptions after encountering "permanent" error response

After talking with MMN about it, turns out some servers/php setups do
return 4xx errors while rebooting, so this anti-feature that was meant
to take load off of the hub is doing more harm than good in terms of
breaking subscriptions

* Update delivery_worker.rb
2017-05-14 03:22:48 +02:00
Matt Jankowski 0d70fe2659 Spec coverage and refactor for pubsub/delivery worker (#3021)
* Framework for delivery worker spec

* Refactor of pubsub delivery worker
2017-05-12 20:35:36 +02:00
Matt Jankowski 60ecfb87ae Coverage for domain block worker (#2997) 2017-05-11 22:25:15 +02:00
Matt Jankowski 08949cca41 Pubsub confirmation worker spec (#2974)
* Add specs for valid challenge response from pubsub confirmation worker

* Refactor the pubsub confirmation worker
2017-05-11 00:19:37 +02:00
Matt Jankowski cc9a6a710f Spec feed insert worker (#2965)
* Spec for feed insert worker when missing records

* more specs!

* Refactor feed insert worker
2017-05-10 20:32:05 +02:00
Matt Jankowski d142544159 Spec coverage and refactor of digest mailer worker (#2961) 2017-05-10 15:45:43 +02:00
Eugen Rochko 3a38322a54 Add spec for Pubsubhubbub::DistributionWorker. PuSH-deliver public items (#2954)
to all subscribers. IDN-normalize callback URLs for subscriptions on insert.
2017-05-10 02:55:43 +02:00
Eugen Rochko b8e166894b Fix Scheduler::SubscriptionsScheduler (#2834)
* Fix Scheduler::SubscriptionsScheduler, add worker test for it

* Change production log level of Sidekiq to "warn" instead of "info"
2017-05-06 13:05:03 +02:00