Commit Graph

114 Commits (3ccbe066f05bd87623497717d9bf15938be18567)

Author SHA1 Message Date
Eugen Rochko f07b0dc82f Remove unneeded indices, improve error handling in background workers, don't needlessly reload reblogged status, send Devise e-mails asynchronously 2016-11-22 17:32:51 +01:00
Eugen Rochko 3fc7820d6e Call NotifyService about remote reblogs 2016-11-21 09:56:01 +01:00
Eugen Rochko 38025dfea3 Adding unified streamable notifications 2016-11-20 19:39:58 +01:00
Eugen Rochko b373fb0ff6 Catch validation errors in ProcessFeedService so that one failing entry wouldn't stop others from going through 2016-11-18 23:19:38 +01:00
Eugen Rochko d2029a8845 Don't create attachments if remote file cannot be fetched 2016-11-18 23:16:34 +01:00
Eugen Rochko e71b152d89 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko 6f0d312dc3 Force utf-8 encoding when processing XML 2016-11-13 19:12:40 +01:00
Eugen Rochko 0b03bbad67 Discard misattributed remote statuses, improve timelines filter 2016-11-10 00:15:49 +01:00
Eugen Rochko c0ee14756f Adding more logging 2016-11-08 19:37:08 +01:00
Eugen Rochko ac79369f2b Fix for thread resolve service and process feed service url method 2016-11-08 19:09:22 +01:00
Eugen Rochko 9da30e31c8 Fix region setting for AWS gem 2016-11-08 18:55:46 +01:00
Eugen Rochko 663c0edd8d Fix typo 2016-11-08 01:48:17 +01:00
Eugen Rochko 434c3fb2f7 ProcessFeedService refactor 2016-11-08 01:44:51 +01:00
Eugen Rochko 8f0869876b Improved configuration from ENV, cleaned up timeline filter methods
to be more readable, add extra logging to process feed service
2016-11-07 23:20:52 +01:00
Eugen Rochko cb22dce970 Adding hashtags 2016-11-05 17:13:14 +01:00
Eugen Rochko 4762bb6b7a Fix duplication of media attachments when a remote status reblogs a local one 2016-10-14 20:15:37 +02:00
Eugen Rochko 34bfc0f0dc Fix up a few exceptions 2016-10-12 19:25:46 +02:00
Eugen Rochko 1af00c8193 Fixing namespaces issue 2016-10-10 19:16:23 +02:00
Eugen Rochko 61d7c88408 Fix namespace parsing in Atom feeds 2016-10-10 18:16:07 +02:00
Eugen Rochko 80217fd58b Use Account#find_remote method when possible 2016-10-10 17:30:49 +02:00
Eugen Rochko 3d229e3937 Don't return unsaved status from ProcessFeedService 2016-10-10 16:49:05 +02:00
Eugen Rochko 33668b91f8 Adding test for ProcessFeedService 2016-10-10 16:03:38 +02:00
Eugen Rochko 5da603fb6f Improve process feed service 2016-10-10 15:27:39 +02:00
Eugen Rochko 7cb55af441 Fix default assumed object type (note->activity), make stream entry threaded? check aware of orphaned replies 2016-10-10 02:55:30 +02:00
Eugen Rochko b23c4b488c Better comparison of "local" domain 2016-10-06 16:36:16 +02:00
Eugen Rochko 70e9dd0b5b Blocking will prevent e-mail notifications from blocked user, blocks in UI 2016-10-03 18:49:52 +02:00
Eugen Rochko d74da1a89a Small fixes 2016-10-02 23:46:25 +02:00
Eugen Rochko 927333f4f8 Improve code style 2016-09-29 21:28:21 +02:00
Eugen Rochko c6b0311b86 Fix #54 - Fetch remote accounts by URL from mentions
Fetching atom extracted from FetchRemoteAccountService and FetchRemoteStatusService
into FetchAtomService. Mentions of the constant "http://activityschema.org/collection/public"
skipped as it's not a real URL/user.
2016-09-26 16:44:40 +02:00
Eugen Rochko b1a670af8d Handle remote account mentions a little better by trying a URL lookup in the db 2016-09-22 21:10:36 +02:00
Eugen Rochko 4a670780f0 Fix #51 - prevent duplicate attachments for remote statuses 2016-09-22 20:42:20 +02:00
Eugen Rochko 4bec613897 Fix #24 - Thread resolving for remote statuses
This is a big one, so let me enumerate:

Accounts as well as stream entry pages now contain Link headers that
reference the Atom feed and Webfinger URL for the former and Atom entry
for the latter. So you only need to HEAD those resources to get that
information, no need to download and parse HTML <link>s.

ProcessFeedService will now queue ThreadResolveWorker for each remote
status that it cannot find otherwise. Furthermore, entries are now
processed in reverse order (from bottom to top) in case a newer entry
references a chronologically previous one.

ThreadResolveWorker uses FetchRemoteStatusService to obtain a status
and attach the child status it was queued for to it.

FetchRemoteStatusService looks up the URL, first with a HEAD, tests
if it's an Atom feed, in which case it processes it directly. Next
for Link headers to the Atom feed, in which case that is fetched
and processed. Lastly if it's HTML, it is checked for <link>s to the Atom
feed, and if such is found, that is fetched and processed. The account for
the status is derived from author/name attribute in the XML and the hostname
in the URL (domain). FollowRemoteAccountService and ProcessFeedService
are used.

This means that potentially threads are resolved recursively until a dead-end
is encountered, however it is performed asynchronously over background jobs,
so it should be ok.
2016-09-21 01:50:31 +02:00
Eugen Rochko 059ebbf48d Separate PuSH subscriptions from following, add mastodon:push:refresh task,
respect hub.lease_seconds (fix #46)
2016-09-20 00:43:36 +02:00
Eugen Rochko adffc7a495 Fix #43 2016-09-18 12:28:49 +02:00
Eugen Rochko 183a23943b Fix how other services used old FollowRemoteAccountService 2016-09-17 17:07:45 +02:00
Eugen Rochko 3cc47beb6e Refactored generation of unique tags, URIs and object URLs into own classes,
as well as formatting of content
2016-09-09 20:04:34 +02:00
Eugen 1efa8e48d1 Fix enclosures not being parsed for reblogged statuses 2016-09-06 12:30:15 +02:00
Eugen Rochko eec0dc46a6 PostStatusService can attach media to status, ProcessFeedService likewise 2016-09-05 18:39:53 +02:00
Eugen Rochko a289c1d52f Handle delete Salmons, todo: clean up timelines 2016-09-04 14:44:16 +02:00
Eugen Rochko da4b675aca Fix favourite handling in ProcessInteractionService 2016-03-26 01:34:12 +01:00
Eugen Rochko 8eeec389c1 Add link to github project to footer, move FanOutOnWriteService calls to
DistributionWorker. That isn't the heaviest service, yet, but gotta start
somewhere
2016-03-25 03:22:26 +01:00
Eugen Rochko a08e724476 Fix subscriptions:clear task, refactor feeds, refactor streamable activites
and atom feed generation to some extent, as well as the way mentions are
stored
2016-03-25 02:13:30 +01:00
Eugen Rochko 97d5aba1ec Do not try to send notification e-mails for reblogs if the reblog is not of a
local account
2016-03-24 13:25:33 +01:00
Eugen Rochko b640f35621 Writing out more tests, fixed some bugs 2016-03-20 13:03:06 +01:00
Eugen Rochko 2b116131d7 Adding e-mail notifications about mentions, follows, favourites and reblogs. Fixing another mention recording bug 2016-03-19 19:20:07 +01:00
Eugen Rochko 66a2107527 Turns out that replies don't have to have the type comment. Changed handling
of those to depend on presence of <thr:in-reply-to /> instead
2016-03-19 11:59:23 +01:00
Eugen Rochko f97fc9744f Use FanOutOnWriteService AFTER processing mentions 2016-03-19 00:41:29 +01:00
Eugen Rochko 48aeacd5f9 Fixed ANOTHER first_or_create bug. Seriously who came up with that API smh 2016-03-18 23:47:02 +01:00
Eugen Rochko 9a7485d034 Fix typo on /api/accounts/:id/statuses, fix potential case-sensitivity issue
when registering incoming status mentions, add Travis CI file
2016-03-18 23:23:19 +01:00
Eugen Rochko d022975444 Fixing small bug with FollowRemoteAccountService usage 2016-03-16 21:20:50 +01:00
Eugen Rochko 75b3339a99 Respect "delete" verb on remote feeds 2016-03-16 10:46:15 +01:00
Eugen Rochko 23d08c6749 Changing the use of config constants to the Rails configuration object 2016-02-29 20:06:39 +01:00
Eugen Rochko 11ff92c9d7 Adding a test for ReblogService, fixing mentions for remote statuses 2016-02-28 21:22:56 +01:00
Eugen Rochko 47d50b0e39 A lot of fixes from a live test 2016-02-28 15:46:29 +01:00
Eugen Rochko ad5ae3f60e Update profile information and download avatar of remote accounts 2016-02-28 14:26:26 +01:00
Eugen Rochko 44c5958203 Fixed small bugs and code style offences 2016-02-25 00:22:46 +01:00
Eugen Rochko 71fe24096c Adding a Mention model, test stubs 2016-02-25 00:17:01 +01:00
Eugen Rochko 8da8387afe Add shared statuses to the database 2016-02-24 17:23:59 +01:00
Eugen Rochko fa7868675d Send Salmon interactions 2016-02-24 12:57:29 +01:00
Eugen Rochko 10eb47a33e Do not PuSH-subscribe to remote accounts when creating them for salmon 2016-02-24 03:05:40 +01:00
Eugen Rochko 9411e8c35a Feed processing except fetching remote statuses 2016-02-24 01:28:53 +01:00
Eugen Rochko 79baf2fd99 Process favourites, reblogs and replies from Salmon 2016-02-24 00:57:47 +01:00
Eugen Rochko 3b0bc18db9 Improved Atom rendering templates, added README 2016-02-23 13:08:01 +01:00
Eugen Rochko 709c6685a9 Made some progress 2016-02-22 16:00:20 +01:00