Commit Graph

95 Commits (a51c8074dfced016d967b062132d9d8213daf543)

Author SHA1 Message Date
Shunsuke Michii 0dfffb6dcb Make faster ProcessFeedService (#3080)
* Add index accounts on uri.

* Remove a blank line.
2017-05-16 12:12:29 +02:00
Eugen Rochko 5abdc77c80 Add conversation model, <ostatus:conversation /> (#3016)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Fix conversation migration

* More spec coverage for status before_create

* Prevent n+1 query when generating Atom with the new conversations

* Improve code style

* Remove redundant local variable
2017-05-12 19:09:21 +02:00
Eugen Rochko 58bcd50f7f Fix #1426 - Trim long usernames in public follower/following lists (#2993)
Fix #2221 - Catch OpenSSL exceptions when loading remote avatars/headers/attachments
Don't strip "rel" attribute from <a> tags when sanitizing (microformats)
2017-05-11 23:14:00 +02:00
Eugen Rochko 13c16b4e95 Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses (#2734)
* Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses

If a delete arrives before the original status, cache that information
for 6h, and if the original status arrives in that window, ignore it

* Add test case
2017-05-04 04:34:57 +02:00
Eugen Rochko bafd22ecf4 Fix #2706 - Always respond with 200 to PuSH payloads (#2733)
Fix #2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix #2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
2017-05-03 17:02:18 +02:00
yhirano 8325866c61 Downgrade rubocop 0.48.1 => 0.46.0 (#2628)
* downgrade rubocop 0.48.1 => 0.46.0

* exclude vendor/**/* from rubocop target files

* add frozen_string_literal comment line

* fix percent literal delimited by ( and )

* fix alignment

* remove comment disabling unknown cop
2017-04-30 00:23:45 +02:00
Eugen Rochko 2af4f3c4e2 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Eugen 17c591ffba Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Eugen e4af4898de Add language detection (#1772)
* Add language detection via WhatLanguage and (de)serialization of it through Atom

* Fix default language in ProcessFeedService

* Re-add newline before 'react-rails' Gem to fix groupings

Fixes Code Climate issue
2017-04-16 20:32:17 +02:00
Eugen babbb2135e Fix #1813 - Alleviate extra requests when processing mentions (#1938)
The <link rel="mentioned" /> tag refers to accounts by href. So we were
matching the DB by the url attribute, and falling back to HTTP look-up.
However, GS and Mastodon use profile URLs as URIs, too, and the match
for that was missing. This could potentially alleviate some extra network
requests
2017-04-16 18:01:48 +02:00
Eugen 5d710b1139 Make file attachment on MediaAttachment optional (#1865)
Create MediaAttachment but without actual file download when domain is blocked with reject_media set to true
Clean up old media files when creating a new domain block with reject_media set to true
Return remote_url in media attachments API if local file is not present
Undo domain block action in admin UI
Ability to enable reject_media from admin UI
2017-04-16 12:51:30 +02:00
ThibG a9529d3b4b Allow running mastodon on a different domain as the one used for identifying users (#1267)
* Allow running mastodon on a different domain as the one used for identifying users

* Alter documentation of WEB_DOMAIN to make clear it shouldn't be used unless the admin knows what they are doing

* Compare to web_domain instead of local_domain when dealing with feeds/API

* Correctly identify mentions to local accounts

Mentions URLs point to the person's web profile, i.e., the user page served on WEB_DOMAIN.
2017-04-15 02:15:46 +02:00
Eugen 982fef811e Fix #1141, fix #1126 - Avatar/profile info fetching (#1215)
* Fix #1141, fix #1126 - Work through UpdateRemoteProfileService for both <feed> and <entry> top-level tags

* Improve code quality, remove line unrelated to fix
2017-04-08 13:26:03 +02:00
Eugen Rochko 5442083b3c Split SalmonWorker into smaller parts, move profile updating into another job 2017-04-05 21:43:10 +02:00
Eugen Rochko a5daa806f2 Fix casuality of processing remote mentions such that notifications
about them would be processed only after the entire status is processed
2017-03-13 16:34:15 +01:00
Eugen Rochko 53ae431867 Skip remote media URLs that don't have a hostname 2017-02-22 19:55:14 +01:00
Eugen Rochko dc851c922e Mentions in private statuses allow mentioned people to see them 2017-02-11 15:10:22 +01:00
Eugen Rochko 6331ed16e5 Fix #614 - extra reply-boolean on statuses to account for cases when replied-to
status is not in the system at time of distribution; fix #607 - reset privacy
settings to defaults when cancelling replies
2017-02-09 20:25:39 +01:00
Eugen Rochko 450ad43180 Do not run FetchLinkCardService on local URLs, increase file size limit to 8MB,
fix ProcessFeedService pushing status into distribution if called a second time
while the first is still running (i.e. when a PuSH comes after a Salmon slap),
fix not running escape on spoiler text before emojify
2017-01-27 16:57:23 +01:00
Eugen Rochko 3beb24ad55 Use <summary> to encode content warnings instead 2017-01-25 16:53:30 +01:00
Eugen Rochko 999cde94a6 Instead of using spoiler boolean and spoiler_text, simply check for non-blank spoiler_text
Federate spoiler_text using warning attribute on <content /> instead of a <category term="spoiler" />
Clean up schema file from accidental development migrations
2017-01-25 01:29:16 +01:00
Eugen Rochko 434cf8237e Optional domain block attribute that prevents media attachments from being downloaded 2017-01-23 21:36:08 +01:00
Eugen Rochko 9bd3b11cfb Instead of refusing to create accounts, domain blocks auto-suspend new accounts from that domain 2017-01-20 20:14:02 +01:00
Eugen Rochko 05abd977c1 Fix preview cards layout, do preview card crawling for remote statuses 2017-01-20 18:31:49 +01:00
Eugen f63f0c4625 Fix too late return 2017-01-14 02:22:16 +01:00
Eugen Rochko 7951e7ffd5 Add ruby version to Gemfile, move devDependencies in package.json to dependencies,
fix bug in process feed service
2017-01-11 15:39:31 +01:00
Eugen Rochko 668013265c Restoring old async behaviour of thread resolving as it proved to be more robust 2016-12-12 21:12:19 +01:00
Eugen Rochko f90133d2ad Thread resolving no longer needs to be separate from ProcessFeedService,
since that is only ever called in the background
2016-12-11 22:23:11 +01:00
Eugen Rochko 39cc9fde8a Add account suspension 2016-12-05 22:59:30 +01:00
Eugen Rochko 14bd46946d Per-status control for unlisted mode, also federation for unlisted mode
Fix #233, fix #268
2016-11-30 21:34:59 +01:00
Eugen Rochko 2cb3dc5e5a Update hub URL and re-subscribe if hub URL changes 2016-11-26 15:18:21 +01:00
Eugen Rochko 45c7ee39b3 Remove unneeded indices, improve error handling in background workers, don't needlessly reload reblogged status, send Devise e-mails asynchronously 2016-11-22 17:32:51 +01:00
Eugen Rochko 29b12f9e0a Call NotifyService about remote reblogs 2016-11-21 09:56:01 +01:00
Eugen Rochko da2ef4d676 Adding unified streamable notifications 2016-11-20 19:39:58 +01:00
Eugen Rochko 2c58e7e06a Catch validation errors in ProcessFeedService so that one failing entry wouldn't stop others from going through 2016-11-18 23:19:38 +01:00
Eugen Rochko e4f56fa942 Don't create attachments if remote file cannot be fetched 2016-11-18 23:16:34 +01:00
Eugen Rochko fdc17bea58 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko 7ce4670164 Force utf-8 encoding when processing XML 2016-11-13 19:12:40 +01:00
Eugen Rochko aabf884c5f Discard misattributed remote statuses, improve timelines filter 2016-11-10 00:15:49 +01:00
Eugen Rochko b6832553ff Adding more logging 2016-11-08 19:37:08 +01:00
Eugen Rochko cff0b03cbb Fix for thread resolve service and process feed service url method 2016-11-08 19:09:22 +01:00
Eugen Rochko 2f21f4cc01 Fix region setting for AWS gem 2016-11-08 18:55:46 +01:00
Eugen Rochko 096bfbad96 Fix typo 2016-11-08 01:48:17 +01:00
Eugen Rochko 11cbe49ffc ProcessFeedService refactor 2016-11-08 01:44:51 +01:00
Eugen Rochko dbe00a4156 Improved configuration from ENV, cleaned up timeline filter methods
to be more readable, add extra logging to process feed service
2016-11-07 23:20:52 +01:00
Eugen Rochko 48b9619439 Adding hashtags 2016-11-05 17:13:14 +01:00
Eugen Rochko 91144d46ec Fix duplication of media attachments when a remote status reblogs a local one 2016-10-14 20:15:37 +02:00
Eugen Rochko c257b29d86 Fix up a few exceptions 2016-10-12 19:25:46 +02:00
Eugen Rochko 77efdfa110 Fixing namespaces issue 2016-10-10 19:16:23 +02:00
Eugen Rochko 87b618ab02 Fix namespace parsing in Atom feeds 2016-10-10 18:16:07 +02:00