Commit Graph

95 Commits (6bf6a21658285d2baa4613888e35514a46d6d788)

Author SHA1 Message Date
Shunsuke Michii 37c8d4ce7e Make faster ProcessFeedService (#3080)
* Add index accounts on uri.

* Remove a blank line.
2017-05-16 12:12:29 +02:00
Eugen Rochko 12ef99556e Add conversation model, <ostatus:conversation /> (#3016)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Fix conversation migration

* More spec coverage for status before_create

* Prevent n+1 query when generating Atom with the new conversations

* Improve code style

* Remove redundant local variable
2017-05-12 19:09:21 +02:00
Eugen Rochko db707c7af3 Fix #1426 - Trim long usernames in public follower/following lists (#2993)
Fix #2221 - Catch OpenSSL exceptions when loading remote avatars/headers/attachments
Don't strip "rel" attribute from <a> tags when sanitizing (microformats)
2017-05-11 23:14:00 +02:00
Eugen Rochko 78df86a7c8 Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses (#2734)
* Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses

If a delete arrives before the original status, cache that information
for 6h, and if the original status arrives in that window, ignore it

* Add test case
2017-05-04 04:34:57 +02:00
Eugen Rochko 91a8a5eab8 Fix #2706 - Always respond with 200 to PuSH payloads (#2733)
Fix #2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix #2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
2017-05-03 17:02:18 +02:00
yhirano 7c65bcbdb7 Downgrade rubocop 0.48.1 => 0.46.0 (#2628)
* downgrade rubocop 0.48.1 => 0.46.0

* exclude vendor/**/* from rubocop target files

* add frozen_string_literal comment line

* fix percent literal delimited by ( and )

* fix alignment

* remove comment disabling unknown cop
2017-04-30 00:23:45 +02:00
Eugen Rochko 97dff125a8 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Eugen 87f7a3922c Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Eugen b9b9edd4c8 Add language detection (#1772)
* Add language detection via WhatLanguage and (de)serialization of it through Atom

* Fix default language in ProcessFeedService

* Re-add newline before 'react-rails' Gem to fix groupings

Fixes Code Climate issue
2017-04-16 20:32:17 +02:00
Eugen c2b4bafc33 Fix #1813 - Alleviate extra requests when processing mentions (#1938)
The <link rel="mentioned" /> tag refers to accounts by href. So we were
matching the DB by the url attribute, and falling back to HTTP look-up.
However, GS and Mastodon use profile URLs as URIs, too, and the match
for that was missing. This could potentially alleviate some extra network
requests
2017-04-16 18:01:48 +02:00
Eugen ba6b4c6e62 Make file attachment on MediaAttachment optional (#1865)
Create MediaAttachment but without actual file download when domain is blocked with reject_media set to true
Clean up old media files when creating a new domain block with reject_media set to true
Return remote_url in media attachments API if local file is not present
Undo domain block action in admin UI
Ability to enable reject_media from admin UI
2017-04-16 12:51:30 +02:00
ThibG c45c67c2ac Allow running mastodon on a different domain as the one used for identifying users (#1267)
* Allow running mastodon on a different domain as the one used for identifying users

* Alter documentation of WEB_DOMAIN to make clear it shouldn't be used unless the admin knows what they are doing

* Compare to web_domain instead of local_domain when dealing with feeds/API

* Correctly identify mentions to local accounts

Mentions URLs point to the person's web profile, i.e., the user page served on WEB_DOMAIN.
2017-04-15 02:15:46 +02:00
Eugen 697e4da6cb Fix #1141, fix #1126 - Avatar/profile info fetching (#1215)
* Fix #1141, fix #1126 - Work through UpdateRemoteProfileService for both <feed> and <entry> top-level tags

* Improve code quality, remove line unrelated to fix
2017-04-08 13:26:03 +02:00
Eugen Rochko cd68e54a7d Split SalmonWorker into smaller parts, move profile updating into another job 2017-04-05 21:43:10 +02:00
Eugen Rochko 27d4971979 Fix casuality of processing remote mentions such that notifications
about them would be processed only after the entire status is processed
2017-03-13 16:34:15 +01:00
Eugen Rochko b734ae3029 Skip remote media URLs that don't have a hostname 2017-02-22 19:55:14 +01:00
Eugen Rochko ce193afa4f Mentions in private statuses allow mentioned people to see them 2017-02-11 15:10:22 +01:00
Eugen Rochko 81be690f95 Fix #614 - extra reply-boolean on statuses to account for cases when replied-to
status is not in the system at time of distribution; fix #607 - reset privacy
settings to defaults when cancelling replies
2017-02-09 20:25:39 +01:00
Eugen Rochko 27ad4ce9e8 Do not run FetchLinkCardService on local URLs, increase file size limit to 8MB,
fix ProcessFeedService pushing status into distribution if called a second time
while the first is still running (i.e. when a PuSH comes after a Salmon slap),
fix not running escape on spoiler text before emojify
2017-01-27 16:57:23 +01:00
Eugen Rochko 4d011ee829 Use <summary> to encode content warnings instead 2017-01-25 16:53:30 +01:00
Eugen Rochko 959e064186 Instead of using spoiler boolean and spoiler_text, simply check for non-blank spoiler_text
Federate spoiler_text using warning attribute on <content /> instead of a <category term="spoiler" />
Clean up schema file from accidental development migrations
2017-01-25 01:29:16 +01:00
Eugen Rochko 3710d18889 Optional domain block attribute that prevents media attachments from being downloaded 2017-01-23 21:36:08 +01:00
Eugen Rochko d8d22cfa6f Instead of refusing to create accounts, domain blocks auto-suspend new accounts from that domain 2017-01-20 20:14:02 +01:00
Eugen Rochko 9d09b2c6b1 Fix preview cards layout, do preview card crawling for remote statuses 2017-01-20 18:31:49 +01:00
Eugen 4b384d6914 Fix too late return 2017-01-14 02:22:16 +01:00
Eugen Rochko 992c8d2c41 Add ruby version to Gemfile, move devDependencies in package.json to dependencies,
fix bug in process feed service
2017-01-11 15:39:31 +01:00
Eugen Rochko b7c2c5d81d Restoring old async behaviour of thread resolving as it proved to be more robust 2016-12-12 21:12:19 +01:00
Eugen Rochko fb36684678 Thread resolving no longer needs to be separate from ProcessFeedService,
since that is only ever called in the background
2016-12-11 22:23:11 +01:00
Eugen Rochko ca0757a6cf Add account suspension 2016-12-05 22:59:30 +01:00
Eugen Rochko 8d4ef0b6c3 Per-status control for unlisted mode, also federation for unlisted mode
Fix #233, fix #268
2016-11-30 21:34:59 +01:00
Eugen Rochko 23162b2893 Update hub URL and re-subscribe if hub URL changes 2016-11-26 15:18:21 +01:00
Eugen Rochko f07b0dc82f Remove unneeded indices, improve error handling in background workers, don't needlessly reload reblogged status, send Devise e-mails asynchronously 2016-11-22 17:32:51 +01:00
Eugen Rochko 3fc7820d6e Call NotifyService about remote reblogs 2016-11-21 09:56:01 +01:00
Eugen Rochko 38025dfea3 Adding unified streamable notifications 2016-11-20 19:39:58 +01:00
Eugen Rochko b373fb0ff6 Catch validation errors in ProcessFeedService so that one failing entry wouldn't stop others from going through 2016-11-18 23:19:38 +01:00
Eugen Rochko d2029a8845 Don't create attachments if remote file cannot be fetched 2016-11-18 23:16:34 +01:00
Eugen Rochko e71b152d89 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko 6f0d312dc3 Force utf-8 encoding when processing XML 2016-11-13 19:12:40 +01:00
Eugen Rochko 0b03bbad67 Discard misattributed remote statuses, improve timelines filter 2016-11-10 00:15:49 +01:00
Eugen Rochko c0ee14756f Adding more logging 2016-11-08 19:37:08 +01:00
Eugen Rochko ac79369f2b Fix for thread resolve service and process feed service url method 2016-11-08 19:09:22 +01:00
Eugen Rochko 9da30e31c8 Fix region setting for AWS gem 2016-11-08 18:55:46 +01:00
Eugen Rochko 663c0edd8d Fix typo 2016-11-08 01:48:17 +01:00
Eugen Rochko 434c3fb2f7 ProcessFeedService refactor 2016-11-08 01:44:51 +01:00
Eugen Rochko 8f0869876b Improved configuration from ENV, cleaned up timeline filter methods
to be more readable, add extra logging to process feed service
2016-11-07 23:20:52 +01:00
Eugen Rochko cb22dce970 Adding hashtags 2016-11-05 17:13:14 +01:00
Eugen Rochko 4762bb6b7a Fix duplication of media attachments when a remote status reblogs a local one 2016-10-14 20:15:37 +02:00
Eugen Rochko 34bfc0f0dc Fix up a few exceptions 2016-10-12 19:25:46 +02:00
Eugen Rochko 1af00c8193 Fixing namespaces issue 2016-10-10 19:16:23 +02:00
Eugen Rochko 61d7c88408 Fix namespace parsing in Atom feeds 2016-10-10 18:16:07 +02:00