Commit Graph

296 Commits (09b93aaf856371afcbcb32048ac047c4c2a1ac05)

Author SHA1 Message Date
Eugen Rochko 2af4f3c4e2 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Eugen Rochko 88725d6ce8 OEmbed support for PreviewCard (#2337)
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix #1681 - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
2017-04-27 14:42:22 +02:00
Matt Jankowski 8857cabca4 Domain block service cleanup (#2490)
* Add coverage for domain block service with silence

* Get rid of warning about find_each and order

* Move domain_block to attr_reader

* Move optional clear_media into silence_accounts method

* Use blocked_domain method to reduce passed vars

* Extract blocked_domain_accounts method to find accounts on the domain

* Extract media_from_blocked_domain method to find relevant attachments

* Separate destruction of account images and account attachments
2017-04-26 20:09:01 +02:00
Keiji, Yoshimi 8fe36654ef attach_media should be in transaction of creation status (#2426) 2017-04-26 03:47:44 +02:00
Eugen Rochko 8b5179d006 Fix #2402 - Add Idempotency-Key header to PostStatusService that prevents (#2419)
duplicates. Web UI regenerates UUID for that header every time the compose
form is changed or successfully submitted

Also, fix Farsi i18n overwriting the English one
2017-04-25 15:04:49 +02:00
178inaba d2159deaf2 Optimize account search (#2421) 2017-04-25 04:44:43 +02:00
Eugen 17c591ffba Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Yamagishi Kazutoshi d670f72830 Show multibyte URI preview card (#2363) 2017-04-23 19:08:59 +02:00
Eugen 9d3be5579a Improve bio compatibility (#2278)
* Fix #1057 (close #1819) - Move HTML-formatted bio from <poco:note /> to <summary type="html" />

* Ensure <poco:note /> is plaintext for remote accounts, also, by stripping out HTML
2017-04-22 03:33:24 +02:00
ik-fib 47a3036ea6 Add Mastodon version to user agent (#2253) 2017-04-21 17:26:23 +02:00
Eugen 1d47910d3b Fix possibility of unrightful webfinger redirect (#2147)
* Fix possibility of unrightful webfinger redirect

* Add more tests for FollowRemoteAccountService
2017-04-19 17:28:35 +02:00
Matt Jankowski 297c11dba2 Language detection refactor (#2099)
* Extract detect_language to separate class

* Use default locale, not just en

* Add spec to confirm that whatlanguage cant identify empty string

* Allow account locale to override default in language detector

* PostStatusService supplies an account to detect language
2017-04-18 22:20:12 +02:00
ik-fib 23ecc681c4 Add "Mastodon" to user agent (#2073)
Remove non-const version string

Freeze option is unnecessary
2017-04-18 16:04:13 +02:00
Tomohiro Suwa 3399dd7a66 Fix nil query_username (#2013) 2017-04-17 19:57:02 +02:00
Keiji, Yoshimi 7a5086729a reduce unneed query when post without attachements. (#1907)
* reduce unneed query when post without attachements.

This commit reduce following query:

```
 MediaAttachment Load (0.9ms)  SELECT "media_attachments".* FROM "media_attachments" WHERE "media_attachments"."status
_id" IS NULL AND 1=0 ORDER BY id asc
```

* fixed for more simple changes.
2017-04-17 04:23:13 +02:00
Eugen 8ebed7fc68 Fix #1957 - WhatLanguage can return null. Fallback to 'en' (#1959) 2017-04-16 23:12:19 +02:00
Eugen e4af4898de Add language detection (#1772)
* Add language detection via WhatLanguage and (de)serialization of it through Atom

* Fix default language in ProcessFeedService

* Re-add newline before 'react-rails' Gem to fix groupings

Fixes Code Climate issue
2017-04-16 20:32:17 +02:00
alpaca-tc 19f63ff801 Check @recipient.user at the first (#1939) 2017-04-16 18:04:05 +02:00
Eugen babbb2135e Fix #1813 - Alleviate extra requests when processing mentions (#1938)
The <link rel="mentioned" /> tag refers to accounts by href. So we were
matching the DB by the url attribute, and falling back to HTTP look-up.
However, GS and Mastodon use profile URLs as URIs, too, and the match
for that was missing. This could potentially alleviate some extra network
requests
2017-04-16 18:01:48 +02:00
Matt Jankowski e0b5a94a4b Clean up check that account needs a webfinger update (#1932) 2017-04-16 16:38:29 +02:00
alpaca-tc 8f8319852c Fixed NoMethodError in UnfollowService (#1918) 2017-04-16 14:55:43 +02:00
Eugen 5d710b1139 Make file attachment on MediaAttachment optional (#1865)
Create MediaAttachment but without actual file download when domain is blocked with reject_media set to true
Clean up old media files when creating a new domain block with reject_media set to true
Return remote_url in media attachments API if local file is not present
Undo domain block action in admin UI
Ability to enable reject_media from admin UI
2017-04-16 12:51:30 +02:00
alpaca-tc f9d7ec8971 ActiveRecord::Relation does not respond to `#id` (#1834) 2017-04-15 13:16:24 +02:00
Matt Jankowski 40fd1de488 Account search service refactor (#1791)
* Begin coverage for account search service

* Coverage for hashtag query

* Coverage for calling local vs remote find based on domain presence

* Spec to check that exact matches are not duped

* Coverage of resolve option

* Coverage for account being provided

* Start to refactor account search service

* Isolate query username and domain methods

* Isolate exact_match method

* Extract methods for local and remote results

* Simplify local vs remote and account isoliation

* Extract methods for local and remote results

* Simplify de-dupe of exact match

* Simplify logic to check for non exact remotes

* Cache some methods

* Remove nil from exact_match from results array

* Return exact matches first

* Use find_remote even with no domain

Account.find_local is just an alias for Account.find_remote(user, nil) - so we
can not bother with the conditional here, and call find_remote directly.
2017-04-15 03:17:07 +02:00
ThibG 31f0bcf804 Refresh webfinger (#1323)
* Refresh local info for remote accounts when webfinger returns new values

It only refreshes account info if one of the URLs or the public-key changes,
in which cases it refreshes the full info, re-downloading the feeds from that
user.

Some special handling should probably be done when the public key changes,
but I have been unable to find any use for it in Mastodon yet.

* Re-fetch remote users we aren't subscribed to.

This might induce performance issues, we might want to only do that for users
we explicitly attempted to subscribe but failed to.

* Refactor changes

* Do not refresh existing remote account details more than once a day

* Avoid re-fetching webfinger info in tests unless otherwise specified
2017-04-15 03:16:05 +02:00
ThibG a9529d3b4b Allow running mastodon on a different domain as the one used for identifying users (#1267)
* Allow running mastodon on a different domain as the one used for identifying users

* Alter documentation of WEB_DOMAIN to make clear it shouldn't be used unless the admin knows what they are doing

* Compare to web_domain instead of local_domain when dealing with feeds/API

* Correctly identify mentions to local accounts

Mentions URLs point to the person's web profile, i.e., the user page served on WEB_DOMAIN.
2017-04-15 02:15:46 +02:00
Matt Jankowski 323671a653 Silence more scope order warnings (#1604) 2017-04-12 15:58:08 +02:00
Matt Jankowski 53850bce93 Remove unused AtomBuilderHelper (#1364)
This was used in the views/atom/user_stream.xml.ruby file, which no longer is
used.
2017-04-11 19:31:22 +02:00
ThibG d19ed18388 Get handle from atom feed's author/email field instead of guessing from URL (#1344)
The goal of this change is to enhance Mastodon's handling of remote domains
for which the APIs reside on a different host (see issue #1032).

Indeed, when a remote user unknown to Mastodon is mentionned, only its profile
URL (e.g. https://social.example.org/users/User) is known, and Mastodon has to
build a @username@domain handle for it. To do so, Mastodon fetches the user's
atom feed (e.g., https://social.example.org/users/User.atom) and uses its
content to get the username part of the handle, and the URL's host part to
build the domain (e.g., @User@social.example.org). This handle is then used
for a Webfinger request.

In the case where example.org serves the Webfinger info for @User@example.org
and all feeds and APIs are hosted at social.example.org, Mastodon will still
build @User@social.example.org and fail at resolving the account's details
through Webfinger.

This patch changes this behaviour by using the author's email address from
the atom feed to build the handle. In Mastodon-generated atom feeds, the
email address is always the handle it expects for federation.
2017-04-09 18:43:48 +02:00
Eugen 982fef811e Fix #1141, fix #1126 - Avatar/profile info fetching (#1215)
* Fix #1141, fix #1126 - Work through UpdateRemoteProfileService for both <feed> and <entry> top-level tags

* Improve code quality, remove line unrelated to fix
2017-04-08 13:26:03 +02:00
Eugen 33849acfa7 Merge pull request #1218 from R0ckweb/patch-2
Fix #1141 on remote follow
2017-04-08 11:37:13 +02:00
Yann GUERN 485d75a805 #1141 on remote follow
The async action is send before persist, account.id not yet generated

Pull queue receive 'nil' so no profile update.
2017-04-08 03:24:35 +02:00
Eugen Rochko b2a7218ab7 Fix #801 - Respect webfinger's canonical response of username/domain 2017-04-08 01:07:42 +02:00
Chad Pytel ad5ddd5e95 Use I18n for media attachment validation errors
These are currently user facing errors, but are not localized. This adds the
ability for these messages to be localized.
2017-04-07 14:23:18 -04:00
Eugen 6d6a429af8 Rewrite Atom generation from stream entries to use Ox instead of Nokogiri (#1124)
* Rewrite Atom generation from stream entries to use Ox instead of Nokogiri::Builder

StreamEntry is now limited to only statuses, which allows some optimization. Removed
extra queries on AccountsController#show. AtomSerializer instead of AtomBuilderHelper
used in AccountsController#show, StreamEntriesController#show, StreamEntryRenderer
and PubSubHubbub::DistributionWorker

PubSubHubbub::DistributionWorker moves n+1 DomainBlock query to PubSubHubbub::DeliveryWorker
instead.

All Salmon slaps that aren't based on StreamEntry still use AtomBuilderHelper and Nokogiri

* All Salmon slaps now use Ox instead of Nokogiri. No touch from status on account
2017-04-07 05:56:56 +02:00
Eugen Rochko 51d7caaf19 Fix wrong pubsub channel on public timelines 2017-04-06 04:03:23 +02:00
Eugen Rochko dbd529109e Fix notifications delivered to wrong pubsub channel, optimized RemoveStatusService,
slightly optimized FanOutOnWriteService again
2017-04-06 02:26:59 +02:00
Eugen Rochko 5442083b3c Split SalmonWorker into smaller parts, move profile updating into another job 2017-04-05 21:43:10 +02:00
Eugen Rochko 5b95be1c42 Replace calls to FeedManager#inline_render and #broadcast 2017-04-05 19:45:18 +02:00
Eugen Rochko 220bc48e8e Only render public payload once in FanOutOnWrite 2017-04-05 14:26:17 +02:00
Eugen Rochko 6fd865c000 Spawn FeedInsertWorker to deliver status into personal feed 2017-04-04 19:21:37 +02:00
Eugen Rochko 82aaedec46 Reduce number of items in feeds, optimize regeneration worker slightly,
make regeneration worker unique, (only schedule/execute once at a time)
2017-04-04 13:58:34 +02:00
Eugen Rochko 8232f76c48 Add check for visibility.nil? even though it can't ever be, to check for race conditions 2017-04-03 22:54:46 +02:00
Eugen Rochko b7c1b12367 Make default admin UI page reports. Add admin UI for creating a domain block 2017-04-03 18:55:06 +02:00
Eugen Rochko d6b965cf08 Fix issue with feed merge-in code as well 2017-04-02 15:58:25 +02:00
Eugen Rochko e809caa0e1 Fix feed regeneration bug 2017-04-02 15:46:31 +02:00
Eugen Rochko d93d6f5124 Fix reworked search 2017-03-31 22:45:56 +02:00
Eugen Rochko d6ed2eb512 Prettier account and stream entry URLs 2017-03-22 19:55:14 +01:00
Eugen Rochko 33fac87e81 Shorter timeout on FetchAtomService 2017-03-22 17:41:52 +01:00
Eugen Rochko 5aa3df017b Fix full-text search query quotation, improve tag search performance with an index,
add ability to open status by URL from search (fix #53)
2017-03-22 17:36:34 +01:00