Commit Graph

196 Commits (2999c955968c8d8a6b4a35d13f9b4e7f6769b014)

Author SHA1 Message Date
Eugen Rochko f86ee4b59f
Fix IDN mentions not being processed, IDN domains not being rendered (#12715)
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.

Fix #7812, fix #12246
2019-12-30 19:20:43 +01:00
ThibG 2ee5a9d9c3 Clean up OStatus-related codepaths (#12173)
* Remove “protocol” argument and return value, as only ActivityPub is supported

* Remove FetchRemoteAccountService, only use ActivityPub::FetchRemoteAccountService

* Fix tests
2019-12-17 13:32:57 +01:00
Thomas Citharel 8094955461 Add Event activity-type support (#12637)
This adds support for Event AP type in Mastodon. Events are converted
into toots by taking their title (AS name) and their URL (AP ID). Event
picture is also brought in if available.

Testable by fetching event content from https://test.mobilizon.org

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2019-12-16 23:55:28 +01:00
ThibG c656cc2191 Fix FetchLinkCardServices crashing on a tags without a target (#12159)
* Add test for links without targets

* Fix FetchLinkCardServices crashing on a tags without a target
2019-11-21 16:04:52 +01:00
Eugen Rochko d14e74eff5
Add cache for OEmbed endpoints to avoid extra HTTP requests (#12403)
* add youtube oembed endpoint

* add check for oembed endpoint

* change unless for a more readable if

* clear blank lines

* endpoint via https

* Fix string literal in condition

* use cache for endpoints

* use cache for endpoints

* clean up and adding check

* clean up and remove redundant return

* add html check

* add false to return

* use double quotes

* use double quotes

* Clean up
2019-11-17 18:40:33 +01:00
BSKY fccf83e1f2 Add noopener and/or noreferrer (#12202) 2019-10-24 22:44:42 +02:00
Eugen Rochko eb83d6256e
Add `reason` param to `POST /api/v1/accounts` REST API (#12064)
For approval-required registrations mode
2019-10-03 17:50:59 +02:00
ThibG 3a4d994c40 Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid (#12037)
* Add test to handle suspended and missing users in BootstrapTimelineService

* Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid
2019-10-01 15:10:00 +02:00
Eugen Rochko 9ba40a6bfd
Remove HEAD request from fetching link previews (#12028)
It is not really necessary and we need to reduce requests
2019-10-01 04:54:10 +02:00
Eugen Rochko ab33c4df94
Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
2019-09-28 01:02:21 +02:00
ThibG 18b451c0e6 Change silences to always require approval on follow (#11975)
* Change silenced accounts to require approval on follow

* Also require approval for follows by people explicitly muted by target accounts

* Do not auto-accept silenced or muted accounts when switching from locked to unlocked

* Add `follow_requests_count` to verify_credentials

* Show “Follow requests” menu item if needed even if account is locked

* Add tests

* Correctly reflect that follow requests weren't auto-accepted when local account is silenced

* Accept follow requests from user-muted accounts to avoid leaking mutes
2019-09-27 21:13:51 +02:00
Eugen Rochko 8fdff2748f
Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
2019-08-16 01:24:03 +02:00
ThibG bced70469a Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Eugen Rochko b9fbcbfe4e
Add search syntax for operators and phrases (#11411) 2019-07-27 04:42:08 +02:00
Eugen Rochko bd87e66679
Remove WebSub subscriptions (#11303) 2019-07-21 04:08:00 +02:00
ThibG 730c4053d6 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
2019-07-19 01:44:42 +02:00
Eugen Rochko 4e8dcc5dbb
Add HTTP signatures to all outgoing ActivityPub GET requests (#11284) 2019-07-11 14:49:55 +02:00
Eugen Rochko 5d3feed191
Refactor fetching of remote resources (#11251) 2019-07-10 18:59:28 +02:00
Eugen Rochko 4e92183227
Refactor domain block checks (#11268) 2019-07-09 03:27:35 +02:00
Eugen Rochko b851456139
Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247) 2019-07-07 16:16:51 +02:00
Eugen Rochko 23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
ThibG 915c619394 Add support for Audio activities (#11189)
Fixes #11127
2019-06-26 19:32:36 +02:00
ThibG a353dc6030 Fix NotifyService test with regards to reblogs (#10928)
Fixes #10890
2019-06-02 18:08:26 +02:00
ThibG a1519a8ef5 Prevent from publicly boosting one's own private toots (#10775) 2019-05-18 00:28:51 +02:00
ThibG 14f6ce2885 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
ThibG 62f5235b6f Prevent silenced local users from notifying remote users not following them (#10575)
* Prevent silenced local users from notifying remote users not following them

This is an attempt to extend the local restrictions of silenced users to the
federation.

* Add tests

* Add tests for making sure private status don't get sent over OStatus
2019-05-09 22:05:43 +02:00
ThibG cb71c95e22 Export and import show_reblogs together with following list (#10495)
* Refactor imports

* Export show_reblogs when exporting list of followed users

* Add support for importing show_reblogs with following collection

* Fix tests
2019-04-08 07:28:27 +02:00
Eugen Rochko 67b3b62b98
Improve blocked view of profiles (#10491)
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"

This reverts commit 120544067f.

* Revert "Hide blocking accounts from blocked users (#10442)"

This reverts commit 62bafa20a1.

* Improve blocked view of profiles

- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
2019-04-07 04:59:13 +02:00
ThibG d4882aa64a Export and import `hide_notifications` alongside user mutes (#10335)
* Export hide_notifications along with user mutes

* Import hide_notifications along with muted users list

* Add headers for CSV exports
2019-04-03 18:17:43 +02:00
ThibG 62bafa20a1 Hide blocking accounts from blocked users (#10442)
* Revert "Add indication that you have been blocked in web UI (#10420)"

This reverts commit bd02ec6daa.

* Revert "Add `blocked_by` relationship to the REST API (#10373)"

This reverts commit 9745de883b.

* Hide blocking accounts from search results

* Filter blocking accouts from account followers

* Filter blocking accouts from account's following accounts

* Filter blocking accounts from “reblogged by” and “favourited by” lists

* Remove blocking account from URL search

* Return 410 on trying to fetch user data from a user who blocked us

* Return 410 in /api/v1/account/statuses for suspended or blocking accounts

* Fix status filtering when performing URL search

* Restore some React improvements

Restore some cleanup from bd02ec6daa

* Refactor by adding `without_blocking` scope
2019-04-01 20:06:13 +02:00
ThibG 2acd8940de Fix more keybase-related test failures (#10443) 2019-04-01 19:10:44 +02:00
ThibG abecaba317 Fix failing keybase-related test (#10428) 2019-03-30 18:15:23 +01:00
Eugen Rochko 1714ea5978
Add ActivityPub representation for identity proofs (#10414)
* Add ActivityPub representation for identity proofs

* Add tests
2019-03-30 02:12:06 +01:00
ThibG a20354a20b Set and store report URIs (#10303)
Fixes #10271
2019-03-17 15:34:56 +01:00
ThibG 5e38ef87a7 Fix reblogs privacy (#10302)
* Fix reblogs privacy

* Fix Announce processing specs
2019-03-17 14:54:09 +01:00
Eugen Rochko 51e154f5e8
Admission-based registrations mode (#10250)
Fix #6856
Fix #6951
2019-03-14 05:28:30 +01:00
ThibG c11dff5049 Reject existing Follows when suspending a remote account (#10230)
* Reject existing Follows when suspending a remote account

Partial fix to #10229

* Add tests
2019-03-10 16:18:58 +01:00
ThibG 9d3c6f1849 Improved remote thread fetching (#10106)
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
2019-02-28 15:22:21 +01:00
Eugen Rochko e7f20cc43f
Add type, limit, offset, min_id, max_id, account_id to search API (#10091)
* Add type, limit, offset, min_id, max_id, account_id to search API

Fix #8939

* Make the offset work on accounts and hashtags search as well

* Assure brakeman we are not doing mass assignment here

* Do not allow paginating unless a type is chosen

* Fix search query and index id field on statuses instead of created_at
2019-02-26 15:21:36 +01:00
ThibG e2a5be6e9a Prevent posting toots with media attachments from someone else (#9921) 2019-01-26 23:59:39 +01:00
ThibG 061feb63ed Fix scheduled toot with media immediately creating a toot (#9894)
* Add test for not persisting status when attaching media to scheduled toot

* Prevent status used for validation from being persisted to the database

Fixes #9893

Thanks to tateisu for the help investigating this.
2019-01-21 20:03:04 +01:00
ThibG aeb124491d Reject existing Follow in addition to sending a Block (#9811)
Mastodon expects remote servers to remove follow relationships upon receiving
a Block. However, the spec only evokes Block activities in a C2S context, never
in a S2S context.

This PR, in addition to federating the Block, explicitly sends a Reject for any
affected follow relationship, which makes a bit more sense with regards to the
spec.
2019-01-18 15:57:19 +01:00
Renato "Lond" Cerqueira 5c5e14c816 Fix undefined method error in sidekiq (#9807)
* Fix undefined method error in sidekiq

Body can be not nil but still be empty, which causes a
`NoMethodError: undefined method `[]' for nil:NilClass` further in the
code. This checks for an empty body to avoid the issue.

* Fix codeclimate issue
2019-01-14 17:28:41 +01:00
ysksn 9a38357111 Remove `pending` (#9752)
Some specs have already been added.
2019-01-08 09:42:56 +01:00
Eugen Rochko a49d43d112
Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
ThibG 290932602b Reduce usage of LD signatures (#9659)
* Do not LDS-sign Follow, Accept, Reject, Undo, Block

* Do not use LDS for Create activities of private toots

* Minor cleanup

* Ignore unsigned activities instead of misattributing them

* Use status.distributable? instead of querying visibility directly
2018-12-30 09:48:59 +01:00
Eugen Rochko 5d2fc6de32
Add REST API for creating an account (#9572)
* Add REST API for creating an account

The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.

The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.

The method is rate-limited by IP to 5 requests per 30 minutes.

* Redirect users back to app from confirmation if they were created with an app

* Add tests

* Return 403 on the method if registrations are not open

* Require agreement param to be true in the API when creating an account
2018-12-24 19:12:38 +01:00
ThibG 5f387995d9 Limit maximum visibility of local silenced users to unlisted (#9583)
Fixes #9580
2018-12-24 19:06:14 +01:00
ThibG e709b8da0d Ignore low-confidence CharlockHolmes guesses when parsing link cards (#9510)
* Add failing test for windows-1251 link cards

* Ignore low-confidence CharlockHolmes guesses

Fixes #9466

* Fix no method error when charlock holmes cannot detect charset
2018-12-17 19:19:45 +01:00
valerauko db9aea34de Ensure replied-to is a status not a boost (#9129)
* Ensure replied-to is a status not a boost

* Consider case of not a reply

* Add test case for replying to boost

* Move reblog-reply resolution to model

* Remove unnecessary comment
2018-11-25 16:35:21 +01:00