Commit Graph

1151 Commits (8556ddd063faa081c4a11d8b9dc60ad17e438541)

Author SHA1 Message Date
ThibG 82262a6100 Fix invalid votes from the API being accepted (#12601)
* Fix invalid votes from the API being accepted

Fixes #12556

- Ensure `choice` is an integer instead of silently converting to 0
- Ensure `choice` corresponds to an actual choice of the poll

* Please CodeClimate
2020-01-12 14:17:03 +01:00
Daigo 3 Dango f994905ebb Clarify keyword arguments with ** (#12769)
This change is to suppress the warning below on on ruby-2.7.0:

- warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call

https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/
2020-01-11 05:57:05 +09:00
ThibG 057764b126 Use quality values in Accept field to explicitly prefer JSON over HTML (#12806) 2020-01-10 00:10:29 +01:00
Thibaut Girka 2591cedcad Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/application_controller.rb`:
  Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
  Conflict due to theming system.
2020-01-04 23:04:42 +01:00
ThibG 683b263a43 Fix URL search not returning private toots user has access to (#12742) 2020-01-03 05:01:45 +01:00
Eugen Rochko c72c27756f Fix IDN mentions not being processed, IDN domains not being rendered (#12715)
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.

Fix #7812, fix #12246
2019-12-30 19:20:43 +01:00
chr v1.x 38ccdb696f Fix backups stopping due to read timeouts (#12281)
* Make BackupService resilient to read timeouts

If an attachment read times out, assume that the resources is
inaccessible and continue the backup without it. This fixes #12280.

* Both errors on one line
2019-12-30 03:17:54 +01:00
Thibaut Girka fa4a92e340 Merge branch 'master' into glitch-soc/master
Conflicts:
- `config/locales/en.yml`
  No real conflict, upstream added a translatable string “too close” to
  one specific to glitch-soc
- `lib/mastodon/statuses_cli.rb`
  Fixes made upstream, while changed in glitch-soc to keep bookmarked statuses
- `package.json`
  No real conflict, additional dependency in glitch-soc
2019-12-19 13:52:54 +01:00
ThibG fc94f3bd12 Fix link crawler not specifying accepted content-type (#12646)
The link crawler expects HTML documents, so set the `Accept`
header accordingly.

Fixes #12618
2019-12-18 16:56:06 +01:00
Takeshi Umeda b08d72ffec Add signature to exported status (#12649) 2019-12-18 16:55:21 +01:00
Takeshi Umeda df28debc53 Fix an error when ActivityPub::FetchRemoteStatusService url is called with nil (#12652) 2019-12-18 16:54:03 +01:00
ThibG 357bee5ab9 Clean up OStatus-related codepaths (#12173)
* Remove “protocol” argument and return value, as only ActivityPub is supported

* Remove FetchRemoteAccountService, only use ActivityPub::FetchRemoteAccountService

* Fix tests
2019-12-17 13:32:57 +01:00
Alice Gaudon a285b3c63c Admin setting to disable default follows (#12566) 2019-12-16 23:55:50 +01:00
Thibaut Girka 2cc60e8b09 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- package.json
  Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
  Not really a conflict, caused by an additional dependency in glitch-soc.
2019-12-07 12:28:59 +01:00
trwnh a2f67154a4 Fix account search with no query (#12549)
* Fix account search with no query

Modeled after #12541. Fix #12548

* fix codeclimate
2019-12-06 19:44:23 +01:00
Eugen Rochko 5d810aae4f Fix error when using search API with no query (#12541)
Fix #12462
2019-12-04 04:34:08 +01:00
ThibG 60d26cabf0 Add follow_request notification type (#12198)
* Add follow_request notification type

The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.

Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.

Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.

* Show follow request link in single-column if there are pending requests, even if account isn't locked

* Push follow requests from notifications to the follow_requests list

* Offer to accept or reject follow request from the notification

* Redesign follow request notification
2019-12-01 17:25:29 +01:00
Thibaut Girka e48f63787d Merge branch 'master' into glitch-soc/merge-upstream 2019-11-30 17:29:44 +01:00
ThibG 1fc03686ea Fix empty poll options not being filtered on remote poll update (#12484)
If a poll contains empty options (which is apparently possible on Pleroma),
it is created without them. However, the poll update code doesn't filter
empty options, and thus:
1. Clear known votes, as it assumes the set of options has changed
2. Errors out because it tries adding empty options, which fails validation

This commit fixes that by filtering them out the same way they are filtered
out at poll creation time.
2019-11-26 22:46:31 +01:00
Thibaut Girka 5aa79af7cb Merge branch 'master' into glitch-soc/merge-upstream 2019-11-23 19:03:01 +01:00
Eugen Rochko 0342c42e05 Fix OEmbed discovery not handling different URL variants in query (#12439)
Fix #12433
2019-11-21 19:03:07 +01:00
ThibG 32ef133fa6 Fix FetchLinkCardServices crashing on a tags without a target (#12159)
* Add test for links without targets

* Fix FetchLinkCardServices crashing on a tags without a target
2019-11-21 16:04:52 +01:00
ThibG 27b7ec1043 Change domain block behavior to update user records before deleting data (#12247)
When suspending or silencing a domain, one probably wants the silence or
suspend parts to take effect as soon as possible. Deleting media files can
take a while (possibly days), so perform silencing as soon as possible, and
clean up media afterwards.
2019-11-21 11:36:15 +01:00
Thibaut Girka a96375d2ef Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
  discarded upstream changes
- app/controllers/api/v1/bookmarks_controller.rb
  finally merged upstream, some code style fixes
  and slightly changed pagination code
- app/controllers/application_controller.rb
  changed upstream to always return HTML error pages
  slight conflict caused by theming code
- app/models/bookmark.rb
  finally merged upstream, no real conflict
- spec/controllers/api/v1/bookmarks_controller_spec.rb
  finally merged upstream, slightly changed pagination code
2019-11-20 15:36:09 +01:00
Eugen Rochko 357a2e5564 Add cache for OEmbed endpoints to avoid extra HTTP requests (#12403)
* add youtube oembed endpoint

* add check for oembed endpoint

* change unless for a more readable if

* clear blank lines

* endpoint via https

* Fix string literal in condition

* use cache for endpoints

* use cache for endpoints

* clean up and adding check

* clean up and remove redundant return

* add html check

* add false to return

* use double quotes

* use double quotes

* Clean up
2019-11-17 18:40:33 +01:00
ThibG 517c1cd062 Add bookmarks (#7107)
* Add backend support for bookmarks

Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.

* Add spec for bookmark endpoints

* Add front-end support for bookmarks

* Introduce OAuth scopes for bookmarks

* Add bookmarks to archive takeout

* Fix migration

* Coding style fixes

* Fix rebase issue

* Update bookmarked_statuses to latest UI changes

* Update bookmark actions to properly reflect status changes in state

* Add bookmarks item to single-column layout

* Make active bookmarks red
2019-11-13 23:02:10 +01:00
ThibG 78e251fc77 Fix already-known remote private toots not being searchable by URL (#12336)
* Fix already-known remote private toots not being searchable by URL

Whenever an URL search fails, and an account is provided, try getting a
private status matching that URL from database.

* Fix searching by `id` field instead of `uri`
2019-11-11 00:04:48 +02:00
Thibaut Girka 0fc784f8d0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`
  conflict because of the poll option limit being different than upstream's
2019-11-07 13:43:56 +01:00
ThibG 28b68ec875 Fix various issues with account migration (#12301)
* Fix being able to follow oneself by moving to an account that was following the old one

* Add specs

* Add spec to catch MoveWorker issue with local followers following both accounts

* Fix move worker breaking when a local account follows both source and target accounts

* Fix migration from remote to local account not sending Undo Follow

* Fix show_reblogs not being preserved for moved account's followers
2019-11-07 09:05:07 +02:00
Thibaut Girka e7655951d7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`
2019-11-06 14:38:28 +01:00
ThibG 6b5aebf0ca Add abilityto add oneself to lists (#12271)
* Add ability to add oneself to lists

* Change search results to include oneself when searching through followers

* Mark follow relation as optional in ListAccount
2019-11-04 13:02:01 +01:00
Thibaut Girka e76f4056e5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/helpers/statuses_helper.rb
  Upstream moved account helpers to their own file, we had extra
  helpers there, moved too.
- app/lib/sanitize_config.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- config/locales/simple_form.en.yml
- spec/lib/sanitize_config_spec.rb
2019-10-27 12:14:01 +01:00
nightpool d4ddfc486a microformat mentions can have an implicit property (#12189)
See the first example here: http://microformats.org/wiki/microformats2#hyperlinked_person
2019-10-24 22:46:15 +02:00
Thibaut Girka 9506d9cafc Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/javascript/styles/mastodon/components.scss
  conflicts caused by image URLs being different
- app/models/status.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
- app/views/statuses/_simple_status.html.haml
- config/locales/en.yml
  some strings were changed upstream
- spec/models/status_spec.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
2019-10-10 17:26:08 +02:00
Eugen Rochko b665e99b2f Fix hashtag timeline REST API accepting too many hashtags (#12091) 2019-10-07 03:56:21 +02:00
Thibaut Girka 07abea7ba8 Merge tag 'v3.0.0' into glitch-soc/merge-upstream 2019-10-04 19:49:43 +02:00
Eugen Rochko 059287ad08 Add `reason` param to `POST /api/v1/accounts` REST API (#12064)
For approval-required registrations mode
2019-10-03 17:50:59 +02:00
Thibaut Girka c2d6ef1b64 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream raised max image size from 8MB to 10MB while our limit is
  configurable. Raised the default to 10MB.
2019-10-03 11:10:12 +02:00
ThibG 081cae4c22 Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid (#12037)
* Add test to handle suspended and missing users in BootstrapTimelineService

* Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid
2019-10-01 15:10:00 +02:00
Eugen Rochko 67713f8b08 Remove HEAD request from fetching link previews (#12028)
It is not really necessary and we need to reduce requests
2019-10-01 04:54:10 +02:00
abcang 2b89873dfc Fix follow requests N+1 (#12004) 2019-09-30 17:02:03 +02:00
Thibaut Girka cc94c90bf1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- Gemfile.lock
- app/controllers/about_controller.rb
- app/controllers/auth/sessions_controller.rb
2019-09-30 12:23:57 +02:00
ThibG f088e090e4 Add voters count support (#11917)
* Add voters count to polls

* Add ActivityPub serialization and parsing of voters count

* Add support for voters count in WebUI

* Move incrementation of voters count out of redis lock

* Reword “voters” to “people”
2019-09-29 22:58:01 +02:00
Eugen Rochko ae03161ad9 Fix account migration not affecting followers on origin server (#11980) 2019-09-29 16:23:13 +02:00
Eugen Rochko 386dc65671 Fix preview card image not being re-fetched even if link is re-posted (#11981)
Fix #11956
2019-09-28 01:33:16 +02:00
Eugen Rochko 66d3e13777 Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
2019-09-28 01:02:21 +02:00
ThibG 70990720c5 Change silences to always require approval on follow (#11975)
* Change silenced accounts to require approval on follow

* Also require approval for follows by people explicitly muted by target accounts

* Do not auto-accept silenced or muted accounts when switching from locked to unlocked

* Add `follow_requests_count` to verify_credentials

* Show “Follow requests” menu item if needed even if account is locked

* Add tests

* Correctly reflect that follow requests weren't auto-accepted when local account is silenced

* Accept follow requests from user-muted accounts to avoid leaking mutes
2019-09-27 21:13:51 +02:00
Eugen Rochko e2372fcdc2 Fix unreviewed hashtag not being found by exact case-insensitive match (#11976) 2019-09-27 19:40:26 +02:00
Jeong Arm 565a36ab61 Fix search error when ElasticSearch is enabled but not available (#11954)
* Fallback to Database search when ES not available

* Prevent double work if ES gives 0 result

* Apply suggestion from code review
2019-09-26 18:06:08 +02:00
Thibaut Girka 4837d6275a Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb
  Minor conflict due to glitch-soc's theming code
2019-09-18 17:25:56 +02:00
Eugen Rochko 195ea5bce5 Change spam check to apply to local accounts and add a threshold (#11806)
Instead of detecting spam on first duplicate message, add a
threshold of 5 such messages to reduce false positives
2019-09-18 12:53:13 +02:00
Eugen Rochko d321515c11 Fix Move handler queuing jobs that will fail if account is suspended (#11864)
Don't put Move handler on cooldown if it didn't run. Skip unmerging
from timelines to save unnecessary work.
2019-09-17 08:44:45 +02:00
Thibaut Girka 2e38d889a3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
  Conflict because we changed the number of default results to be
  configurable
- app/lib/settings/scoped_settings.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
2019-09-13 18:13:43 +02:00
Eugen Rochko ee1c152083 Fix search API not resolving URL when type is given (#11822)
Fix #11771
2019-09-13 16:01:39 +02:00
Eugen Rochko 4291b74031 Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Thibaut Girka eac7c66d2a Merge branch 'master' into glitch-soc/merge-upstream 2019-09-05 11:36:41 +02:00
Eugen Rochko 7ef93513f7 Change account deletion page to have better explanations (#11753)
Fix deletion of unconfirmed account not freeing up the username

Add prefill of logged-in user's email in the reconfirmation form
2019-09-04 04:13:54 +02:00
Thibaut Girka c6591e1283 Fix auto-added media icons when a CW is set but no content text 2019-08-31 18:23:24 +02:00
Thibaut Girka 741b53651e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/directories/index.html.haml
  Upstream has redesigned the profile directory, and we
  had a glitch-soc-specific change to hide follower counts.
  Ported that change to the new design.
2019-08-30 13:38:41 +02:00
Eugen Rochko 79922ae20f Add profile directory to web UI (#11688)
* Add profile directory to web UI

* Add a line of bio to the directory
2019-08-30 00:14:36 +02:00
Thibaut Girka 449a0727a7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb

All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
2019-08-29 12:07:50 +02:00
Eugen Rochko e4605bbb4a Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
2019-08-22 04:17:12 +02:00
Thibaut Girka c060523dc2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/home_controller.rb
- app/controllers/shares_controller.rb
- app/javascript/packs/public.js
- app/models/status.rb
- app/serializers/initial_state_serializer.rb
- app/views/home/index.html.haml
- app/views/layouts/public.html.haml
- app/views/public_timelines/show.html.haml
- app/views/shares/show.html.haml
- app/views/tags/show.html.haml
- config/initializers/content_security_policy.rb
- config/locales/en.yml
- config/webpack/shared.js
- package.json
2019-08-19 21:49:35 +02:00
Eugen Rochko 848cb2a697 Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
2019-08-18 03:45:51 +02:00
Eugen Rochko 291f316022 Fix accounts search by full/partial display name and others (#11580)
- Restrict followers counts to local users to minimize local advantage
- Fix emoji shortcodes causing error in search
- Fix search syntax parse errors not being caught
2019-08-16 13:00:30 +02:00
Eugen Rochko 6393a094c6 Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
2019-08-16 01:24:03 +02:00
Thibaut Girka f0362fa0ee Merge branch 'master' into glitch-soc/merge-upstream 2019-08-12 16:24:22 +02:00
Eugen Rochko 826bc61c2c Fix account search always returning exact match on paginated results (#11525)
Fix #11365
2019-08-11 13:14:51 +02:00
Thibaut Girka 81c6f2fd0e Merge branch 'master' into glitch-soc/merge-upstream 2019-08-08 08:44:33 +02:00
Eugen Rochko 10b486f627 Fix acct URIs with IDN domains not being resolved (#11520)
Fix #11494
2019-08-07 21:14:08 +02:00
ThibG 072e696aa1 Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Thibaut Girka ff76f6de4f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/views/admin/dashboard/index.html.haml
- config/locales/simple_form.en.yml
2019-08-07 13:56:48 +02:00
Eugen Rochko 277acb8772 Fix non-lowercase hashtags not being picked up by the streaming API (#11508)
Regression from b838607

Fix hashtag links always being lowercase
2019-08-07 10:01:19 +02:00
Thibaut Girka 4973ba2d1f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/directories_controller.rb
- package.json
- yarn.lock
2019-07-30 12:22:33 +02:00
Eugen Rochko 825dc3ca22 Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Thibaut Girka 6c7a3268c9 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-28 16:28:05 +02:00
Eugen Rochko b8386076c5 Change hashtags to preserve first-used casing (#11416) 2019-07-28 05:59:51 +02:00
Eugen Rochko 9c617b3b3a Add search syntax for operators and phrases (#11411) 2019-07-27 04:42:08 +02:00
ThibG 52cda33b82 Fix invites not being disabled upon account suspension (#11412)
* Disable invite links from disabled/suspended users

* Add has_many invites relationship to users

* Destroy unused invites when suspending an account
2019-07-26 18:55:33 +02:00
Eugen Rochko 93a6ab5a32 Change account domain block to clear out notifications and follows (#11393) 2019-07-25 04:17:35 +02:00
Thibaut Girka 579ccb0bb9 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/application_controller.rb
- app/controllers/auth/confirmations_controller.rb
- app/controllers/auth/sessions_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
2019-07-23 10:51:07 +02:00
Eugen Rochko f9ed4354b1 Remove WebSub subscriptions (#11303) 2019-07-21 04:08:00 +02:00
Thibaut Girka eecce7e59b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile.lock
- app/controllers/accounts_controller.rb
- app/controllers/admin/dashboard_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/following_accounts_controller.rb
- app/controllers/remote_follow_controller.rb
- app/controllers/stream_entries_controller.rb
- app/controllers/tags_controller.rb
- app/javascript/packs/public.js
- app/lib/sanitize_config.rb
- app/models/account.rb
- app/models/form/admin_settings.rb
- app/models/media_attachment.rb
- app/models/stream_entry.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/services/batched_remove_status_service.rb
- app/services/post_status_service.rb
- app/services/process_mentions_service.rb
- app/services/reblog_service.rb
- app/services/remove_status_service.rb
- app/views/admin/settings/edit.html.haml
- config/locales/simple_form.pl.yml
- config/settings.yml
- docker-compose.yml
2019-07-19 18:26:49 +02:00
ThibG a7b0224699 Disable LDSigning when AUTHORIZED_FETCH is set to true (#11295) 2019-07-15 02:29:39 +02:00
ThibG 3f12a0b8fd Fix leaking private statuses the admin account follows (#11300)
Now that the request is signed, it can return private toots. Do not leak them.
2019-07-15 02:29:04 +02:00
ThibG 33545631e1 Fix BlockService trying to reject incorrect follow request (#11288)
Fixes #11148
2019-07-15 00:48:50 +02:00
ThibG 1d8a374f2a Fix BackupService crashing when an attachment is missing (#11241)
* Fix BackupService crashing when an attachment is missing

For various reasons such as admin error or out-of-sync media and
database backups, it might be possible for local attachments to be lost.

This commit allows the BackupService to continue its work even if some media
file is missing.

* Change error message
2019-07-15 00:48:49 +02:00
Eugen Rochko f70b20a01c Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
2019-07-13 16:45:50 +02:00
ThibG 4bf0ee9467 Fix BlockService trying to reject incorrect follow request (#11288)
Fixes #11148
2019-07-11 14:50:27 +02:00
Eugen Rochko 317b79d673 Add HTTP signatures to all outgoing ActivityPub GET requests (#11284) 2019-07-11 14:49:55 +02:00
Eugen Rochko 6baf5099a6 Refactor fetching of remote resources (#11251) 2019-07-10 18:59:28 +02:00
ThibG e39e4786b4 Fix handling of webfinger redirects in ResolveAccountService (#11279) 2019-07-10 17:10:12 +02:00
Eugen Rochko d0b0b63b1a Refactor domain block checks (#11268) 2019-07-09 03:27:35 +02:00
Eugen Rochko 56f0203c66 Refactor controllers for statuses, accounts, and more (#11249) 2019-07-08 12:03:45 +02:00
ThibG 12d6c0ff9b Fix BackupService crashing when an attachment is missing (#11241)
* Fix BackupService crashing when an attachment is missing

For various reasons such as admin error or out-of-sync media and
database backups, it might be possible for local attachments to be lost.

This commit allows the BackupService to continue its work even if some media
file is missing.

* Change error message
2019-07-08 02:24:25 +02:00
ThibG 53b0221e7d Merge pull request #1152 from ThibG/glitch-soc/merge-upstream
Merge upstream changes
2019-07-07 22:26:10 +02:00
Eugen Rochko fbbcbd940d Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247) 2019-07-07 16:16:51 +02:00
Eugen Rochko 4931208dd8 Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Thibaut Girka 017b0ea6ff Allow serializing local-only toots in backup service
Fixes #1153
2019-07-05 21:50:19 +02:00
Thibaut Girka 9866c2bfff Merge branch 'master' into glitch-soc/merge-upstream 2019-07-04 16:21:39 +02:00
Eugen Rochko 9ac82421f8 Change domain block behaviour to prevent creation of accounts from suspended domains (#11219) 2019-07-01 19:13:30 +02:00
Thibaut Girka ec1fcf1584 Use a redis-cached feed for the DM timeline 2019-06-27 16:44:12 +02:00
Thibaut Girka acbed64f3d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/preferences_controller.rb
- app/lib/user_settings_decorator.rb
- app/models/user.rb
- config/locales/simple_form.en.yml
2019-06-26 23:19:22 +02:00
ThibG 28806d5d6d Add support for Audio activities (#11189)
Fixes #11127
2019-06-26 19:32:36 +02:00
Thibaut Girka 1177694c3e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream added audio attachment support
- app/serializers/initial_state_serializer.rb
  Upstream added audio attachment support and how mimetypes are returned
- app/serializers/rest/instance_serializer.rb
  Upstream added a few fields
- config/application.rb
  Upstream added a different paperclip transcoder
2019-06-24 15:02:59 +02:00
Eugen Rochko eac19f2300 Change domain blocks to automatically support subdomains (#11138)
* Change domain blocks to automatically support subdomains

If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)

* Match subdomains of existing accounts when blocking/unblocking domains

* Improve code style
2019-06-22 00:13:10 +02:00
Eugen Rochko d7e2f554fb Add audio uploads (#11123)
* Add audio uploads

Fix #4827

Accept uploads of OGG, WAV, FLAC, OPUS and MP3 files, and converts
them to OGG. Media attachments get a new `audio` type. In the UI,
audio uploads are displayed identically to video uploads.

* Improve code style
2019-06-19 23:42:38 +02:00
Thibaut Girka 00552d2f79 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
- app/controllers/stream_entries_controller.rb
2019-06-07 17:00:36 +02:00
Eugen Rochko 57098a1b8d Refactor all ActivityPub deliveries to be serialized and signed through one concern (#10966) 2019-06-04 23:11:18 +02:00
Thibaut Girka 08a793cbd9 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-18 10:04:32 +02:00
ThibG 1fec2ff78f Prevent from publicly boosting one's own private toots (#10775) 2019-05-18 00:28:51 +02:00
Thibaut Girka 7aece1eec5 Add option for default toot content-type 2019-05-17 23:51:14 +02:00
Thibaut Girka 91ab9e2e4f Add API support for setting status content-type 2019-05-17 23:51:14 +02:00
Thibaut Girka 422dd06806 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-15 17:11:40 +02:00
ThibG f1a3135809 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
Thibaut Girka 73bc58c656 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
2019-05-04 16:37:26 +02:00
Thibaut Girka c4865cbd25 Add site-wide options to show reblogs and replies in local/public timelines
Fixes #1021
2019-05-01 22:33:09 +02:00
ThibG 140be28787 Reject follow requests of blocked users (#10633) 2019-04-25 02:47:33 +02:00
Thibaut Girka 2d2eca5bda Do not leak local-only toots to remote mentioned users 2019-04-22 23:37:50 +02:00
Thibaut Girka 6f85bb754b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/javascript/mastodon/features/compose/components/compose_form.js
  Upstream cleaned up a bit, including on lines in which
  we replaced the hardcoded 500 character limit with a maxChar
  constant. Applied the changes while keeping maxChar instead of 500.
- app/javascript/packs/public.js
  Moved upstream's new animated avatar hover handling in
  app/javascript/core/public.js
- app/javascript/styles/fonts/montserrat.scss
  Upstream fixed local font name, applied those changes.
- app/javascript/styles/fonts/roboto.scss
  Upstream fixed local font name, applied those changes.
- lib/mastodon/version.rb
  Upstream made repo URL configurable, did the same, but
  default to glitch-soc
2019-04-22 20:40:04 +02:00
Daniel Aleksandersen e3dda38c8b Treat meta[property] as a space-separated list (#10604)
The @property attribute in HTML is a space-separated list of values.
This change normalizes whitespace and finds the desired value in
the list instead of requiring an exact single-value match.

More details:
https://www.ctrl.blog/entry/rdfa-socialmedia-metadata.html
2019-04-21 04:48:19 +02:00
Thibaut Girka 5c8f9a3220 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/pl.yml
  Conflict caused by new upstream string too close to glitch-specific
  “flavour” string. Took both strings.
2019-04-08 15:57:56 +02:00
ThibG 110dbb5286 Export and import show_reblogs together with following list (#10495)
* Refactor imports

* Export show_reblogs when exporting list of followed users

* Add support for importing show_reblogs with following collection

* Fix tests
2019-04-08 07:28:27 +02:00
Eugen Rochko c84c30c542 Improve blocked view of profiles (#10491)
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"

This reverts commit 0317f37c6f.

* Revert "Hide blocking accounts from blocked users (#10442)"

This reverts commit 4cd944d364.

* Improve blocked view of profiles

- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
2019-04-07 04:59:13 +02:00
Thibaut Girka a835ea0e8f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/following_accounts_controller.rb
  Conflicts were due to glitch-soc's theming system.
- app/javascript/packs/public.js
  Some code has been change upstream, but it has been
  moved to app/javascript/core/settings.js in glitch-soc.
  Applied the changes there.
2019-04-04 14:53:52 +02:00
ThibG e87214826b Export and import `hide_notifications` alongside user mutes (#10335)
* Export hide_notifications along with user mutes

* Import hide_notifications along with muted users list

* Add headers for CSV exports
2019-04-03 18:17:43 +02:00
Thibaut Girka 5eb2632410 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/form/admin_settings.rb
- config/locales/ja.yml
2019-04-01 21:28:31 +02:00
ThibG 4cd944d364 Hide blocking accounts from blocked users (#10442)
* Revert "Add indication that you have been blocked in web UI (#10420)"

This reverts commit d31affe69b.

* Revert "Add `blocked_by` relationship to the REST API (#10373)"

This reverts commit a8bb10633d.

* Hide blocking accounts from search results

* Filter blocking accouts from account followers

* Filter blocking accouts from account's following accounts

* Filter blocking accounts from “reblogged by” and “favourited by” lists

* Remove blocking account from URL search

* Return 410 on trying to fetch user data from a user who blocked us

* Return 410 in /api/v1/account/statuses for suspended or blocking accounts

* Fix status filtering when performing URL search

* Restore some React improvements

Restore some cleanup from d31affe69b

* Refactor by adding `without_blocking` scope
2019-04-01 20:06:13 +02:00
Thibaut Girka 99336afaff Merge branch 'master' into glitch-soc/merge-upstream 2019-03-30 12:36:24 +01:00
Eugen Rochko a82bc7f5ae Add ActivityPub representation for identity proofs (#10414)
* Add ActivityPub representation for identity proofs

* Add tests
2019-03-30 02:12:06 +01:00
Thibaut Girka 0117de0a01 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/workers/activitypub/distribute_poll_update_worker.rb
- config/locales/pl.yml
2019-03-28 13:01:33 +01:00
Eugen Rochko 364e49ae1f Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
2019-03-28 04:44:59 +01:00
Thibaut Girka e97d76632d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/settings_controller.rb
- app/models/form/admin_settings.rb

Conflicts caused by upstream refactoring, while we have
flavours and skins, with the flavour_and_skin pseudo-setting.
2019-03-26 16:01:59 +01:00
Eugen Rochko 2b25f64bf1 Fix vote-after-expiration not showing error, count vote as interaction (#10352) 2019-03-23 14:07:17 +01:00
Thibaut Girka 8410fcd3a7 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-22 13:05:17 +01:00
ThibG f398b9ce02 Fix processing poll Updates (#10333)
ActivityPub::ProcessPollService was checking the JSON-LD context although
it was passed only the `Question` object embedded in the `Update` activity.
2019-03-21 14:24:09 +01:00
Thibaut Girka c37bc0b268 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/en.yml
  Conflict caused by the glitch-soc-specific “flavour” string being too close
  to the newly introduced “identity_proofs” string. Just included both.
2019-03-20 13:54:00 +01:00
ThibG e9fc1e8ccb Do not distribute Delete when rejecting unapproved accounts (#10321) 2019-03-19 16:33:30 +01:00
Thibaut Girka 365de06816 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/accounts_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/statuses_controller.rb

All conflicts caused by the additional `use_pack` used for glitch-soc's theming
system.
2019-03-18 18:03:27 +01:00
ThibG 9efb9d7f72 Set and store report URIs (#10303)
Fixes #10271
2019-03-17 15:34:56 +01:00
Thibaut Girka 3f8f1b01c0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/services/remove_status_service.rb
  Conflict because we still handle the direct timeline.
  Took upstream changes, with that one extra function call.
- config/locales/ca.yml
  Conflict because theme names were updated.
  Decided to *keep* the theme names even if they are useless
  to avoid future conflicts.
- config/locales/oc.yml
  Decided to *keep* the theme names even if they are useless
  to avoid future conflicts.
- config/locales/pl.yml
  Decided to *keep* the theme names even if they are useless
  to avoid future conflicts.
2019-03-16 20:59:33 +01:00
ThibG 289199a107 Avoid race condition when streaming deleted statuses (#10280)
* Avoid race condition when streaming deleted statuses

* Move redis lock to DistributionWorker to avoid extra Redis value
2019-03-16 20:18:47 +01:00
Thibaut Girka eed61a37ec Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/follower_domains_controller.rb
  Removed upstream. Did the same here. Maybe we should not have?
- config/locales/en.yml
  Upstream removed the “Authorized followers” page and associated
  translations. This is too close in the file to our glitch-soc-specific
  “flavour” string. No actual conflict.
- config/locales/ja.yml
  Same as above.
- config/locales/pl.yml
  Same as above.
- config/navigation.rb
  No real conflict. New route added too close to the glitch-soc-specific
  “flavours” one.
- config/webpack/configuration.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- config/webpack/loaders/babel.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
  The contents of this file have been moved to package.json.
- config/webpack/shared.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- config/webpacker.yml
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- jest.config.js
  The contents of this file have been moved to package.json.
- package.json
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- yarn.lock
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
2019-03-16 14:14:20 +01:00
Eugen Rochko 0b6bcb9384 Fix detailed poll validation errors not being returned in the API (#10261)
No more "Owned poll is invalid"
2019-03-15 13:36:38 +01:00
Eugen Rochko 1279d85ff4 Add `visibility` param to reblog REST API (#9851)
Use async worker for creating reblog notification to improve performance
2019-03-15 04:36:41 +01:00
Thibaut Girka 741766a01d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/admin/settings/edit.html.haml:
  Conflict due to how the theming settings change.
2019-03-14 17:17:20 +01:00
Eugen Rochko 73fb7bfa0f Admission-based registrations mode (#10250)
Fix #6856
Fix #6951
2019-03-14 05:28:30 +01:00
Thibaut Girka d86033c352 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/about_controller.rb
- app/controllers/tags_controller.rb
- app/views/about/show.html.haml
- spec/views/about/show.html.haml_spec.rb
2019-03-13 15:16:02 +01:00
Eugen Rochko d5377d0d45 Fix poll update handler calling method was that was not available (#10246)
* Fix poll update handler calling method was that was not available

Fix regression from #10209

* Refactor VoteService

* Refactor ActivityPub::DistributePollUpdateWorker and optimize it

* Fix typo

* Fix typo
2019-03-12 22:58:59 +01:00
Eugen Rochko 45d257a20f Optimize RemoveStatusService by removing n+1 query and using shared inboxes (#10247) 2019-03-12 22:34:55 +01:00
Thibaut Girka 33001891b6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/services/post_status_service.rb
  Small conflict due to handling of instance-local toots.
  A subsequent change is required to ensure instance-local polls are not leaked
  through Update.
2019-03-11 11:23:50 +01:00
ThibG 11ce444b8b Support pushing and receiving updates to poll tallies (#10209)
* Process incoming poll tallies update

* Send Update on poll vote

* Do not send Updates for a poll more often than once every 3 minutes

* Include voters in people to notify of results update

* Schedule closing poll worker on poll creation

* Add new notification type for ending polls

* Add front-end support for ended poll notifications

* Fix UpdatePollSerializer

* Fix Updates not being triggered by local votes

* Fix tests failure

* Fix web push notifications for closing polls

* Minor cleanup

* Notify voters of both remote and local polls when those close

* Fix delivery of poll updates to mentioned accounts and voters
2019-03-11 00:49:31 +01:00
Thibaut Girka 88d4b28714 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-10 16:48:44 +01:00
ThibG 05aa8b9303 Reject existing Follows when suspending a remote account (#10230)
* Reject existing Follows when suspending a remote account

Partial fix to #10229

* Add tests
2019-03-10 16:18:58 +01:00
Thibaut Girka 9b093dfcde Merge branch 'master' into glitch-soc/merge-upstream 2019-03-08 00:57:13 +01:00
ThibG dcbe6c16ba Do not allow adding votes to expired polls (#10214)
* Do not allow adding votes to expired polls

* Only validate expires_at on create
2019-03-08 00:54:50 +01:00
ThibG 641384d288 Handle StaleObjectError when retrieving polls (#10208) 2019-03-07 15:52:38 +01:00
Thibaut Girka c0b876f726 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- db/schema.rb

Both conflicts are caused by us having extra database columns.
2019-03-05 19:23:16 +01:00
ThibG 0c8e3beb51 Do not error out on unsalvageable errors in FetchRepliesService (#10175)
* Do not error out on unsalvageable errors in FetchRepliesService

Fixes #10152

* Fix FetchRepliesWorker erroring out on deleted statuses
2019-03-05 15:21:14 +01:00
ThibG fbe97b9643 Fix suspended account's fields being set as empty dict instead of list (#10178)
Fixes #10177
2019-03-05 15:19:54 +01:00
Eugen Rochko 31a1fa3dcf Fix last_fetched_at not being set on polls (#10170) 2019-03-05 04:10:01 +01:00
Eugen Rochko 1c6a9a6ea8 Fix another typo in ActivityPub::FetchRemotePollService (#10146) 2019-03-04 00:52:18 +01:00
Eugen Rochko 1bbbc31cc0 Fix typo in ActivityPub::FetchRemotePollService (#10145) 2019-03-04 00:50:56 +01:00
Eugen Rochko 952045604c Fix remote poll expiration time (#10144) 2019-03-04 00:44:34 +01:00
ThibG ad11260825 Correctly make polls and media mutually exclusive (#10141) 2019-03-04 00:40:21 +01:00
ThibG 7d0dc066a7 Make sure the poll is created before storing its id (#10142)
* Make sure the poll is created before storing its id

* Fix updating poll results

* Support fetching Question activities from the search bar
2019-03-04 00:39:06 +01:00
Eugen Rochko 4a3acdc916 Add polls (#10111)
* Add polls

Fix #1629

* Add tests

* Fixes

* Change API for creating polls

* Use name instead of content for votes

* Remove poll validation for remote polls

* Add polls to public pages

* When updating the poll, update options just in case they were changed

* Fix public pages showing both poll and other media
2019-03-03 22:18:23 +01:00
Thibaut Girka 91934eeb74 Merge branch 'master' into glitch-soc/merge-upstream 2019-02-28 21:35:53 +01:00
ThibG 9f3f47e91f Improved remote thread fetching (#10106)
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
2019-02-28 15:22:21 +01:00
Thibaut Girka 76949bc007 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Dockerfile
  Took upstream version and reverted b6d0dcc5d3.
  Hopefuly it's not needed anymore.
- app/controllers/api/v1/search_controller.rb
  The conflict was due to us raising the number of results returned.
  Upstream raised it further, so took it.
- config/locales/de.yml
  Took upstream changes to theme translation strings.
- config/locales/gl.yml
  Took upstream changes to theme translation strings.
- config/locales/nl.yml
  Took upstream changes to theme translation strings.
- config/locales/sk.yml
  Took upstream changes to theme translation strings.
2019-02-26 17:04:44 +01:00
Eugen Rochko 693f2353bc Add type, limit, offset, min_id, max_id, account_id to search API (#10091)
* Add type, limit, offset, min_id, max_id, account_id to search API

Fix #8939

* Make the offset work on accounts and hashtags search as well

* Assure brakeman we are not doing mass assignment here

* Do not allow paginating unless a type is chosen

* Fix search query and index id field on statuses instead of created_at
2019-02-26 15:21:36 +01:00
NOGISAKA Sadata ae346fd6d1 Skip some methods in BatchedRemoveStatusService when account is nil (#10095)
* Skip some methods in BatchedRemoveStatusService when account is nil

Sometimes `bin/tootctl accounts cull` fails with below error:

undefined method `followers_for_local_distribution' for nil:NilClass (NoMethodError)

This commit makes BatchedRemoveStatusService to skip below methods when
`account` is nil:

- unpush_from_home_timelines()
- unpush_from_list_timelines()
- batch_stream_entries()

* Fix rubocop error: Use `next` to skip iteration.
2019-02-24 14:28:36 +01:00
Thibaut Girka c8086b6efb Merge branch 'master' into glitch-soc/merge-upstream 2019-02-15 18:02:45 +01:00
Eugen Rochko ddbf75ea87 Filter incoming Create activities by relation to local activity (#10005)
Reject those from accounts with no local followers, from relays
that are not enabled, which do not address local accounts and are
not replies to accounts that do have local followers
2019-02-13 18:42:47 +01:00
Thibaut Girka de60ff3809 Merge branch 'master' into glitch-soc/merge-upstream 2019-02-12 20:32:48 +01:00
ThibG b27f715601 Move sending account Delete to anyone but the account's followers to the pull̀ queue (#10016) 2019-02-12 05:10:43 +01:00
Thibaut Girka 1c37ad108c Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/oauth/authorized_applications_controller.rb
  Two changes too close to each other
- app/controllers/settings/sessions_controller.rb
- app/lib/user_settings_decorator.rb
  Two changes too close to each other
- app/models/media_attachment.rb
  New changes too close to glitch-soc only changes.
- app/models/user.rb
  Two changes too close to each other.
- app/services/remove_status_service.rb
  Kept direct timeline code which had been removed upstream.
- app/views/settings/preferences/show.html.haml
  Two changes too close to each other.
- config/locales/en.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/ja.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/pl.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/simple_form.en.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/locales/simple_form.pl.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/settings.yml
  Reverted upstream's decision of enabling posting application by default.
2019-02-10 21:10:09 +01:00
abcang 4a8a8bcb8a Fix Tombstone.delete_all ArgumentError (#9978) 2019-02-06 02:50:52 +01:00
Eugen Rochko d2d4d38f22 Add featured hashtags to profiles (#9755)
* Add hashtag filter to profiles

GET /@:username/tagged/:hashtag
GET /api/v1/accounts/:id/statuses?tagged=:hashtag

* Display featured hashtags on public profile

* Use separate model for featured tags

* Update featured hashtag counters on-write

* Limit featured tags to 10
2019-02-04 04:25:59 +01:00
Eugen Rochko 1f303e5591 Add option to overwrite imported data (#9962)
* Add option to overwrite imported data

Fix #7465

* Add import for domain blocks
2019-02-03 03:59:51 +01:00
ysksn 9cad360caa Create Redisable#redis (#9633)
* Create Redisable

* Use #redis instead of Redis.current
2019-02-02 19:11:38 +01:00
Thibaut Girka 0ccf92ad09 Merge branch 'master' into glitch-soc/merge-upstream 2019-01-27 13:57:25 +01:00
Thibaut Girka 9bd95f11e5 Revert to storing application info, and display it to the author via API 2019-01-27 13:56:41 +01:00
Thibaut Girka bb035bb0ab Make storing and displaying application used to toot opt-in 2019-01-27 13:56:41 +01:00
ThibG b1c76ef610 Prevent posting toots with media attachments from someone else (#9921) 2019-01-26 23:59:39 +01:00
Thibaut Girka a9c3e77e81 Merge branch 'master' into glitch-soc/merge-upstream 2019-01-25 11:37:49 +01:00
Renato "Lond" Cerqueira cde2fe25b9 Rescue SSL error in verify link service (#9914)
If the first link to be verified contains a rel=me link with a SSL
error, the VerifyAccountLinksWorker will fail and not try the following
links. This rescues the SSL error when fetching the link, avoiding this
issue.
2019-01-24 15:38:18 +01:00
ThibG a238beb870 Fix scheduled toot with media immediately creating a toot (#9894)
* Add test for not persisting status when attaching media to scheduled toot

* Prevent status used for validation from being persisted to the database

Fixes #9893

Thanks to tateisu for the help investigating this.
2019-01-21 20:03:04 +01:00
Thibaut Girka 827933cd09 Merge branch 'master' into glitch-soc/merge-upstream 2019-01-20 16:49:23 +01:00
Eugen Rochko 17a8c10a03 Fix error when `scheduled_at` attribute is malformed (#9866) 2019-01-19 22:01:01 +01:00
Thibaut Girka b49d415f37 Merge branch 'master' into glitch-soc/merge-upstream
No conflicts.
2019-01-19 18:28:37 +01:00
ThibG 8de5d28891 Reject existing Follow in addition to sending a Block (#9811)
Mastodon expects remote servers to remove follow relationships upon receiving
a Block. However, the spec only evokes Block activities in a C2S context, never
in a S2S context.

This PR, in addition to federating the Block, explicitly sends a Reject for any
affected follow relationship, which makes a bit more sense with regards to the
spec.
2019-01-18 15:57:19 +01:00
ThibG d0008b5051 Add tombstones for remote statuses (#9830)
* Add Tombstone model to remember object deletion

* Do not recreate a status if it has been deleted

* Record Tombstone for remote deleted items

Also, only record deleted items from same-host actors

* Clear an user's tombstones when their key change
2019-01-18 15:56:55 +01:00
Eugen Rochko 9a98e3fe76 Fix code style of regeneration-related code (#9843) 2019-01-18 01:02:51 +01:00
Thibaut Girka a06d3a6082 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/simple_form.pl.yml
2019-01-16 14:20:07 +01:00
Renato "Lond" Cerqueira ed55b4f8fb Fix undefined method error in sidekiq (#9807)
* Fix undefined method error in sidekiq

Body can be not nil but still be empty, which causes a
`NoMethodError: undefined method `[]' for nil:NilClass` further in the
code. This checks for an empty body to avoid the issue.

* Fix codeclimate issue
2019-01-14 17:28:41 +01:00
Thibaut Girka a78447a6a4 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .eslintrc.yml
  Removed, as upstream removed it.
- app/controllers/admin/statuses_controller.rb
  Minor code cleanup when porting one of our features.
- app/models/account.rb
  Note length validation has changed upstream.
  We now use upstream's validation (dropped legacy glitch-soc
  account metadata stuff) but with configurable limit.
- app/services/post_status_service.rb
  Upstream has added support for scheduled toots, refactoring
  the code a bit. Adapted our changes to this refactoring.
- app/views/stream_entries/_detailed_status.html.haml
  Not a real conflict, changes too close.
- app/views/stream_entries/_simple_status.html.haml
  Not a real conflict, changes too close.
2019-01-10 21:00:30 +01:00
Thibaut Girka 8d47495251 Revert "Revert "Add handler for Move activity (#9629)""
This reverts commit 6ab727bf6a.
2019-01-10 18:46:17 +01:00
ThibG e60999c3c5 Improvements to signature verification (#9667)
* Refactor signature verification a bit

* Rescue signature verification if recorded public key is invalid

Fixes #8822

* Always re-fetch AP signing key when HTTP Signature verification fails

But when the account is not marked as stale, avoid fetching collections and
media, and avoid webfinger round-trip.

* Apply stoplight to key/account update as well as initial key retrieval
2019-01-07 21:45:13 +01:00
Eugen Rochko d62a2a69e4 Add locale param to sign-up API (#9747)
Fix #9627
2019-01-07 14:50:20 +01:00
Eugen Rochko bf1ee883d8 Fix wrong param name in scheduled statuses and return params in API (#9725)
The database column and API param are called in_reply_to_id, not
in_reply_to_status_id, so it makes no sense to encode it that way
2019-01-06 12:03:27 +01:00
Eugen Rochko ae1aaa3b8a Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
Naoki Kosaka ef6de324c4 Use Contact User as Relay, Report, Subscribe. (#9661)
* Use Contact User as Relay, Report, Subscribe.

* Use Account.representative to fetch contact user.

* Use find_local.

* No reason to use Account.representative in subscribe_service.

* Don't required representative!

* Fallback is included in Account.representative method.
2019-01-05 07:17:12 +01:00
Thibaut Girka 6ab727bf6a Revert "Add handler for Move activity (#9629)"
This reverts commit 4e3131f02f.
2019-01-02 15:36:59 +01:00
Thibaut Girka 44f2224606 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts manually resolved:
- app/services/post_status_service.rb
- config/locales/simple_form.pl.yml
- config/routes.rb
- config/webpack/loaders/sass.js
- config/webpack/shared.js
- package.json
- yarn.lock
2019-01-02 15:36:53 +01:00
kodai 9648c59d6f Fix links not being verified (#9673)
Fix #9365
2018-12-31 15:29:10 +01:00
Jeong Arm c176b4ec80 Skip full text search if failed (#9654)
* Skip full text search if failed

* Move rescue block to "perform_statuses_search!"
2018-12-30 19:00:29 +01:00
ThibG 0244534700 Avoid duplicate work by merging ReplyDistributionWorker into DistributionWorker (#9660) 2018-12-30 19:00:04 +01:00
ThibG bdc44c3558 Reduce usage of LD signatures (#9659)
* Do not LDS-sign Follow, Accept, Reject, Undo, Block

* Do not use LDS for Create activities of private toots

* Minor cleanup

* Ignore unsigned activities instead of misattributing them

* Use status.distributable? instead of querying visibility directly
2018-12-30 09:48:59 +01:00
Eugen Rochko 4e3131f02f Add handler for Move activity (#9629) 2018-12-29 02:24:36 +01:00
Eugen Rochko 8958e58bd4 Improve admin UI for account view (#9643) 2018-12-28 03:38:41 +01:00
Eugen Rochko bbf9f4f93b Add REST API for creating an account (#9572)
* Add REST API for creating an account

The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.

The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.

The method is rate-limited by IP to 5 requests per 30 minutes.

* Redirect users back to app from confirmation if they were created with an app

* Add tests

* Return 403 on the method if registrations are not open

* Require agreement param to be true in the API when creating an account
2018-12-24 19:12:38 +01:00
ThibG a2492a89fb Limit maximum visibility of local silenced users to unlisted (#9583)
Fixes #9580
2018-12-24 19:06:14 +01:00
Thibaut Girka 720cb01715 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/base_controller.rb
  Some refactoring made upstream, no real conflict.
- app/javascript/mastodon/features/compose/components/compose_form.js
  Updated using upstream's code but using maxChars instead of the
  hardcoded length of 500 characters per toot.
- app/javascript/styles/mastodon/components.scss
  Upstream redesigned the onboarding modal. Not sure why we had a
  conflict there.
2018-12-18 16:55:15 +01:00
ThibG f76665a276 Ignore low-confidence CharlockHolmes guesses when parsing link cards (#9510)
* Add failing test for windows-1251 link cards

* Ignore low-confidence CharlockHolmes guesses

Fixes #9466

* Fix no method error when charlock holmes cannot detect charset
2018-12-17 19:19:45 +01:00
Thibaut Girka 04204427d0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/directories_controller.rb
- app/controllers/settings/applications_controller.rb
- app/controllers/settings/base_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/exports_controller.rb
- app/controllers/settings/follower_domains_controller.rb
- app/controllers/settings/imports_controller.rb
- app/controllers/settings/migrations_controller.rb
- app/controllers/settings/notifications_controller.rb
- app/controllers/settings/preferences_controller.rb
- app/controllers/settings/sessions_controller.rb
- app/controllers/settings/two_factor_authentication/confirmations_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
- app/controllers/settings/two_factor_authentications_controller.rb

Conflicts were due to some refactoring already made in glitch-soc
when introducing flavours.
2018-12-15 10:45:53 +01:00
Sumit Khanna b1b6e5911d Error message for avatar image that's too large. #9204 (#9518)
* Error message for avatar image that's too large. #9204

* Code climate/formatting

* Removed avatar error message

* Moved valid image dimentions check to update service

* removed unnescessary begin block

* code climate formatting

* code climate indent fix
2018-12-14 05:07:21 +01:00
ysksn 316bc93abd Remove redundant (#9503) 2018-12-12 05:54:34 +01:00
ysksn 201a9e844a Use find_or_initialize_by (#9504) 2018-12-12 05:54:09 +01:00
Thibaut Girka b62953b3d2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/javascript/packs/public.js
- app/models/user.rb
- config/settings.yml
- db/schema.rb

Moved public.js changes to settings.js.
2018-12-09 16:08:04 +01:00
Eugen Rochko fd8ca11f19 Add profile directory (#9427)
Fix #5578
2018-12-06 17:36:11 +01:00
Thibaut Girka 4aca9b4e0f Merge branch 'master' into glitch-soc/merge-upstream 2018-12-06 12:44:38 +01:00
Eugen Rochko 626a544d08 Improve account suspension speed and completeness (#9290)
- Some associations were missing from the clean-up
- Some attributes were not reset on suspension
- Skip federation and streaming deletes when purging a dead domain
- Move account association definitions to concern
2018-12-03 01:32:08 +01:00
Thibaut Girka 478abe3a58 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb

Resolved by taking both changes (not a real conflict, just changes too close
to each other).
2018-11-27 13:23:02 +01:00
valerauko a6ebcda59b Ignore JSON-LD profile in mime type comparison (#9179)
Ignore JSON-LD profile in mime type comparison
2018-11-22 12:49:07 +01:00
James Kiesel b874d9dac7 Simplify hashtag query service, perf increase for 'any' tags (#9298) 2018-11-19 12:49:50 +01:00
Eugen Rochko 7a939f7cfc Remove intermediary arrays when creating hash maps from results (#9291) 2018-11-16 15:02:18 +01:00
Thibaut Girka b3553f992d Merge branch 'master' into glitch-soc/merge-upstream 2018-11-13 16:28:07 +01:00
ThibG b6e377a340 Fix emoji update date processing (#9255) 2018-11-10 23:59:51 +01:00
ThibG 49889c0b8b Check that twitter:player is valid before using it (#9254)
Fixes #9251
2018-11-10 20:42:04 +01:00
Thibaut Girka bf26c23205 Merge branch 'master' into glitch-soc/merge-upstream 2018-11-09 14:56:31 +01:00
Eugen Rochko 5f613d55e6 Optimize the process of following someone (#9220)
* Eliminate extra accounts select query from FollowService

* Optimistically update follow state in web UI and hide loading bar

Fix #6205

* Asynchronize NotifyService in FollowService

And fix failing test

* Skip Webfinger resolve routine when called from FollowService if possible

If an account is ActivityPub, then webfinger re-resolving is not necessary
when called from FollowService. Improve options of ResolveAccountService
2018-11-08 21:05:42 +01:00
Thibaut Girka e18018e930 Merge branch 'master' into glitch-soc/merge-upstream 2018-11-06 15:22:44 +01:00
m.b 1093295902 Update resolve_url_service.rb (#9188) 2018-11-05 18:54:07 +01:00
James Kiesel d2a9ea58da Allow joining several hashtags in a single column (#8904)
* Nascent tag menu on frontend

* Hook up frontend to search

* Tag intersection backend first pass

* Update yarnlock

* WIP

* Fix for tags not searching correctly

* Make radio buttons function

* Simplify radio buttons with modeOption

* Better naming

* Rearrange options

* Add all/any/none functionality on backend

* Small PR cleanup

* Move to service from scope

* Small cleanup, add proper service tests

* Don't use send with user input :D

* Set appropriate column header

* Handle auto updating timeline

* Fix up toggle function

* Use tag value correctly

* A bit more correct to use 'self' rather than 'all' in status scope

* Fix some style issues

* Fix more code style issues

* Style select dropdown more better

* Only use to_id'ed value to ensure no SQL injection

* Revamp frontend to allow for multiple selects

* Update backend / col header to account for more flexible tagging

* Update brakeman ignore

* Codeclimate suggestions

* Fix presenter tag_url

* Implement initial PR feedback

* Handle additional tag streaming

* CodeClimate tweak
2018-11-05 18:53:25 +01:00
Thibaut Girka 33ad16faa7 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-31 10:08:36 +01:00
Eugen Rochko 3981da6f90 Revert "Fix FetchAtomService content type handling (#9132)" (#9171)
This reverts commit c20d55f7dd.
2018-10-31 00:43:34 +01:00
Thibaut Girka fd18ed4cea Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb

Upstream reverted something we partially reverted already.
Reverted the rest to match upstream.
2018-10-30 17:52:08 +01:00
valerauko c20d55f7dd Fix FetchAtomService content type handling (#9132)
* Add profile to json+ld in Accept

It's required by the ActivityPub spec

* Use headers['Content-type'] instead of mime_type

mime_type strips the profile from the content type, but it's still available raw in the headers hash

* Add test for ld+json with profile
2018-10-30 15:07:57 +01:00
Eugen Rochko d84886f35e Accept the same payload in multiple inboxes and deliver (#9150) 2018-10-30 15:03:55 +01:00
Eugen Rochko c15b8ee75b Always let through notifications from staff (#9152)
* Always let through notifications from staff

Follow-up to #8993

* Let messages from staff through, but no other notifications
2018-10-30 15:02:55 +01:00
ThibG cfe92b50bb Fix Pleroma mentions being fetched as preview cards (#9158) 2018-10-30 15:02:24 +01:00
Thibaut Girka 7406141a71 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-30 11:22:25 +01:00
ThibG 1cbcb09d67 Do not hide boost notifications from followed people with hidden boosts (#9147)
* Do not hide boost notifications from followed people with hidden boosts

Not displaying boosts from a followed user in the Home timeline and not
having notifications when they reblog your own content are two very
separate concerns, tying them together seem counter-intuitive and unwanted.

* Update specs accordingly
2018-10-30 00:47:31 +01:00
Thibaut Girka c954f89bdd Merge branch 'master' into glitch-soc/merge-upstream 2018-10-28 08:37:49 +01:00
Eugen Rochko cf2ab9c394 Include preview cards in status entity in REST API (#9120)
* Include preview cards in status entity in REST API

* Display preview card in-stream

* Improve in-stream display of preview cards
2018-10-28 06:35:03 +01:00
Thibaut Girka 65cc5faf80 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-27 18:54:26 +02:00
Eugen Rochko 7c01fbf4fd Add locality check to ActivityPub::FetchRemoteAccountService (#9109)
* Add locality check to ActivityPub::FetchRemoteAccountService

Fix #8643

Because there are a few places where it is called, it is difficult
to confirm if they all previously checked it for locality. It's better
to make sure within the service.

* Remove faux-remote duplicates of local accounts
2018-10-26 23:08:34 +02:00
Thibaut Girka c32a5f86b6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/base_controller.rb
- app/controllers/filters_controller.rb
- app/controllers/invites_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/exports_controller.rb
- app/controllers/settings/follower_domains_controller.rb
- app/controllers/settings/migrations_controller.rb
- app/controllers/settings/notifications_controller.rb
- app/controllers/settings/preferences_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
- app/javascript/packs/public.js
- app/views/settings/profiles/show.html.haml

Conflicts were mostly due to the addition of body classes to the settings page,
this was caused by rejecting upstream changes for most of those files and
modifying Settings::BaseController instead.

Another cause of conflicts was the deletion of client-side checking of
display name / bio length, this was modified in app/javascript/core/settings.js
instead.
2018-10-26 20:41:43 +02:00
abcang 95ef0e144a Skip link-back check if body is nil (#9107) 2018-10-26 03:31:23 +02:00