Commit Graph

194 Commits (ccb6a8e2d2cbfdd478696409267a6237a567f025)

Author SHA1 Message Date
Akihiko Odaki f9dd41621f Do not filter statuses with unknown languages (#5045) 2017-09-22 19:33:17 +02:00
Eugen Rochko eb345f8e8b Custom emoji (#4988)
* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted
2017-09-19 02:42:40 +02:00
unarist 3978894d05 Validate uri presence for remote status (#4985) 2017-09-17 15:21:57 +02:00
Akihiko Odaki 8c96e0d496 Remove local_only scope in Status (#4977) 2017-09-17 05:29:43 +02:00
Eugen Rochko 8bb036350a Fix #4834 - Adjust Status#local and Status#remote scopes (#4839) 2017-09-07 20:18:34 +02:00
Eugen Rochko 5ad3153b0b Fix locking migration on statuses table. Nullable column and NO default value (#4825) 2017-09-06 20:57:52 +02:00
Eugen Rochko 73c613dcfc Switch to static URIs, new URI format in both protocols for new statuses (#4815)
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
Eugen Rochko e9e271878e Make PreviewCard records reuseable between statuses (#4642)
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
Eugen Rochko 3135d20283 Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730)
Requires moving Atom rendering from DistributionWorker (where
`stream_entry.status` is already nil) to inline (where
`stream_entry.status.destroyed?` is true) and distributing that.

Unfortunately, such XML renderings can no longer be easily chained
together into one payload of n items.
2017-08-29 16:11:05 +02:00
Eugen Rochko 3e20cb7025 Pinned statuses (#4675)
* Pinned statuses

* yarn manage:translations
2017-08-25 01:41:18 +02:00
Eugen Rochko 5138dde794 Fix #4149, fix #1199 - Store emojis as unicode (#4189)
- Use unicode when selecting emoji through picker
- Convert shortcodes to unicode when storing text input server-side
- Do not convert shortcodes in JS anymore
2017-07-14 19:47:53 +02:00
Eugen Rochko 4e7288d18a Fix boolean columns sometimes having a null value (#4162)
* Fix boolean columns sometimes having a null value

* Fix wrong value being set instead of null
2017-07-13 03:12:25 +02:00
Eugen Rochko 348de9dbc3 Fix #1010 - When spoiler text is set, enforce sensitivity too (#4176) 2017-07-13 01:47:23 +02:00
Eugen Rochko e7810e4031 Fix regression from #3842 (#3892)
* Fix regression from #3842

Simplify the query by omitting all direct statuses. Private statuses
are allowed because they are from accounts we are following (so
by definition)

Resolves #3887 (alternative)

* Adjust test
2017-06-22 02:38:50 +02:00
Akihiko Odaki (@fn_aki@pawoo.net) 77dcf442e7 Filter direct statuses in Status.as_home_timeline (#3842)
The classes using Status.as_home_timeline, namely Feed and
PrecomputeFeedService are expected to filter direct statuses as
FanOutWriteService does, but their filtering were incomplete or missing.

This commit solves the problem by filtering direct statuses in
as_home_timeline as the other similar methods such as as_public_timeline
does.
2017-06-20 20:41:23 +02:00
Matt Jankowski b439018342 Language detection defaults to nil (#3666)
* Default to nil for statuses.language

* Language detection defaults to nil instead of instance UI default
2017-06-09 18:09:37 +02:00
Matt Jankowski d339ac2c76 Introduce StatusThreadingConcern (#3490)
* Add a StatusFilter class to identify visibility of statuses by accounts

* Extract StatusThreadingConcern from Status

* Clarify purpose of checking for nil account
2017-06-05 16:07:44 +02:00
Matt Jankowski 533a8cc68b Misc tidying and clean ups (#3445)
* Remove trailing whitespace in i18n mailers

* Use query methods instead of #present? on AR attributes

* Delegate Status#account_domain method

* Delegate Mention #account_username and #account_acct methods
2017-05-31 20:38:17 +02:00
Jack Jennings faf53a5a3e Extract authorization policy for viewing statuses (#3150) 2017-05-29 18:22:22 +02:00
Akihiko Odaki 9cc7a3c209 Cover AccountsController more in spec (#3229)
* Introduce recent scope to Status and StreamEntry

Introduce recent scope to Status and StreamEntry as Account has.

* Cover AccountsController more in AccountsController
2017-05-23 02:53:01 +02:00
Matt Jankowski 1122249e51 Filter languages with opt out (#3175)
* Remove allowed_languages and add filtered_languages

* Use filtered_languages instead of allowed_languages
2017-05-20 17:32:44 +02:00
beatrix 4f3b6cc3be guard against empty domain block list in status scope (#3161) 2017-05-20 13:38:13 +02:00
Eugen Rochko de5e67238b Fix federated timeline excluding local toots when any domain blocks are set (#3151) 2017-05-19 21:19:41 +02:00
Eugen Rochko 5695449335 Add buttons to block and unblock domain (#3127)
* Add buttons to block and unblock domain

* Relationship API now returns "domain_blocking" status for accounts,
rename "block entire domain" to "hide entire domain", fix unblocking domain,
do not block notifications from domain-blocked-but-followed people, do
not send Salmons to domain blocked users

* Add test

* Personal domain blocks shouldn't affect Salmon after all, since in this
direction of communication the control is very thin when it comes to
public stuff. Best stay consistent and not affect federation in this way

* Ignore followers and follow request from domain blocked folks,
ensure account domain blocks are not created for empty domain,
and avoid duplicates in validation

* Purge followers when blocking domain (without soft-blocks, since they
are useless here)

* Add tests, fix local timeline being empty when having any domain blocks
2017-05-19 21:05:32 +02:00
Matt Jankowski 6619cfe934 Conditional validations no longer accept strings for if/unless (#3124) 2017-05-19 03:11:23 +02:00
Eugen Rochko 0cafe62561 Account domain blocks (#2381)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation

* Adding account domain blocks that filter notifications and public timelines

* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home

* Add API for listing and creating account domain blocks

* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them

* Filter domains in streaming API

* Update account_domain_block_spec.rb
2017-05-19 01:14:30 +02:00
alpaca-tc 5523fd6efb Refactor Status#ancestors/descendants (#3092) 2017-05-17 15:38:16 +02:00
alpaca-tc 0d7dc192c3 Optimize Status#permitted_for 24x (#3069)
* Build query with arel node

* Add spec for current Status#permitted_for implementation

* Refactor status.rb

* Order by visibility to optimize query
2017-05-16 02:54:17 +02:00
Eugen Rochko bd7dc50186 Fix change of status callbacks not setting in_reply_to_account_id and (#3072)
possibly others when expected. Add some tests for it
2017-05-15 21:20:55 +02:00
Eugen Rochko 5039bc93d5 Feature conversations muting (#3017)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation
2017-05-15 03:04:13 +02:00
alpaca-tc 4b5edb0d16 Fixes NoMethodError: undefined method 'first' for nil:NilClass (#3036) 2017-05-13 15:56:12 +02:00
Eugen Rochko 12ef99556e Add conversation model, <ostatus:conversation /> (#3016)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Fix conversation migration

* More spec coverage for status before_create

* Prevent n+1 query when generating Atom with the new conversations

* Improve code style

* Remove redundant local variable
2017-05-12 19:09:21 +02:00
yhirano e5eb070430 annotate models (#2697)
* add annotate to Gemfile

* rails g annotate:install

* configure annotate_models

* add schema info to models

* fix rubocop to add frozen_string_literal
2017-05-02 02:14:47 +02:00
Matt Jankowski c44ef23e50 Filter on allowed user language preferences (#2361)
* Naive approached to timeline filtering

* Convert allowed_languages into a db column

* Allow users to choose languages to see statuses in

* Style list items as two columns

* Add a hint to explain language filtering preference
2017-05-01 17:42:13 +02:00
Matt Jankowski 3e819257ab More status specs (#2564)
* Add rough outline of coverage needed for public timeline

* Specs for visibility, replies, boosts

* Specs for silenced account

* Specs for local_only option

* Specs for blocks and mutes

* Add tentative spec around including other silenced account statuses

* Add with_public_visibility scope

* Add simple coverage for tag_timeline

* Tag timeline includes replies

* Replace tag.statuses with a tagged_with scope in tag timeline method

* Use with_public_visibility in tag timeline

* Extract common scope between public and tag timelines to method

* Extract local domain check to local_only scope

* Extract local_only check to starting scope method

* Move list of excluded from timeline account ids to account model

* Simplify excluded accounts list on account model

* Only join accounts when needed

* Rename method for account specific filtering

* Extract method for account exclusions

* Fix bug where silenced accounts were not including statuses from other silenced accounts

* DRY up filter application from account or no account

* timeline_scope can be private

* Add spec showing that account can find its excluded accounts ids

* Add spec which fails if local_only does not have a left outer join

* rubocop
2017-04-28 15:10:41 +02:00
Eugen Rochko 886aa14cfc Cache Status#ancestors recursive query results (#2527) 2017-04-27 17:07:15 +02:00
Tomohiro Suwa 7bb45d9ed5 Don't need to include ActiveModel::Validations (#2521) 2017-04-27 12:02:55 +02:00
Evan Minto 227f45402d ActivityPub: Add basic, read-only support for Outboxes, Notes, and Create/Announce Activities (#2197)
* Clean up collapsible components

* Expose user Outboxes and AS2 representations of statuses

* Save work thus far.

* Fix bad merge.

* Save my work

* Clean up pagination.

* First test working.

* Add tests.

* Add Forbidden error template.

* Revert yarn.lock changes.

* Fix code style deviations and use localized instead of hardcoded English text.
2017-04-23 05:21:10 +02:00
alpaca-tc e40ea6cd9d Extract error messages to locale file (#2162) 2017-04-19 22:19:32 +02:00
839 95e8ab21a1 Avoid dynamic methods due to processing speed (#2080) 2017-04-18 15:57:46 +02:00
Eugen Rochko 8867792e53 Rename cache key to exclude_account_ids_for:id, adjust formatting 2017-04-18 01:14:02 +02:00
Eugen Rochko 8af5ceb661 Merge branch 'fix/cache_blocking' of git://github.com/pixiv/mastodon into pixiv-fix/cache_blocking 2017-04-18 01:09:57 +02:00
alpaca-tc 4f0d96aa5e Required foreign keys (#2003)
* Add `required: true` option to foreign column

* Fixes NoMethodError

```
> Favourite.new.valid?
NoMethodError: undefined method `reblog?' for nil:NilClass
```
2017-04-17 15:54:33 +02:00
Effy Elden d52abe1cc7 Merge branch 'master' into fix/cache_blocking 2017-04-17 01:41:33 +10:00
Matt Jankowski a3a4fe2163 Simplify the og:image and og:description code in stream_entries/show (#1934) 2017-04-16 16:38:02 +02:00
Keiji Yoshimi 6ca3a6c9f4 cache blockings for reducing queries. 2017-04-16 12:20:38 +09:00
Eugen d30a82a22f Fix #1339 - better Atom titles (#1343) 2017-04-09 20:55:54 +02:00
Joël Quenneville 8a67fb5eca DRY up reblog vs original status check
Checking reblog vs original status was happening in multiple places
across the app. For views, this logic was encapsulated in a helper
method named `proper_status` but in the other layers of the app, the
logic was duplicated.

Because the logic is used at all layers of the app, we extracted it into
a `Status#proper` method on the model and changed all uses of the logic
to use this method. There is now a single source of truth for this
condition.

We added test coverage to untested methods that got refactored.
2017-04-07 14:18:30 -04:00
Brad Urani 522410589e Eliminate unnecessary queries and query clauses with none and all 2017-04-05 06:02:58 +00:00
Eugen Rochko 24793cdbaa Fix ActionController::Parameters in API issue 2017-04-04 01:33:34 +02:00
Eugen Rochko 96e58cf289 Add counter caches for a large performance increase on API requests 2017-03-30 15:06:59 +02:00
Eugen Rochko f1b9644cfd Add "direct" visibility level in the backend. Web UI is not yet
adjusted to allow choosing it, yet
2017-03-15 22:55:45 +01:00
Eugen Rochko d1e1f26878 Improved /api/v1/accounts/:id/statuses with new params: only_media, exclude_replies
Redirect /:username to /users/:username
Redirect /:username/:id to /users/:username/updates/:id
Updated API documentation and sponsors
2017-03-05 17:27:17 +01:00
Eugen Rochko 9989890e2e Fix #231 - Muting 2017-03-02 18:49:32 +01:00
Kit Redgrave 4554ccd5d0 Mute button progress so far. WIP, doesn't entirely work correctly. 2017-03-01 22:31:21 -06:00
Eugen Rochko c7427bf3c9 Split public timeline into "public timeline" which is local, and
"whole known network" which is what public timeline used to be

Only domain blocks with suspend severity will block PuSH subscriptions
Silenced accounts should not appear in conversations unless followed
2017-02-19 20:25:54 +01:00
Eugen Rochko 6917e53fe0 Adding index overview for reports in admin UI 2017-02-16 02:28:10 +01:00
Eugen Rochko c001ef3df6 Fix Status.permitted_for scope query 2017-02-11 19:16:10 +01:00
Eugen Rochko ce193afa4f Mentions in private statuses allow mentioned people to see them 2017-02-11 15:10:22 +01:00
Eugen Rochko f0d4a2c9b7 Fix the fix 2017-02-09 21:22:49 +01:00
Eugen Rochko 81be690f95 Fix #614 - extra reply-boolean on statuses to account for cases when replied-to
status is not in the system at time of distribution; fix #607 - reset privacy
settings to defaults when cancelling replies
2017-02-09 20:25:39 +01:00
Eugen Rochko 94cda4cfb9 Add "clear notifications" button, exclude posts from people who have blocked *you* from public/hashtag timelines 2017-02-07 00:06:40 +01:00
Eugen Rochko b1f2683ecc Add API modifiers to limit returned toots from public/hashtag timelines
to only those from local users; Add link to "extended information" to
getting started in the UI; Add defaults for posting privacy; Change
how publish button looks depending on posting privacy chosen
2017-02-06 23:16:20 +01:00
Eugen Rochko 2f126b1225 Removing failed push notification API, make context loads use cache 2017-02-05 17:51:44 +01:00
Eugen Rochko 727d236fcc Cleaning up format of broadcast real-time messages, removing
redis-backed "mentions" timeline as redundant (given notifications)
2017-02-02 00:03:31 +01:00
Eugen Rochko edcfbee9f5 Fix #529 - Make hashtag timelines show conversations, fix hashtag loading in the UI 2017-01-31 22:35:38 +01:00
Eugen Rochko 959e064186 Instead of using spoiler boolean and spoiler_text, simply check for non-blank spoiler_text
Federate spoiler_text using warning attribute on <content /> instead of a <category term="spoiler" />
Clean up schema file from accidental development migrations
2017-01-25 01:29:16 +01:00
blackle e25fc71c2c Implement a click-to-view spoiler system 2017-01-23 21:07:40 -05:00
Eugen Rochko f748a91ec7 Fix #463 - Fetch and display previews of URLs using OpenGraph tags 2017-01-20 01:00:14 +01:00
Eugen Rochko 5d88ef90c9 Fix tests, add applications to eager loading/cache for statuses, fix
application website validation, don't link to app website if website isn't set,
also comment out animated boost icon from #464 until it's consistent with non-animated version
2017-01-15 14:01:33 +01:00
Effy Elden 8e0c1914fb Add tracking of OAuth app that posted a status, extend OAuth apps to have optional website field, add application details to API, show application name and website on detailed status views. Resolves #11 2017-01-15 08:58:50 +11:00
Eugen Rochko d7bc0d8a33 Set in_reply_to_account on statuses to non-self value when possible, thus
resolving the confusion from self-chain replies ultimately linking to a
non-self status. Adjust filters
2016-12-31 14:36:25 +01:00
Eugen Rochko 1ed3bb3a02 Don't show statuses to blocked users 2016-12-26 19:13:56 +01:00
Eugen Rochko 9e0bd72434 Fix account timelines bug 2016-12-23 01:13:16 +01:00
Eugen Rochko 238233440f Follow call on locked account creates follow request instead
Reflect "requested" relationship in API and UI
Reflect inability of private posts to be reblogged in the UI
Disable Webfinger for locked accounts
2016-12-22 23:03:57 +01:00
Eugen Rochko 11e2159859 Merge branch 'master' into development 2016-12-22 11:35:00 +01:00
Eugen Rochko 8038c663b0 Ensure that reblogs and favs always refer to the original status rather than a reblog wrapper 2016-12-22 11:34:05 +01:00
Eugen Rochko d417da7d3a Private visibility on statuses prevents non-followers from seeing those
Filters out hidden stream entries from Atom feed
Blocks now generate hidden stream entries, can be used to federate blocks
Private statuses cannot be reblogged (generates generic 422 error for now)
POST /api/v1/statuses now takes visibility=(public|unlisted|private) param instead of unlisted boolean
Statuses JSON now contains visibility=(public|unlisted|private) field
2016-12-21 20:04:13 +01:00
Eugen Rochko a4d3947c88 Fix 2016-12-06 00:24:33 +01:00
Eugen Rochko ca0757a6cf Add account suspension 2016-12-05 22:59:30 +01:00
Eugen Rochko 047bdf6fe0 Improve notification model 2016-12-03 20:04:19 +01:00
Eugen Rochko 2d4ce8a867 Fix #248 - Reload all accounts when fetching from cache 2016-12-03 18:21:26 +01:00
Eugen Rochko 23a4f93345 Fix #323 - self-replies to appear in public timelines again 2016-12-02 14:33:20 +01:00
Eugen Rochko 211049ae59 Fix #329 - avatar errors no longer prevent remote accounts from being saved
(without avatar). Also improved search position of exact matches
2016-12-02 14:14:49 +01:00
Eugen Rochko 8d4ef0b6c3 Per-status control for unlisted mode, also federation for unlisted mode
Fix #233, fix #268
2016-11-30 21:34:59 +01:00
Eugen Rochko 1bb1ec3b8d Further abstract caching for includes 2016-11-30 15:57:56 +01:00
Eugen Rochko 31108d4659 Public and hashtag timelines now exclude reblogs and replies
Fix #289 - don't download avatar unless the URL is http/https
Fix #293 - reblog/reblogged is now boost/boosted
2016-11-26 15:45:35 +01:00
Eugen Rochko dda6354c76 Implement includes caching for timelines APIs 2016-11-23 08:34:35 +01:00
Eugen Rochko f6a975af8b More query optimizations 2016-11-22 23:18:54 +01:00
Eugen Rochko 30010a6dbd Moving some counter queries out of subqueries in the API 2016-11-22 22:59:54 +01:00
Eugen Rochko 83cdfefa7d Remove orphaned notifications, add scopes param to app create API 2016-11-21 14:59:13 +01:00
Eugen Rochko 8d767b477f Validate presence of text even for remote statuses 2016-11-18 23:14:41 +01:00
Eugen Rochko c6f5eb8aa7 Fix #144 - Filter statuses from blocked users out of ancestors/descendants results 2016-11-15 17:33:41 +01:00
Eugen Rochko e71b152d89 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko d8c2044e65 Fix public timeline query 2016-11-10 21:58:22 +01:00
Eugen Rochko 0b03bbad67 Discard misattributed remote statuses, improve timelines filter 2016-11-10 00:15:49 +01:00
Eugen Rochko 6d9f8ee11e Improve filtering of public/hashtag timelines, both in backlog and real-time 2016-11-10 00:03:33 +01:00
Eugen Rochko beb36e24fe API pagination for all collections using Link header 2016-11-09 17:48:44 +01:00
Eugen Rochko d98b43cf56 Move timelines API from statuses to its own controller, add a check for
resources that require a user context vs those that don't (such as public timeline)

/api/v1/statuses/public   -> /api/v1/timelines/public
/api/v1/statuses/home     -> /api/v1/timelines/home
/api/v1/statuses/mentions -> /api/v1/timelines/mentions
/api/v1/statuses/tag/:tag -> /api/v1/timelines/tag/:tag
2016-11-08 23:29:08 +01:00
Eugen Rochko cb22dce970 Adding hashtags 2016-11-05 17:13:14 +01:00