Commit Graph

274 Commits (27545edf3fd752d64174037f463a03fb8edbf86e)

Author SHA1 Message Date
Eugen Rochko f70b20a01c Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
2019-07-13 16:45:50 +02:00
Eugen Rochko fbbcbd940d Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247) 2019-07-07 16:16:51 +02:00
Eugen Rochko 4931208dd8 Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Thibaut Girka 9866c2bfff Merge branch 'master' into glitch-soc/merge-upstream 2019-07-04 16:21:39 +02:00
Eugen Rochko e782fcef4f Add request pool to improve delivery performance (#10353)
* Add request pool to improve delivery performance

Fix #7909

* Ensure connection is closed when exception interrupts execution

* Remove Timeout#timeout from socket connection

* Fix infinite retrial loop on HTTP::ConnectionError

* Close sockets on failure, reduce idle time to 90 seconds

* Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server

* Use a shared pool size, 512 by default, to stay below open file limit

* Add some tests

* Add more tests

* Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds

* Use a shared pool that returns preferred connection but re-purposes other ones when needed

* Fix wrong connection being returned on subsequent calls within the same thread

* Reduce mutex calls on flushes from 2 to 1 and add test for reaping
2019-07-02 00:34:38 +02:00
Thibaut Girka acbed64f3d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/preferences_controller.rb
- app/lib/user_settings_decorator.rb
- app/models/user.rb
- config/locales/simple_form.en.yml
2019-06-26 23:19:22 +02:00
ThibG 42c803fe61 Apply filters to poll options (#11174)
* Apply filters to poll options in WebUI

Fixes #11128

* Apply filters to poll options server-side

* Add poll options to searchable text
2019-06-25 14:45:14 +02:00
Thibaut Girka 22439f9ded Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/lib/sanitize_config.rb
  Keep our version, we support the tags upstream transforms.
- package.json
- yarn.lock
2019-06-19 18:36:16 +02:00
Eugen Rochko ffbca47356 Fix sanitizer making block level elements unreadable (#10836)
Fix #10834
2019-06-16 21:46:36 +02:00
Thibaut Girka 00552d2f79 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
- app/controllers/stream_entries_controller.rb
2019-06-07 17:00:36 +02:00
ThibG be6e54a70c Do not misattribute inlined boosts if `attributedTo` isn't present (#10967)
* Do not misattribute inlined boosts if `attributedTo` isn't present

Fixes #10950

* Fix tests
2019-06-04 23:24:31 +02:00
Thibaut Girka 422dd06806 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-15 17:11:40 +02:00
ThibG f1a3135809 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
Thibaut Girka 1bf48b01e3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
  minor conflict because of glitch-soc's theming system
- app/controllers/stream_entries_controller.rb
  minor conflict because of glitch-soc's theming system
2019-05-10 17:09:12 +02:00
ThibG a571b07557 Prevent silenced local users from notifying remote users not following them (#10575)
* Prevent silenced local users from notifying remote users not following them

This is an attempt to extend the local restrictions of silenced users to the
federation.

* Add tests

* Add tests for making sure private status don't get sent over OStatus
2019-05-09 22:05:43 +02:00
Thibaut Girka 5a24934449 Merge branch 'master' into glitch-soc/merge-upstream 2019-04-10 21:19:21 +02:00
Eugen Rochko 6302bed0b5 Fix Keybase verification using wrong domain for remote accounts (#10547) 2019-04-10 20:28:43 +02:00
Thibaut Girka 0117de0a01 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/workers/activitypub/distribute_poll_update_worker.rb
- config/locales/pl.yml
2019-03-28 13:01:33 +01:00
Eugen Rochko 364e49ae1f Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
2019-03-28 04:44:59 +01:00
Eugen Rochko 367ab6764f Remove unused ActivityPub `@context` values depending on response (#10378)
Fix #8078
2019-03-27 15:55:23 +01:00
Thibaut Girka c37bc0b268 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/en.yml
  Conflict caused by the glitch-soc-specific “flavour” string being too close
  to the newly introduced “identity_proofs” string. Just included both.
2019-03-20 13:54:00 +01:00
Eugen Rochko a3c41f3532 Add Keybase integration (#10297)
* create account_identity_proofs table

* add endpoint for keybase to check local proofs

* add async task to update validity and liveness of proofs from keybase

* first pass keybase proof CRUD

* second pass keybase proof creation

* clean up proof list and add badges

* add avatar url to keybase api

* Always highlight the “Identity Proofs” navigation item when interacting with proofs.

* Update translations.

* Add profile URL.

* Reorder proofs.

* Add proofs to bio.

* Update settings/identity_proofs front-end.

* Use `link_to`.

* Only encode query params if they exist.

URLs without params had a trailing `?`.

* Only show live proofs.

* change valid to active in proof list and update liveness before displaying

* minor fixes

* add keybase config at well-known path

* extremely naive feature flagging off the identity proof UI

* fixes for rubocop

* make identity proofs page resilient to potential keybase issues

* normalize i18n

* tweaks for brakeman

* remove two unused translations

* cleanup and add more localizations

* make keybase_contacts an admin setting

* fix ExternalProofService my_domain

* use Addressable::URI in identity proofs

* use active model serializer for keybase proof config

* more cleanup of keybase proof config

* rename proof is_valid and is_live to proof_valid and proof_live

* cleanup

* assorted tweaks for more robust communication with keybase

* Clean up

* Small fixes

* Display verified identity identically to verified links

* Clean up unused CSS

* Add caching for Keybase avatar URLs

* Remove keybase_contacts setting
2019-03-18 21:00:55 +01:00
Thibaut Girka 365de06816 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/accounts_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/statuses_controller.rb

All conflicts caused by the additional `use_pack` used for glitch-soc's theming
system.
2019-03-18 18:03:27 +01:00
ThibG 9efb9d7f72 Set and store report URIs (#10303)
Fixes #10271
2019-03-17 15:34:56 +01:00
ThibG 3daa23c783 Fix reblogs privacy (#10302)
* Fix reblogs privacy

* Fix Announce processing specs
2019-03-17 14:54:09 +01:00
Thibaut Girka eed61a37ec Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/follower_domains_controller.rb
  Removed upstream. Did the same here. Maybe we should not have?
- config/locales/en.yml
  Upstream removed the “Authorized followers” page and associated
  translations. This is too close in the file to our glitch-soc-specific
  “flavour” string. No actual conflict.
- config/locales/ja.yml
  Same as above.
- config/locales/pl.yml
  Same as above.
- config/navigation.rb
  No real conflict. New route added too close to the glitch-soc-specific
  “flavours” one.
- config/webpack/configuration.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- config/webpack/loaders/babel.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
  The contents of this file have been moved to package.json.
- config/webpack/shared.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- config/webpacker.yml
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- jest.config.js
  The contents of this file have been moved to package.json.
- package.json
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- yarn.lock
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
2019-03-16 14:14:20 +01:00
Eugen Rochko 793587060b Fix language detection of non-latin alphabets even at few characters (#10276) 2019-03-15 05:07:09 +01:00
Thibaut Girka 9b093dfcde Merge branch 'master' into glitch-soc/merge-upstream 2019-03-08 00:57:13 +01:00
ThibG dcbe6c16ba Do not allow adding votes to expired polls (#10214)
* Do not allow adding votes to expired polls

* Only validate expires_at on create
2019-03-08 00:54:50 +01:00
Thibaut Girka c0b876f726 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- db/schema.rb

Both conflicts are caused by us having extra database columns.
2019-03-05 19:23:16 +01:00
Eugen Rochko 53336aa646 Add test ensuring that unknown object types are rejected (#10166) 2019-03-05 11:46:36 +09:00
ThibG 7da087dabe Store remote votes URI (#10158)
* Store remote votes URI

* Add spec for accepting remote votes

* Make poll vote id generation work the same way as follows
2019-03-04 22:51:23 +01:00
Eugen Rochko 03b5da7b6f Add tests for ActivityPub poll processing (#10143) 2019-03-04 01:13:42 +01:00
Thibaut Girka 561520a2be Merge branch 'master' into glitch-soc/merge-upstream 2019-02-17 15:43:12 +01:00
Eugen Rochko 6068b479f6 Fix Announce activities of unknown statuses not fetching those statuses (#10065)
Regression from #9998
2019-02-17 15:16:36 +01:00
Thibaut Girka da24c6aced Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/serializers/rest/instance_serializer.rb
- app/views/layouts/error.html.haml
- public/oops.png
  Took our version (same as upstream but without the things that only
  make sense in an animation).

Additional changes:
- app/javascript/flavours/vanilla/theme.yml
  Include upstream's javascript in error pages.
2019-02-17 12:26:55 +01:00
Eugen Rochko 397f180493 Add logging for rejected ActivityPub payloads and add tests (#10062) 2019-02-17 03:38:25 +01:00
Eugen Rochko 0230b3c41d Filter incoming Announce activities by relation to local activity (#10041)
* Filter incoming Announce activities by relation to local activity

Reject if announcer is not followed by local accounts, and is not
from an enabled relay, and the object is not a local status

Follow-up to #10005

* Fix tests
2019-02-15 18:19:45 +01:00
Thibaut Girka c8086b6efb Merge branch 'master' into glitch-soc/merge-upstream 2019-02-15 18:02:45 +01:00
ThibG 99dcb6d910 Alternative handling of private self-boosts (#9998)
* When self-boosting, embed original toot into Announce serialization

* Process unknown self-boosts from Announce object if it is more than an URI

* Add some self-boost specs

* Only serialize private toots in self-Announces
2019-02-13 18:36:23 +01:00
Thibaut Girka 1c37ad108c Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/oauth/authorized_applications_controller.rb
  Two changes too close to each other
- app/controllers/settings/sessions_controller.rb
- app/lib/user_settings_decorator.rb
  Two changes too close to each other
- app/models/media_attachment.rb
  New changes too close to glitch-soc only changes.
- app/models/user.rb
  Two changes too close to each other.
- app/services/remove_status_service.rb
  Kept direct timeline code which had been removed upstream.
- app/views/settings/preferences/show.html.haml
  Two changes too close to each other.
- config/locales/en.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/ja.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/pl.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/simple_form.en.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/locales/simple_form.pl.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/settings.yml
  Reverted upstream's decision of enabling posting application by default.
2019-02-10 21:10:09 +01:00
Eugen Rochko 5adc3f5676 Fix URL linkifier grabbing full-width spaces and quotations (#9997)
Fix #9993
Fix #5654
2019-02-09 20:13:11 +01:00
Hinaloe cce3319bc2 Only URLs extract with pre-escaped text (#9991)
* [test] add japanese hashtag testcase

* Only URLs extract with pre-escaped text

( https://github.com/tootsuite/mastodon/issues/9989 )
2019-02-09 03:39:38 +01:00
Jakub Mendyk 234e8105da Allow most kinds of characters in URL query (fixes #8408) (#8447)
* Allow unicode characters in URL query strings

Fixes #8408

* Alternative approach to unicode support in urls

Adds PoC/idea to approch this problem.
2019-02-02 19:01:18 +01:00
Thibaut Girka a78447a6a4 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .eslintrc.yml
  Removed, as upstream removed it.
- app/controllers/admin/statuses_controller.rb
  Minor code cleanup when porting one of our features.
- app/models/account.rb
  Note length validation has changed upstream.
  We now use upstream's validation (dropped legacy glitch-soc
  account metadata stuff) but with configurable limit.
- app/services/post_status_service.rb
  Upstream has added support for scheduled toots, refactoring
  the code a bit. Adapted our changes to this refactoring.
- app/views/stream_entries/_detailed_status.html.haml
  Not a real conflict, changes too close.
- app/views/stream_entries/_simple_status.html.haml
  Not a real conflict, changes too close.
2019-01-10 21:00:30 +01:00
Thibaut Girka 8d47495251 Revert "Revert "Add handler for Move activity (#9629)""
This reverts commit 6ab727bf6a.
2019-01-10 18:46:17 +01:00
Eugen Rochko ae1aaa3b8a Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
Thibaut Girka 6ab727bf6a Revert "Add handler for Move activity (#9629)"
This reverts commit 4e3131f02f.
2019-01-02 15:36:59 +01:00
Thibaut Girka 44f2224606 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts manually resolved:
- app/services/post_status_service.rb
- config/locales/simple_form.pl.yml
- config/routes.rb
- config/webpack/loaders/sass.js
- config/webpack/shared.js
- package.json
- yarn.lock
2019-01-02 15:36:53 +01:00
ThibG ffcaa4719a Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order (#9687)
* Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order

* Add specs for Block causing unfollow and for out-of-order Block + Undo
2019-01-02 01:12:02 +01:00
Eugen Rochko 4e3131f02f Add handler for Move activity (#9629) 2018-12-29 02:24:36 +01:00
Thibaut Girka 6be659a5ba Add specs for list replies policy 2018-11-28 14:46:07 +01:00
Thibaut Girka 478abe3a58 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb

Resolved by taking both changes (not a real conflict, just changes too close
to each other).
2018-11-27 13:23:02 +01:00
Eugen Rochko f3fee3cd27 Fix connect timeout not being enforced (#9329)
* Fix connect timeout not being enforced

The loop was catching the timeout exception that should stop execution, so the next IP would no longer be within a timed block, which led to requests taking much longer than 10 seconds.

* Use timeout on each IP attempt, but limit to 2 attempts

* Fix code style issue

* Do not break Request#perform if no block given

* Update method stub in spec for Request

* Move timeout inside the begin/rescue block

* Use Resolv::DNS with timeout of 1 to get IP addresses

* Update Request spec to stub Resolv::DNS instead of Addrinfo

* Fix Resolve::DNS stubs in Request spec
2018-11-22 20:12:04 +01:00
Thibaut Girka bc97c2ab8a Merge branch 'master' into glitch-soc/merge-upstream 2018-10-29 14:09:54 +01:00
Eugen Rochko eaf49c5d94 Bump sanitize from 4.6.6 to 5.0.0 (#9140) 2018-10-29 14:05:25 +01:00
Thibaut Girka 46259a36d0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .github/ISSUE_TEMPLATE/bug_report.md
  Took our version.
- CONTRIBUTING.md
  Updated the embedded copy of upstream's version.
- README.md
  Took our version.
- app/policies/status_policy.rb
  Not a real conflict, took code from both.
- app/views/layouts/embedded.html.haml
  Added upstream's changes (dns-prefetch) and fixed
  `%body.embed`
- app/views/settings/preferences/show.html.haml
  Reverted some of upstream changes, as we have a
  page dedicated for flavours and skins.
- config/initializers/content_security_policy.rb
  Kept our version of the CSP.
- config/initializers/doorkeeper.rb
  Not a real conflict, took code from both.
2018-10-22 17:51:38 +02:00
Eugen Rochko 654520ec8c Improve support for aspects/circles (#8950)
* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification
2018-10-17 17:13:04 +02:00
Thibaut Girka 74e411f4e8 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-08 13:51:33 +02:00
ashleyhull-versent 00d31a292c rubocop issues - Cleaning up (#8912)
* cleanup pass

* undo mistakes

* fixed.

* revert
2018-10-08 04:50:11 +02:00
Thibaut Girka 463f250f26 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-05 21:48:25 +02:00
Jeong Arm 708c12af1e Leave unknown language as nil if account is remote (#8861)
* Force use language detector if account is remote

* Set unknown remote toot's language as nil
2018-10-05 19:17:46 +02:00
Thibaut Girka f627ea99e4 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	db/migrate/20170716191202_add_hide_notifications_to_mute.rb
	spec/controllers/application_controller_spec.rb

Took our version, upstream changes were only minor style linting.
2018-10-05 15:23:57 +02:00
aus-social a53bcb6213 Lint pass (#8876) 2018-10-04 12:36:53 +02:00
Thibaut Girka daa6110ce7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/stream_entries/_simple_status.html.haml
- config/locales/nl.yml

Deleted unused translation strings (themes) and adapted minor changes to
_simple_status.html.haml
2018-08-31 19:10:57 +02:00
Renato "Lond" Cerqueira d516e7fa62 Fix autoplay issue with spoiler tag (#8540)
Add tests to avoid similar issues in the future
2018-08-31 15:16:59 +02:00
Thibaut Girka ac4e67e535 Merge branch 'master' into glitch-soc/merge-upstream 2018-08-29 13:57:56 +02:00
sundevour 45af08148d formatter spec fixes & clarification (#8481)
updates some "context" and "it" lines to have clearer explanations
updates "context" lines to properly describe function input, and "it" lines to describe results
2018-08-29 01:20:56 +02:00
Thibaut Girka faeb568601 Merge branch 'master' into glitch-soc/merge-upstream 2018-08-28 17:24:39 +02:00
Eugen Rochko f507c9713f Display pending message on admin relays UI (#8494)
* Add missing specs for relay accept/reject

* Display pending message on admin relays UI
2018-08-28 05:39:43 +02:00
Thibaut Girka 334f478db1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/models/status.rb
	db/migrate/20180528141303_fix_accounts_unique_index.rb
	db/schema.rb

Resolved by taking upstream changes (no real conflicts, just glitch-soc
specific code too close to actual changes).
2018-08-17 17:43:54 +02:00
ThibG 20ced948f2 Implement Undo { Accept { Follow } } (fixes #8234) (#8245)
* Add Follow#revoke_request!

* Implement Undo { Accept { Follow } } (fixes #8234)
2018-08-17 16:24:56 +02:00
Thibaut Girka 4ec907e413 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	Dockerfile
	app/javascript/packs/common.js
	config/webpack/loaders/sass.js
	config/webpack/shared.js
	db/schema.rb
	package.json
	yarn.lock

A lot of the conflicts come from updating webpack.

Even though upstream deleted app/javascript/packs/common.js, I kept
glitch-soc's version as it unifies JS/CSS packs behavior across flavours.

Ported glitch changes to webpack 4.x
2018-07-16 14:50:42 +02:00
Eugen Rochko 64ce00c9c5 Disable language detection for texts shorter than 140 characters (#8010)
If the input text is blank after preparation (only mention, or
only URL, or empty as in a media post), then use nil as language,
since it's OK to show to everyone.

Otherwise, always fall back to the server's default locale
2018-07-14 04:05:36 +02:00
Thibaut Girka 262f7d147d Migrate glitch-soc keyword mutes to Mastodon's
Completely remove glitch-soc's Keyword Mutes, migrate
existing database records to CustomFilters.

Handling of client-side filters is still not implemented
in the glitch-soc front-end.
2018-07-09 19:19:23 +02:00
Thibaut Girka cf8121376b Merge branch 'master' into glitch-soc/tentative-merge
Conflicts:
	README.md
	app/controllers/statuses_controller.rb
	app/lib/feed_manager.rb
	config/navigation.rb
	spec/lib/feed_manager_spec.rb

Conflicts were resolved by taking both versions for each change.
This means the two filter systems (glitch-soc's keyword mutes and tootsuite's
custom filters) are in place, which will be changed in a follow-up commit.
2018-07-09 07:13:59 +02:00
ThibG 4c70bb6d6d Add option to not consider word boundaries when processing keyword filtering (#7975)
* Add option to not consider word boundaries when filtering phrases

* Add a few tests for keyword/phrase filtering
2018-07-09 02:22:09 +02:00
Eugen Rochko d878e3e945 Keyword/phrase filtering (#7905)
* Add keyword filtering

    GET|POST       /api/v1/filters
    GET|PUT|DELETE /api/v1/filters/:id

- Irreversible filters can drop toots from home or notifications
- Other filters can hide toots through the client app
- Filters use a phrase valid in particular contexts, expiration

* Make sure expired filters don't get applied client-side

* Add missing API methods

* Remove "regex filter" from column settings

* Add tests

* Add test for FeedManager

* Add CustomFilter test

* Add UI for managing filters

* Add streaming API event to allow syncing filters

* Fix tests
2018-06-29 15:34:36 +02:00
David Yip b8205550dd keyword mute: use mentions scope in home feed filtering (#454)
If a status shows up in mentions because all keyword mutes that might
apply to it are marked as "don't apply to mentions", then it ought to
show up in the home feed also.
2018-06-12 17:14:35 -05:00
David Yip f9c8642a11 Merge remote-tracking branch 'glitchsoc/master' into 454-allow-keyword-mutes-to-skip-mentions
Conflicts:
 	app/models/glitch/keyword_mute.rb
2018-06-12 16:39:30 -05:00
David Yip c2f7676dfc Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
 	app/models/status.rb

The conflict in the Status model was due to
3114ac9655.
It was resolved by accepting tootsuite's changes.
2018-06-07 05:13:49 -05:00
David Yip 526f2f8e09 Add a FeedManager example demonstrating non-mention keywords. #454. 2018-06-03 23:04:00 -05:00
David Yip a000085b9c Fix spacing in some FeedManager examples. 2018-06-03 23:02:01 -05:00
Shuhei Kitagawa 6dfc8999f7 Add tests for embeds controller (#7719)
* Small refactoring of status_finder_spec

* Add tests for embeds_controller
2018-06-04 10:35:56 +09:00
David Yip 3c3e6d0e9b Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
 	app/controllers/settings/preferences_controller.rb
 	app/lib/user_settings_decorator.rb
 	app/models/user.rb
 	config/locales/simple_form.en.yml
2018-05-18 08:47:10 -05:00
Eugen Rochko 1de7400dfc Resolve unknown status from Add activity, skip Remove if unknown (#7526)
Fix #7518
2018-05-18 11:33:56 +02:00
David Yip c87f1d99e4 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	.travis.yml
 	Gemfile.lock
 	README.md
 	app/controllers/settings/follower_domains_controller.rb
 	app/controllers/statuses_controller.rb
 	app/javascript/mastodon/locales/ja.json
 	app/lib/feed_manager.rb
 	app/models/media_attachment.rb
 	app/models/mute.rb
 	app/models/status.rb
 	app/services/mute_service.rb
 	app/views/home/index.html.haml
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/ca.yml
 	config/locales/en.yml
 	config/locales/es.yml
 	config/locales/fr.yml
 	config/locales/nl.yml
 	config/locales/pl.yml
 	config/locales/pt-BR.yml
 	config/themes.yml
2018-05-03 17:23:44 -05:00
Eugen Rochko ca1c696dbd Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
MIYAGI Hikaru 50e673409f Append '.test' to hostname in stub data (#7260) 2018-04-25 14:12:28 +02:00
Eugen Rochko 599aa39aca Remove "nsfw" category for sensitive statuses in OStatus serializer (#7048)
Fix #7011
2018-04-22 22:09:03 +02:00
David Yip 337fc136fd Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/serializers/initial_state_serializer.rb

The glitch flavour isn't yet pulling custom emoji data on its own (see
https://github.com/tootsuite/mastodon/pull/7047).  Once that gets into
the glitch flavour, we can eliminate the custom_emojis load.
2018-04-08 19:05:02 -05:00
Levi Bard 6828abc1d7 Enable updating additional account information from user preferences via rest api (#6789)
* Enable updating additional account information from user preferences via rest api
Resolves #6553

* Pacify rubocop

* Decoerce incoming settings in UserSettingsDecorator

* Create user preferences hash directly from incoming credentials instead of going through ActionController::Parameters

* Clean up user preferences update

* Use ActiveModel::Type::Boolean instead of manually checking stringified number equivalence
2018-04-08 13:43:10 +02:00
David Yip 21394c5462 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/mastodon/locales/en.json
 	app/javascript/mastodon/locales/ja.json
 	app/javascript/mastodon/locales/pl.json
 	app/views/accounts/_header.html.haml
2018-04-01 20:24:50 -05:00
David Underwood 44274257de [WIP] Enable custom emoji on account pages and in the sidebar (#6124)
Federate custom emojis with accounts
2018-04-01 23:55:42 +02:00
David Yip 7cdf2b05b6 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/styles/mastodon/components.scss
 	app/models/media_attachment.rb
2018-03-27 10:26:47 -05:00
Akihiko Odaki acece7a2e6 Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Jenkins ae55717f50 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-25 15:17:21 +00:00
Akihiko Odaki 11c19f6cc9 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Jenkins b1453c0dbf Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-20 19:17:20 +00:00
Rey Tucker 4c91be94d3 request: in the event of failure, try other IPs (#6761) (#6813)
* request: in the event of failure, try other IPs (#6761)

In the case where a name has multiple A/AAAA records, we should
try subsequent records instead of immediately failing when we have a
failure on the first IP address.

This significantly improves delivery success when there are network
connectivity problems affecting only IPv4 or IPv6.

* fix method call style

* request_spec: adjust test case to use Addrinfo

* request: Request/open: move private addr check to within begin/rescue

* request_spec: add case to test failover, fix exception check

* Double Addrinfo.foreach so that it correctly yields instances
2018-03-20 09:06:08 +01:00